Conventional, classification-based AI-generated image detection methods cannot explain why an image is considered real or AI-generated in a way a human expert would, which reduces the trustworthiness and persuasiveness of these detection tools for real-world applications. Leveraging Multimodal Large Language Models (MLLMs) has recently become a trending solution to this issue. Further, to evaluate the quality of generated explanations, a common approach is to adopt an "MLLM as a judge" methodology to evaluate explanations generated by other MLLMs. However, how well those MLLMs perform when judging explanations for AI-generated image detection generated by themselves or other MLLMs has not been well studied. We therefore propose \textbf{XAIGID-RewardBench}, the first benchmark designed to evaluate the ability of current MLLMs to judge the quality of explanations about whether an image is real or AI-generated. The benchmark consists of approximately 3,000 annotated triplets sourced from various image generation models and MLLMs as policy models (detectors) to assess the capabilities of current MLLMs as reward models (judges). Our results show that the current best reward model scored 88.76\% on this benchmark (while human inter-annotator agreement reaches 98.30\%), demonstrating that a visible gap remains between the reasoning abilities of today's MLLMs and human-level performance. In addition, we provide an analysis of common pitfalls that these models frequently encounter. Code and benchmark are available at https://github.com/RewardBench/XAIGID-RewardBench.

翻译：传统的基于分类的AI生成图像检测方法无法以人类专家可理解的方式解释为何将图像判定为真实或AI生成，这降低了此类检测工具在实际应用中的可信度与说服力。近年来，利用多模态大语言模型（MLLMs）已成为解决该问题的趋势性方案。进一步地，为评估生成解释的质量，常见方法是采用“MLLM作为评判者”的方法论来评估其他MLLM生成的解释。然而，当这些MLLMs评判自身或其他MLLMs生成的AI生成图像检测解释时，其表现尚未得到充分研究。为此，我们提出\\textbf{XAIGID-RewardBench}——首个旨在评估当前MLLMs评判图像真伪解释质量的基准。该基准包含约3,000个标注三元组，数据源自多种图像生成模型及作为策略模型（检测器）的MLLMs，用于评估当前MLLMs作为奖励模型（评判者）的能力。实验结果表明，当前最佳奖励模型在该基准上获得88.76%的分数（而人工标注者间一致性达98.30%），表明当今MLLMs的推理能力与人类水平仍存在明显差距。此外，我们分析了这些模型常见的缺陷类型。代码与基准数据发布于https://github.com/RewardBench/XAIGID-RewardBench。