Knowledge-grounded conversational models are known to suffer from producing factually invalid statements, a phenomenon commonly called hallucination. In this work, we investigate the underlying causes of this phenomenon: is hallucination due to the training data, or to the models? We conduct a comprehensive human study on both existing knowledge-grounded conversational benchmarks and several state-of-the-art models. Our study reveals that the standard benchmarks consist of >60% hallucinated responses, leading to models that not only hallucinate but even amplify hallucinations. Our findings raise important questions on the quality of existing datasets and models trained using them. We make our annotations publicly available for future research.

翻译：据知,基于知识的谈话模式会因产生事实上无效的言论而受到损害,这是一种通常称为幻觉的现象。在这项工作中,我们调查了这一现象的根本原因:是由于培训数据或模型造成的幻觉?我们对现有基于知识的谈话基准和一些最先进的模型进行了全面的人类研究。我们的研究显示,标准基准包含大于60%的致幻反应,不仅导致幻觉,甚至放大幻觉。我们的调查结果提出了关于现有数据集和使用这些数据集所培训模型的质量的重要问题。我们公布我们的说明供今后的研究使用。