Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no control over their presentation. Through a systematic algorithm audit of 1,508 real baby care and pregnancy-related queries, we evaluate the quality and consistency of these information displays. Our robust evaluation framework assesses multiple quality dimensions, including answer consistency, relevance, presence of medical safeguards, source categories, and sentiment alignment. Our results reveal concerning gaps in information consistency, with information in AIO and FS displayed on the same search result page being inconsistent with each other in 33% of cases. Despite high relevance scores, both features critically lack medical safeguards (present in just 11% of AIO and 7% of FS responses). While health and wellness websites dominate source categories for both, AIO and FS, FS also often link to commercial sources. These findings have important implications for public health information access and demonstrate the need for stronger quality controls in AI-mediated health information. Our methodology provides a transferable framework for auditing AI systems across high-stakes domains where information quality directly impacts user well-being.


翻译:谷歌搜索日益通过AI概览与精选摘要等功能呈现AI生成内容,用户虽无法控制其呈现方式,却常依赖此类信息。本研究通过对1,508条真实婴儿护理与孕期相关查询进行系统性算法审计,评估了这些信息展示的质量与一致性。我们构建的稳健评估框架从多维度考察信息质量,包括答案一致性、相关性、医疗安全警示的存在性、来源类别及情感倾向对齐。结果显示信息一致性存在显著缺陷:同一搜索结果页中,AI概览与精选摘要的信息不一致率高达33%。尽管两者相关性评分较高,但均严重缺乏医疗安全警示(仅11%的AI概览与7%的精选摘要包含此类警示)。健康类网站在两者来源中占主导地位,但精选摘要同时频繁链接至商业来源。这些发现对公共卫生信息获取具有重要启示,表明AI介导的健康信息需加强质量控制。本方法论为审计高风险领域(信息质量直接影响用户福祉)的AI系统提供了可迁移的评估框架。

0
下载
关闭预览

相关内容

人工智能杂志AI(Artificial Intelligence)是目前公认的发表该领域最新研究成果的主要国际论坛。该期刊欢迎有关AI广泛方面的论文,这些论文构成了整个领域的进步,也欢迎介绍人工智能应用的论文,但重点应该放在新的和新颖的人工智能方法如何提高应用领域的性能,而不是介绍传统人工智能方法的另一个应用。关于应用的论文应该描述一个原则性的解决方案,强调其新颖性,并对正在开发的人工智能技术进行深入的评估。 官网地址:http://dblp.uni-trier.de/db/journals/ai/
论文浅尝 | Interaction Embeddings for Prediction and Explanation
开放知识图谱
11+阅读 · 2019年2月1日
国家自然科学基金
2+阅读 · 2014年12月31日
VIP会员
Top
微信扫码咨询专知VIP会员