We study the ability of language models to reason about appropriate information disclosure - a central aspect of the evolving field of agentic privacy. Whereas previous works have focused on evaluating a model's ability to align with human decisions, we examine the role of ambiguity and missing context on model performance when making information-sharing decisions. We identify context ambiguity as a crucial barrier for high performance in privacy assessments. By designing Camber, a framework for context disambiguation, we show that model-generated decision rationales can reveal ambiguities and that systematically disambiguating context based on these rationales leads to significant accuracy improvements (up to 13.3% in precision and up to 22.3% in recall) as well as reductions in prompt sensitivity. Overall, our results indicate that approaches for context disambiguation are a promising way forward to enhance agentic privacy reasoning.
翻译:本研究探讨语言模型在判断信息适当披露方面的推理能力——这是代理隐私这一新兴领域的核心问题。与先前研究聚焦于模型决策与人类判断的一致性不同,我们重点考察情境模糊性与语境缺失对模型信息共享决策的影响。我们发现情境模糊性是制约隐私评估性能提升的关键障碍。通过构建Camber框架——一种情境消歧系统,我们证明模型生成的决策依据能够有效揭示模糊信息,且基于这些依据的系统性情境消歧可显著提升决策准确率(精确度最高提升13.3%,召回率最高提升22.3%)并降低提示敏感性。总体而言,研究结果表明情境消歧方法为增强代理隐私推理能力提供了可行路径。