Recent advances in multi-modal Large Language Models (M-LLMs) have demonstrated a powerful ability to synthesize implicit information from disparate sources, including images and text. These resourceful data from social media also introduce a significant and underexplored privacy risk: the inference of sensitive personal attributes from seemingly daily media content. However, the lack of benchmarks and comprehensive evaluations of state-of-the-art M-LLM capabilities hinders the research of private attribute profiling on social media. Accordingly, we propose (1) PRISM, the first multi-modal, multi-dimensional and fine-grained synthesized dataset incorporating a comprehensive privacy landscape and dynamic user history; (2) an Efficient evaluation framework that measures the cross-modal privacy inference capabilities of advanced M-LLM. Specifically, PRISM is a large-scale synthetic benchmark designed to evaluate cross-modal privacy risks. Its key feature is 12 sensitive attribute labels across a diverse set of multi-modal profiles, which enables targeted privacy analysis. These profiles are generated via a sophisticated LLM agentic workflow, governed by a prior distribution to ensure they realistically mimic social media users. Additionally, we propose a Multi-Agent Inference Framework that leverages a pipeline of specialized LLMs to enhance evaluation capabilities. We evaluate the inference capabilities of six leading M-LLMs (Qwen, Gemini, GPT-4o, GLM, Doubao, and Grok) on PRISM. The comparison with human performance reveals that these MLLMs significantly outperform in accuracy and efficiency, highlighting the threat of potential privacy risks and the urgent need for robust defenses. Dataset available at https://huggingface.co/datasets/xaddh/multimodal-privacy

翻译：多模态大语言模型（M-LLMs）的最新进展展现出了从图像和文本等不同来源综合隐含信息的强大能力。这些来自社交媒体的丰富数据也引入了一个重要且尚未充分探索的隐私风险：从看似日常的媒体内容中推断出敏感的个人属性。然而，缺乏对最先进M-LLM能力的基准测试和全面评估，阻碍了社交媒体上私人属性画像的研究。为此，我们提出了（1）PRISM，这是首个融合了全面隐私场景和动态用户历史的多模态、多维度、细粒度合成数据集；（2）一个高效的评估框架，用于衡量先进M-LLM的跨模态隐私推断能力。具体而言，PRISM是一个旨在评估跨模态隐私风险的大规模合成基准。其关键特征是在一组多样化的多模态用户画像上标注了12个敏感属性标签，从而支持有针对性的隐私分析。这些用户画像通过一个复杂的LLM智能体工作流程生成，并受先验分布控制，以确保它们能真实地模拟社交媒体用户。此外，我们提出了一个多智能体推断框架，该框架利用一系列专业LLM的流水线来增强评估能力。我们在PRISM上评估了六种领先的M-LLM（Qwen、Gemini、GPT-4o、GLM、Doubao和Grok）的推断能力。与人类表现的对比表明，这些M-LLM在准确性和效率上显著超越，突显了潜在的隐私风险威胁以及对强大防御措施的迫切需求。数据集可在 https://huggingface.co/datasets/xaddh/multimodal-privacy 获取。