A scene graph is a structured representation of objects and their spatio-temporal relationships in dynamic scenes. Scene Graph Anticipation (SGA) involves predicting future scene graphs from video clips, enabling applications in intelligent surveillance and human-machine collaboration. While recent SGA approaches excel at leveraging visual evidence, long-horizon forecasting fundamentally depends on semantic priors and commonsense temporal regularities that are challenging to extract purely from visual features. To explicitly model these semantic dynamics, we propose Linguistic Scene Graph Anticipation (LSGA), a linguistic formulation of SGA that performs temporal relational reasoning over sequences of textualized scene graphs, with visual scene-graph detection handled by a modular front-end when operating on video. Building on this formulation, we introduce Object-Oriented Two-Stage Method (OOTSM), a language-based framework that anticipates object-set dynamics and forecasts object-centric relation trajectories with temporal consistency regularization, and we evaluate it on a dedicated benchmark constructed from Action Genome annotations. Extensive experiments show that compact fine-tuned language models with up to 3B parameters consistently outperform strong zero- and one-shot API baselines, including GPT-4o, GPT-4o-mini, and DeepSeek-V3, under matched textual inputs and context windows. When coupled with off-the-shelf visual scene-graph generators, the resulting multimodal system achieves substantial improvements on video-based SGA, boosting long-horizon mR@50 by up to 21.9\% over strong visual SGA baselines.

翻译：场景图是动态场景中对象及其时空关系的结构化表示。场景图预测（SGA）旨在从视频片段中预测未来场景图，可应用于智能监控和人机协作领域。尽管近期SGA方法在利用视觉证据方面表现优异，但长时程预测本质上依赖于语义先验和常识性时间规律，这些信息仅从视觉特征中提取具有挑战性。为显式建模这些语义动态，我们提出语言化场景图预测（LSGA），这是一种SGA的语言化表述方法，通过对文本化场景图序列进行时序关系推理来实现预测，在视频处理时通过模块化前端完成视觉场景图检测。基于此表述，我们引入对象导向两阶段方法（OOTSM），这是一个基于语言的框架，能够预测对象集合动态并以前后一致的时间正则化方式预测以对象为中心的关系轨迹，我们在基于Action Genome标注构建的专用基准上对其进行了评估。大量实验表明，在文本输入和上下文窗口匹配的条件下，参数规模达30亿的紧凑型微调语言模型持续优于包括GPT-4o、GPT-4o-mini和DeepSeek-V3在内的强零样本与单样本API基线。当结合现成的视觉场景图生成器时，所构建的多模态系统在视频SGA任务上取得显著提升，长时程mR@50指标较强视觉SGA基线最高提升21.9%。