Large Video-Language Models (Video-LMs) have achieved impressive progress in multimodal understanding, yet their reasoning remains weakly grounded in space and time. We present Know-Show, a new benchmark designed to evaluate spatio-temporal grounded reasoning, the ability of a model to reason about actions and their semantics while simultaneously grounding its inferences in visual and temporal evidence. Know-Show unifies reasoning and localization within a single evaluation framework consisting of five complementary scenarios across spatial (person, object, person-object, and hand-object) and temporal dimensions. Built from Charades, Action Genome, and Ego4D with 2.5K human-authored questions, the benchmark exposes significant gaps between current Video-LMs and human reasoning. To bridge this gap, we propose GRAM, a training-free plug-in that augments Video-LMs with fine-grained grounding through attention-based video token selection and explicit timestamp encoding. Extensive experiments across open and closed Video-LMs (Qwen, VideoLLaVA, GPT-4o, and Gemini, etc.) reveal that existing models struggle to "show what they know" and vice versa, especially in fine-grained hand-object interactions. Know-Show establishes a unified standard for assessing grounded reasoning in video-language understanding and provides insights toward developing interpretable and reliable multimodal reasoning systems. We will release the code at https://github.com/LUNAProject22/Know-Show.

翻译：大型视频语言模型（Video-LMs）在多模态理解方面取得了显著进展，但其推理在空间和时间维度上的基础仍较为薄弱。我们提出了Know-Show，这是一个旨在评估时空基础推理能力的新基准，即模型在推理动作及其语义的同时，将其推断基于视觉和时间证据的能力。Know-Show将推理与定位统一在一个评估框架内，包含跨越空间（人物、物体、人物-物体和手-物体）和时间维度的五个互补场景。该基准基于Charades、Action Genome和Ego4D构建，包含2.5K个人工编写的问题，揭示了当前Video-LMs与人类推理之间的显著差距。为弥合这一差距，我们提出了GRAM，一种无需训练的插件，通过基于注意力的视频令牌选择和显式时间戳编码，增强Video-LMs的细粒度基础能力。在开放和封闭的Video-LMs（如Qwen、VideoLLaVA、GPT-4o和Gemini等）上的大量实验表明，现有模型难以“展示其所知”，反之亦然，尤其是在细粒度的手-物体交互中。Know-Show为评估视频语言理解中的基础推理建立了统一标准，并为开发可解释且可靠的多模态推理系统提供了洞见。代码将在https://github.com/LUNAProject22/Know-Show发布。