Recent advances in Video Large Language Models (VLLMs) have achieved remarkable video understanding capabilities, yet face critical efficiency bottlenecks due to quadratic computational growth with lengthy visual token sequences of long videos. While existing keyframe sampling methods can improve temporal modeling efficiency, additional computational cost is introduced before feature encoding, and the binary frame selection paradigm is found suboptimal. Therefore, in this work, we propose Dynamic Token compression via LLM-guided Keyframe prior (DyToK), a training-free paradigm that enables dynamic token compression by harnessing VLLMs' inherent attention mechanisms. Our analysis reveals that VLLM attention layers naturally encoding query-conditioned keyframe priors, by which DyToK dynamically adjusts per-frame token retention ratios, prioritizing semantically rich frames while suppressing redundancies. Extensive experiments demonstrate that DyToK achieves state-of-the-art efficiency-accuracy tradeoffs. DyToK shows plug-and-play compatibility with existing compression methods, such as VisionZip and FastV, attaining 4.3x faster inference while preserving accuracy across multiple VLLMs, such as LLaVA-OneVision and Qwen2.5-VL. Code is available at https://github.com/yu-lin-li/DyToK .

翻译：视频大语言模型（VLLMs）的最新进展已实现卓越的视频理解能力，但由于长视频视觉令牌序列长度导致的二次计算增长，其面临关键效率瓶颈。现有关键帧采样方法虽能提升时序建模效率，但会在特征编码前引入额外计算成本，且二元帧选择范式被证明并非最优。为此，本研究提出基于LLM引导关键帧先验的动态令牌压缩（DyToK），这是一种无需训练的范式，通过利用VLLMs固有的注意力机制实现动态令牌压缩。我们的分析表明，VLLM注意力层天然编码查询条件化的关键帧先验，DyToK据此动态调整每帧令牌保留比例，优先处理语义丰富的帧并抑制冗余。大量实验证明，DyToK实现了最优的效率-精度权衡。DyToK展现出与现有压缩方法（如VisionZip和FastV）即插即用的兼容性，在LLaVA-OneVision和Qwen2.5-VL等多种VLLMs上实现4.3倍推理加速同时保持精度。代码发布于https://github.com/yu-lin-li/DyToK。