Recent Large Audio-Language Models (LALMs) exhibit impressive capabilities in understanding audio content for conversational QA tasks. However, these models struggle to accurately understand timestamps for temporal localization (e.g., Temporal Audio Grounding) and are restricted to short audio perception, leading to constrained capabilities on fine-grained tasks. We identify three key aspects that limit their temporal localization and long audio understanding: (i) timestamp representation, (ii) architecture, and (iii) data. To address this, we introduce TimeAudio, a novel method that empowers LALMs to connect their understanding of audio content with precise temporal perception. Specifically, we incorporate unique temporal markers to improve time-sensitive reasoning and apply an absolute time-aware encoding that explicitly grounds the acoustic features with absolute time information. Moreover, to achieve end-to-end long audio understanding, we introduce a segment-level token merging module to substantially reduce audio token redundancy and enhance the efficiency of information extraction. Due to the lack of suitable datasets and evaluation metrics, we consolidate existing audio datasets into a new dataset focused on temporal tasks and establish a series of metrics to evaluate the fine-grained performance. Evaluations show strong performance across a variety of fine-grained tasks, such as dense captioning, temporal grounding, and timeline speech summarization, demonstrating TimeAudio's robust temporal localization and reasoning capabilities.

翻译：近期的大型音频-语言模型在理解音频内容以完成对话式问答任务方面展现出卓越能力。然而，这些模型在准确理解时间戳以实现时序定位（如时序音频定位）方面存在困难，且受限于短音频感知能力，导致在细粒度任务上的性能受限。我们识别出制约其时序定位与长音频理解能力的三个关键因素：（i）时间戳表示，（ii）模型架构，以及（iii）训练数据。为此，我们提出TimeAudio——一种创新方法，使大型音频-语言模型能够将音频内容理解与精确时序感知相融合。具体而言，我们引入独特的时间标记以增强时间敏感性推理，并采用绝对时间感知编码技术，将声学特征显式关联至绝对时间信息。此外，为实现端到端的长音频理解，我们设计了段级令牌合并模块，显著降低音频令牌冗余并提升信息提取效率。针对现有数据集与评估指标的不足，我们整合多个音频数据集构建了专注于时序任务的新数据集，并建立了一套评估细粒度性能的指标体系。实验结果表明，该方法在密集描述生成、时序定位及时间线语音摘要等多种细粒度任务中均表现优异，充分验证了TimeAudio在时序定位与推理方面的鲁棒性。