The applicability of Large Language Models (LLMs) in temporal reasoning tasks over data that is not present during training is still a field that remains to be explored. In this paper we work on this topic, focusing on structured and semi-structured anonymized data. We not only develop a direct LLM pipeline, but also compare various methodologies and conduct an in-depth analysis. We identified and examined seventeen common temporal reasoning tasks in natural language, focusing on their algorithmic components. To assess LLM performance, we created the \textit{Reasoning and Answering Temporal Ability} dataset (RATA), featuring semi-structured anonymized data to ensure reliance on reasoning rather than on prior knowledge. We compared several methodologies, involving SoTA techniques such as Tree-of-Thought, self-reflexion and code execution, tuned specifically for this scenario. Our results suggest that achieving scalable and reliable solutions requires more than just standalone LLMs, highlighting the need for integrated approaches.

翻译：大型语言模型（LLMs）在训练数据中未出现的时间推理任务上的适用性仍是一个有待探索的领域。本文聚焦于结构化与半结构化的匿名化数据，对此主题展开研究。我们不仅开发了直接的LLM处理流程，还比较了多种方法并进行了深入分析。我们识别并考察了自然语言中十七种常见的时序推理任务，重点关注其算法构成。为评估LLM性能，我们构建了《时序推理与回答能力》数据集（RATA），该数据集采用半结构化匿名化数据，以确保模型依赖推理而非先验知识。我们比较了多种方法，包括针对此场景专门调整的先进技术，如思维树、自反思和代码执行。研究结果表明，实现可扩展且可靠的解决方案仅靠独立的LLMs是不够的，这凸显了集成化方法的必要性。