Discovering scaling laws for predicting model performance at scale is a fundamental and open-ended challenge, mostly reliant on slow, case specific human experimentation. To investigate the potential for LLMs to automate this process, we collect over 5,000 experiments from existing literature and curate seven diverse scaling law discovery tasks. While existing agents struggle to produce accurate law formulas, this paper introduces SLDAgent, an evolution-based agent that co-optimize the scaling law model and the parameters, enabling it to autonomously explore complex relationships between variables. For the first time, we demonstrates that SLDAgent can automatically discover laws that exhibit consistently more accurate extrapolation than their established, human-derived counterparts across all tasks. Through comprehensive analysis, we elucidate why these discovered laws are superior and verify their practical utility in both pretraining and finetuning applications. This work establishes a new paradigm for agentic scientific discovery, showing that AI systems can understand their own scaling behavior, and can contribute novel and practical knowledge back to the research community.

翻译：发现用于预测大规模模型性能的缩放定律是一项基础且开放性的挑战，目前主要依赖于缓慢、针对特定案例的人工实验。为探究大型语言模型（LLMs）自动化此过程的潜力，我们从现有文献中收集了超过5,000项实验数据，并构建了七个多样化的缩放定律发现任务。尽管现有智能体难以生成精确的定律公式，本文提出了SLDAgent——一种基于进化的智能体，能够协同优化缩放定律模型及其参数，从而自主探索变量间的复杂关系。我们首次证明，SLDAgent能够自动发现在所有任务中均比现有、人工推导的对应定律展现出更准确外推能力的定律。通过全面分析，我们阐明了这些发现定律的优越性，并在预训练与微调应用中验证了其实际效用。这项工作确立了智能体驱动科学发现的新范式，表明人工智能系统能够理解其自身的缩放行为，并能向研究社区贡献新颖且实用的知识。