Dataset distillation (DD) aims to construct compact synthetic datasets that allow models to achieve comparable performance to full-data training while substantially reducing storage and computation. Despite rapid empirical progress, its theoretical foundations remain limited: existing methods (gradient, distribution, trajectory matching) are built on heterogeneous surrogate objectives and optimization assumptions, which makes it difficult to analyze their common principles or provide general guarantees. Moreover, it is still unclear under what conditions distilled data can retain the effectiveness of full datasets when the training configuration, such as optimizer, architecture, or augmentation, changes. To answer these questions, we propose a unified theoretical framework, termed configuration--dynamics--error analysis, which reformulates major DD approaches under a common generalization-error perspective and provides two main results: (i) a scaling law that provides a single-configuration upper bound, characterizing how the error decreases as the distilled sample size increases and explaining the commonly observed performance saturation effect; and (ii) a coverage law showing that the required distilled sample size scales linearly with configuration diversity, with provably matching upper and lower bounds. In addition, our unified analysis reveals that various matching methods are interchangeable surrogates, reducing the same generalization error, clarifying why they can all achieve dataset distillation and providing guidance on how surrogate choices affect sample efficiency and robustness. Experiments across diverse methods and configurations empirically confirm the derived laws, advancing a theoretical foundation for DD and enabling theory-driven design of compact, configuration-robust dataset distillation.

翻译：数据集蒸馏（DD）旨在构建紧凑的合成数据集，使模型在显著降低存储和计算成本的同时，达到与全数据训练相当的性能。尽管实证研究进展迅速，其理论基础仍显不足：现有方法（梯度匹配、分布匹配、轨迹匹配）建立在异构的代理目标与优化假设之上，这导致难以分析其共同原理或提供普适性保证。此外，当训练配置（如优化器、架构或数据增强策略）发生变化时，蒸馏数据在何种条件下仍能保持全数据集的有效性尚不明确。为回答这些问题，我们提出了一个统一的理论框架，称为配置-动态-误差分析。该框架将主流DD方法重新置于统一的泛化误差视角下，并给出两个核心结论：（i）缩放定律提供了单配置误差上界，刻画了误差随蒸馏样本量增加而下降的规律，解释了常见的性能饱和现象；（ii）覆盖定律表明所需蒸馏样本量与配置多样性呈线性缩放关系，并给出了可证明匹配的上界与下界。此外，我们的统一分析揭示各类匹配方法本质上是可互换的代理目标，均通过降低相同泛化误差实现蒸馏，这阐明了它们均能实现数据集蒸馏的原因，并为代理选择如何影响样本效率与鲁棒性提供了理论指导。跨多种方法与配置的实验结果实证验证了所推导的定律，推动了DD理论基础的建立，并为设计紧凑且配置鲁棒的数据集蒸馏方法提供了理论驱动的设计依据。