Fundamental machine learning theory shows that different samples contribute unequally both in learning and testing processes. Contemporary studies on DNN imply that such sample di?erence is rooted on the distribution of intrinsic pattern information, namely sample regularity. Motivated by the recent discovery on network memorization and generalization, we proposed a pair of sample regularity measures for both processes with a formulation-consistent representation. Specifically, cumulative binary training/generalizing loss (CBTL/CBGL), the cumulative number of correct classi?cations of the training/testing sample within training stage, is proposed to quantize the stability in memorization-generalization process; while forgetting/mal-generalizing events, i.e., the mis-classification of previously learned or generalized sample, are utilized to represent the uncertainty of sample regularity with respect to optimization dynamics. Experiments validated the effectiveness and robustness of the proposed approaches for mini-batch SGD optimization. Further applications on training/testing sample selection show the proposed measures sharing the uni?ed computing procedure could benefit for both tasks.

翻译：有关DNN的当代研究表明,这种抽样的二元性根植于内在模式信息的传播,即抽样的规律性。由于最近发现网络记忆和一般化,我们提议对两种过程采取符合拟订要求的抽样常规性措施。具体地说,累积的二元培训/普遍损失(CBTL/CBGL)是培训阶段内正确分类/测试样本的累积数量?建议对培训/测试样本的累积数量进行分类,以量化记忆化一般化过程的稳定性;同时,遗忘/综合化事件,即以前学到的样本或通用样本的分类不当,被用来代表样本在优化动态方面的常规性不确定性。实验证实了拟议的微型批量SGD优化方法的有效性和稳健性。在培训/测试样本选择方面的进一步应用显示,共享统一计算程序的拟议措施对这两项任务都有好处。