Stopping criteria for Stochastic Gradient Descent (SGD) methods play important roles from enabling adaptive step size schemes to providing rigor for downstream analyses such as asymptotic inference. Unfortunately, current stopping criteria for SGD methods are often heuristics that rely on asymptotic normality results or convergence to stationary distributions, which may fail to exist for nonconvex functions and, thereby, limit the applicability of such stopping criteria. To address this issue, in this work, we rigorously develop two stopping criteria for SGD that can be applied to a broad class of nonconvex functions, which we term Bottou-Curtis-Nocedal functions. Moreover, as a prerequisite for developing these stopping criteria, we prove that the gradient function evaluated at SGD's iterates converges strongly to zero for Bottou-Curtis-Nocedal functions, which addresses an open question in the SGD literature. As a result of our work, our rigorously developed stopping criteria can be used to develop new adaptive step size schemes or bolster other downstream analyses for nonconvex functions.

翻译：为解决这一问题,我们在这项工作中严格制定了两项停止标准,可以适用于广泛的非convex函数类别,我们称之为Bottou-Curtis-Nocedal 函数。此外,作为制定这些停止标准的先决条件,我们证明,在SGD中评估的梯度函数对于Bottou-Curtis-Nocedal 函数的梯度函数高度趋同为零,该函数处理的是SGD文献中的一个未决问题。由于我们的工作,我们严格制定的停止标准可以用于开发新的适应性步骤尺寸计划或支持其他非convex 函数的下游分析。