Solving inverse problems, such as parameter estimation and optimal control, is a vital part of science. Many experiments repeatedly collect data and rely on machine learning algorithms to quickly infer solutions to the associated inverse problems. We find that state-of-the-art training techniques are not well-suited to many problems that involve physical processes. The highly nonlinear behavior, common in physical processes, results in strongly varying gradients that lead first-order optimizers like SGD or Adam to compute suboptimal optimization directions. We propose a novel hybrid training approach that combines higher-order optimization methods with machine learning techniques. We take updates from a scale-invariant inverse problem solver and embed them into the gradient-descent-based learning pipeline, replacing the regular gradient of the physical process. We demonstrate the capabilities of our method on a variety of canonical physical systems, showing that it yields significant improvements on a wide range of optimization and learning problems.

翻译：解决反向问题,如参数估计和最佳控制,是科学的重要组成部分。许多实验反复收集数据,并依靠机器学习算法快速推断相关反向问题的解决方案。我们发现,最先进的培训技术并不适合许多涉及物理过程的问题。在物理过程中常见的高度非线性行为导致极不相同的梯度,导致像SGD或Adam这样的一阶优化者计算亚优度优化方向。我们建议采用新的混合培训方法,将高阶优化方法与机器学习技术相结合。我们从一个规模变化的反问题解答器中获取最新信息,并将其嵌入基于梯度的白化学习管道,取代物理过程的常规梯度。我们展示了我们方法在各种精度物理系统上的能力,表明它在广泛的优化和学习问题上取得了显著的改进。