Solving inverse problems, such as parameter estimation and optimal control, is a vital part of science. Many experiments repeatedly collect data and employ machine learning algorithms to quickly infer solutions to the associated inverse problems. We find that state-of-the-art training techniques are not well-suited to many problems that involve physical processes since the magnitude and direction of the gradients can vary strongly. We propose a novel hybrid training approach that combines higher-order optimization methods with machine learning techniques. We replace the gradient of the physical process by a new construct, referred to as the physical gradient. This also allows us to introduce domain knowledge into training by incorporating priors about the solution space into the gradients. We demonstrate the capabilities of our method on a variety of canonical physical systems, showing that physical gradients yield significant improvements on a wide range of optimization and learning problems.

翻译：解决反向问题,如参数估计和最佳控制,是科学的重要组成部分。许多实验反复收集数据和使用机器学习算法,以快速推断相关反向问题的解决办法。我们发现,由于梯度的大小和方向差异很大,因此最先进的培训技术并不适合涉及物理过程的许多问题。我们建议采用新的混合培训方法,将高阶优化方法与机器学习技术相结合。我们用称为物理梯度的新结构取代物理过程的梯度。这也使我们能够将域知识引入培训,将有关解决方案空间的先行纳入梯度。我们展示了我们方法在各种可视物理系统方面的能力,表明物理梯度在广泛的优化和学习问题上产生了显著的改进。