Learning neural ODEs often requires solving very stiff ODE systems, primarily using explicit adaptive step size ODE solvers. These solvers are computationally expensive, requiring the use of tiny step sizes for numerical stability and accuracy guarantees. This paper considers learning neural ODEs using implicit ODE solvers of different orders leveraging proximal operators. The proximal implicit solver consists of inner-outer iterations: the inner iterations approximate each implicit update step using a fast optimization algorithm, and the outer iterations solve the ODE system over time. The proximal implicit ODE solver guarantees superiority over explicit solvers in numerical stability and computational efficiency. We validate the advantages of proximal implicit solvers over existing popular neural ODE solvers on various challenging benchmark tasks, including learning continuous-depth graph neural networks and continuous normalizing flows.

翻译：学习神经代码往往需要解决非常僵硬的 ODE 系统, 主要是使用明确的适应级大小的 ODE 解码器。 这些解码器在计算上费用昂贵, 需要使用小步尺寸来保证数字稳定性和准确性。 本文考虑使用不同订单的隐含 DEDE 解码器来学习神经代码, 使用准ximal 隐含解码器来利用准ximal运算器。 精密的隐含解码器由内部外延码组成: 内部迭代大约每个隐含的更新步骤使用快速优化算法, 而外部迭代则随着时间的推移解决了ODE 系统。 近ximal 隐含解码器保证了在数字稳定性和计算效率方面相对于显性解码器的优势。 我们验证了原始隐含的神经代码解码器在各种具有挑战性的基准任务方面的优势, 包括学习连续深度的图形神经网络和连续的正常流。