Reinforcement learning (RL) is highly suitable for devising control strategies in the context of dynamical systems. A prominent instance of such a dynamical system is the system of equations governing fluid dynamics. Recent research results indicate that RL-augmented computational fluid dynamics (CFD) solvers can exceed the current state of the art, for example in the field of turbulence modeling. However, while in supervised learning, the training data can be generated a priori in an offline manner, RL requires constant run-time interaction and data exchange with the CFD solver during training. In order to leverage the potential of RL-enhanced CFD, the interaction between the CFD solver and the RL algorithm thus have to be implemented efficiently on high-performance computing (HPC) hardware. To this end, we present Relexi as a scalable RL framework that bridges the gap between machine learning workflows and modern CFD solvers on HPC systems providing both components with its specialized hardware. Relexi is built with modularity in mind and allows easy integration of various HPC solvers by means of the in-memory data transfer provided by the SmartSim library. Here, we demonstrate that the Relexi framework can scale up to hundreds of parallel environment on thousands of cores. This allows to leverage modern HPC resources to either enable larger problems or faster turnaround times. Finally, we demonstrate the potential of an RL-augmented CFD solver by finding a control strategy for optimal eddy viscosity selection in large eddy simulations.

翻译：强化学习(RL)非常适合在动态系统的背景下设计控制战略。这种动态系统的一个突出实例是调节流体动态的方程式系统。最近的研究结果表明,RL增强计算流动态(CFD)解决方案的解决方案可以超过当前最新水平,例如在动荡建模领域。然而,在监管的学习中,培训数据可以先行生成,同时,在培训期间,RL需要与CFD解决方案不断运行时间互动和数据交换。为了发挥RL强化的CFD的潜力,CFD解决方案和RL算法之间的互动因此必须在高性能计算(HPC)硬件上高效实施。为此,我们将Relexi作为可缩缩放的RL框架框架框架,以缩小机器学习工作流程和现代CFD解决方案在提供两种专门硬件的系统上的差距。Relexi在思想上构建了模块化,并使得各种HPC解决方案解决方案的e-PC强化 CFDD, 能够通过智能系统内部手段将各种电子解析器整合起来,从而将我们提供的智能系统核心环境的大规模转移。