This paper presents a general mean-field game (GMFG) framework for simultaneous learning and decision-making in stochastic games with a large population. It first establishes the existence of a unique Nash Equilibrium to this GMFG, and demonstrates that naively combining reinforcement learning with the fixed-point approach in classical MFGs yields unstable algorithms. It then proposes value-based and policy-based reinforcement learning algorithms (GMF-V and GMF-P, respectively) with smoothed policies, with analysis of their convergence properties and computational complexities. Experiments on an equilibrium product pricing problem demonstrate that GMF-V-Q and GMF-P-TRPO, two specific instantiations of GMF-V and GMF-P, respectively, with Q-learning and TRPO, are both efficient and robust in the GMFG setting. Moreover, their performance is superior in convergence speed, accuracy, and stability when compared with existing algorithms for multi-agent reinforcement learning in the $N$-player setting.

翻译：本文介绍了在人口众多的随机游戏中同时学习和决策的一般平均场游戏框架(GMFG),它首先确定存在独特的Nash-V-Q和GMF-P-TRPO(GMF-V和GMF-P)之间的平衡产品定价问题实验显示,GMF-V-Q和GMF-P(分别为Q-学习)和TRPO(分别为GMF-V和GMF-V-P)的两次特定即时反应在GMFG环境中既高效又稳健。此外,它们的业绩在趋同速度、准确性和稳定性方面优于在$-美元玩家设置的多剂强化学习的现有算法。