Learning natural, animal-like locomotion from demonstrations has become a core paradigm in legged robotics. Despite the recent advancements in motion tracking, most existing methods demand extensive tuning and rely on reference data during deployment, limiting adaptability. We present APEX (Action Priors enable Efficient Exploration), a plug-and-play extension to state-of-the-art motion tracking algorithms that eliminates any dependence on reference data during deployment, improves sample efficiency, and reduces parameter tuning effort. APEX integrates expert demonstrations directly into reinforcement learning (RL) by incorporating decaying action priors, which initially bias exploration toward expert demonstrations but gradually allow the policy to explore independently. This is combined with a multi-critic framework that balances task performance with motion style. Moreover, APEX enables a single policy to learn diverse motions and transfer reference-like styles across different terrains and velocities, while remaining robust to variations in reward design. We validate the effectiveness of our method through extensive experiments in both simulation and on a Unitree Go2 robot. By leveraging demonstrations to guide exploration during RL training, without imposing explicit bias toward them, APEX enables legged robots to learn with greater stability, efficiency, and generalization. We believe this approach paves the way for guidance-driven RL to boost natural skill acquisition in a wide array of robotic tasks, from locomotion to manipulation. Website and code: https://marmotlab.github.io/APEX/.

翻译：从演示中学习自然、类动物的运动已成为腿式机器人领域的核心范式。尽管运动跟踪技术近期取得了进展，但现有方法大多需要大量调参，并在部署时依赖参考数据，限制了其适应性。我们提出了APEX（动作先验赋能高效探索），这是一种对最先进运动跟踪算法的即插即用扩展，消除了部署时对参考数据的任何依赖，提高了样本效率，并减少了参数调优工作量。APEX通过引入衰减的动作先验，将专家演示直接整合到强化学习（RL）中：这些先验在初始阶段引导探索偏向专家演示，但逐渐允许策略独立探索。该方法与一个多评价器框架相结合，平衡了任务性能与运动风格。此外，APEX使单一策略能够学习多样化运动，并在不同地形和速度间传递类参考风格，同时对奖励设计的变化保持鲁棒性。我们通过在仿真和Unitree Go2机器人上的大量实验验证了该方法的有效性。通过利用演示在RL训练期间引导探索，而不施加明确的偏向，APEX使腿式机器人能够以更高的稳定性、效率和泛化能力进行学习。我们相信，这种方法为引导驱动的RL铺平了道路，可促进从运动到操作等广泛机器人任务中自然技能的习得。项目网站与代码：https://marmotlab.github.io/APEX/。