The low-level sensory and motor signals in deep reinforcement learning, which exist in high-dimensional spaces such as image observations or motor torques, are inherently challenging to understand or utilize directly for downstream tasks. While sensory representations have been extensively studied, the representations of motor actions are still an area of active exploration. Our work reveals that a space containing meaningful action representations emerges when a multi-task policy network takes as inputs both states and task embeddings. Moderate constraints are added to improve its representation ability. Therefore, interpolated or composed embeddings can function as a high-level interface within this space, providing instructions to the agent for executing meaningful action sequences. Empirical results demonstrate that the proposed action representations are effective for intra-action interpolation and inter-action composition with limited or no additional learning. Furthermore, our approach exhibits superior task adaptation ability compared to strong baselines in Mujoco locomotion tasks. Our work sheds light on the promising direction of learning action representations for efficient, adaptable, and composable RL, forming the basis of abstract action planning and the understanding of motor signal space. Project page: https://sites.google.com/view/emergent-action-representation/

翻译：深度强化学习中的低层次感官和运动信号,存在于图像观测或发动机拳击等高维空间中,在深度强化学习中的低层次感官和运动信号,对于直接理解或利用下游任务具有内在的挑战性。虽然对感官表现进行了广泛的研究,但机动行动的表现仍然是一个积极探索的领域。我们的工作表明,当多任务政策网络同时作为投入投入投入和任务嵌入时,就会产生包含有意义行动表现的空间。为提高其代表性而增加中度限制。因此,内插或由嵌入构成的内嵌可以作为该空间中的高级别界面发挥作用,为代理人提供执行有意义行动序列的指示。情感结果表明,拟议的行动表现对于行动内部的相互作用和相互行动构成是有效的,而且学习有限或没有额外的学习。此外,我们的做法显示,与Mujoco loco locomotion 任务中的强基线相比,我们的任务适应能力更强。我们的工作为学习高效、适应和可调适的RL的动作表现提供了有希望的方向,从而形成抽象行动规划的基础和对机动信号空间的理解。项目页面: http://action-action-gogles/golegrationsimmresmissionalpresmationpresmationpagepagepage:http://http://mmmmmmmmus/s/s/s/simmactactalimmactimmactimmactalimmactals/s/s/s/s/s/s/simpactalimactalimactalimactalim/s/smationalimactactimactalimactimactimmtions/smationsmtionalimmationsmationsmationsmations/s/s/s/smationsimpactalimpactalimactalimps/s/s/s/s/s/s/smressmage/smtionsmations/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/s/simpalimpalimpalimpalimpress/s/s/s/s/s/s/smresmresm</s>