Several self-supervised representation learning methods have been proposed for reinforcement learning (RL) with rich observations. For real-world applications of RL, recovering underlying latent states is crucial, particularly when sensory inputs contain irrelevant and exogenous information. In this work, we study how information bottlenecks can be used to construct latent states efficiently in the presence of task-irrelevant information. We propose architectures that utilize variational and discrete information bottlenecks, coined as RepDIB, to learn structured factorized representations. Exploiting the expressiveness bought by factorized representations, we introduce a simple, yet effective, bottleneck that can be integrated with any existing self-supervised objective for RL. We demonstrate this across several online and offline RL benchmarks, along with a real robot arm task, where we find that compressed representations with RepDIB can lead to strong performance improvements, as the learned bottlenecks help predict only the relevant state while ignoring irrelevant information.

翻译：为强化学习(RL)提出了几种自我监督的代表学习方法,并提出了丰富的观察意见。对于RL的实际应用,恢复潜在的潜伏状态至关重要,特别是当感官投入含有无关和外源信息时。在这项工作中,我们研究如何利用信息瓶颈,在与任务相关的信息面前高效建设潜在国家。我们提出了利用以RepDIB(RepDIB)(RepDIB)(RepDIB)(RepDIB)(Reformation and different)(Reformation)(Reformation)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(RL)(L(RL)(L)(L)(L(RL)(L)(RL)(L(RL) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L(L) (L) (L) (L) (L) (L) (L(L(L) (L) (L) (L) (L) (L) (L(L) (L) (L) (L) (L) (L(L(L(L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L(L(L(L(L) (L(L) (L) (L) (L) (L(L(L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L) (L)(L) (L(L)(L(L)(L)