Accurate robot segmentation is a fundamental capability for robotic perception. It enables precise visual servoing for VLA systems, scalable robot-centric data augmentation, accurate real-to-sim transfer, and reliable safety monitoring in dynamic human-robot environments. Despite the strong capabilities of modern segmentation models, surprisingly it remains challenging to segment robots. This is due to robot embodiment diversity, appearance ambiguity, structural complexity, and rapid shape changes. Embracing these challenges, we introduce RobotSeg, a foundation model for robot segmentation in image and video. RobotSeg is built upon the versatile SAM 2 foundation model but addresses its three limitations for robot segmentation, namely the lack of adaptation to articulated robots, reliance on manual prompts, and the need for per-frame training mask annotations, by introducing a structure-enhanced memory associator, a robot prompt generator, and a label-efficient training strategy. These innovations collectively enable a structure-aware, automatic, and label-efficient solution. We further construct the video robot segmentation (VRS) dataset comprising over 2.8k videos (138k frames) with diverse robot embodiments and environments. Extensive experiments demonstrate that RobotSeg achieves state-of-the-art performance on both images and videos, establishing a strong foundation for future advances in robot perception.

翻译：精确的机器人分割是机器人感知的一项基础能力。它能够为视觉语言动作系统实现精准的视觉伺服控制，支持可扩展的以机器人为中心的数据增强，实现准确的真实到仿真环境迁移，并在动态的人机交互环境中提供可靠的安全监控。尽管现代分割模型具备强大的能力，但令人惊讶的是，分割机器人仍然具有挑战性。这源于机器人本体的多样性、外观的模糊性、结构的复杂性以及快速的形状变化。为应对这些挑战，我们提出了RobotSeg，一个用于图像和视频中机器人分割的基础模型。RobotSeg基于通用的SAM 2基础模型构建，但针对机器人分割的三个局限性进行了改进：缺乏对关节式机器人的适应性、依赖手动提示以及需要逐帧训练掩码标注。我们通过引入结构增强的记忆关联器、机器人提示生成器和标签高效训练策略来解决这些问题。这些创新共同实现了一个结构感知、自动化和标签高效的解决方案。我们进一步构建了视频机器人分割数据集，包含超过2.8千个视频（13.8万帧），涵盖了多样化的机器人本体和环境。大量实验表明，RobotSeg在图像和视频上均达到了最先进的性能，为未来机器人感知的进展奠定了坚实的基础。