Deploying autonomous robots that can learn new skills from demonstrations is an important challenge of modern robotics. Existing solutions often apply end-to-end imitation learning with Vision-Language Action (VLA) models or symbolic approaches with Action Model Learning (AML). On the one hand, current VLA models are limited by the lack of high-level symbolic planning, which hinders their abilities in long-horizon tasks. On the other hand, symbolic approaches in AML lack generalization and scalability perspectives. In this paper we present a new neuro-symbolic approach, GraSP-VLA, a framework that uses a Continuous Scene Graph representation to generate a symbolic representation of human demonstrations. This representation is used to generate new planning domains during inference and serves as an orchestrator for low-level VLA policies, scaling up the number of actions that can be reproduced in a row. Our results show that GraSP-VLA is effective for modeling symbolic representations on the task of automatic planning domain generation from observations. In addition, results on real-world experiments show the potential of our Continuous Scene Graph representation to orchestrate low-level VLA policies in long-horizon tasks.

翻译：部署能够从演示中学习新技能的自主机器人是现代机器人学的重要挑战。现有解决方案通常采用端到端的模仿学习结合视觉-语言-动作（VLA）模型，或基于动作模型学习（AML）的符号化方法。一方面，当前VLA模型因缺乏高层符号规划能力而受限，阻碍了其在长时程任务中的表现；另一方面，AML中的符号化方法在泛化性和可扩展性方面存在不足。本文提出一种新的神经符号方法——GraSP-VLA框架，该框架利用连续场景图表示生成人类演示的符号化表征。该表征在推理过程中用于生成新的规划领域，并作为底层VLA策略的协调器，从而提升连续可复现动作的数量级。实验结果表明，GraSP-VLA在基于观察的自动规划领域生成任务中能有效建模符号表征。此外，真实场景实验验证了连续场景图表示在协调长时程任务中底层VLA策略的潜力。