Vision-Language-Action (VLA) policies excel in aligning language, perception, and robot control. However, most VLAs are trained purely by imitation, which overfits to demonstrations, and is brittle under distribution shift. Reinforcement learning (RL) directly optimizes task reward and thus addresses this misalignment, but real-robot interaction is expensive and conventional simulators are hard to engineer and transfer. We address both data efficiency and optimization stability in VLA post-training via a learned world model and an RL procedure tailored to flow-based action heads. Specifically, we introduce Prophet, a unified action-to-video robot actuation pretrained across large-scale, heterogeneous robot data to learn reusable action-outcome dynamics. It is able to few-shot adapt to new robots, objects, and environments, yielding a rollout-ready simulator. Upon Prophet, we reinforce action policies with Flow-action-GRPO (FA-GRPO), which adapts Flow-GRPO to operate on VLA actions, and with FlowScale, a stepwise reweighting that rescales per-step gradients in the flow head. Together, Prophet, FA-GRPO, and FlowScale constitute ProphRL, a practical, data- and compute-efficient path to VLA post-training. Experiments show 5-17% success gains on public benchmarks and 24-30% gains on real robots across different VLA variants.

翻译：视觉-语言-动作（VLA）策略在语言、感知与机器人控制的对齐方面表现出色。然而，大多数VLA策略仅通过模仿学习进行训练，这会导致对演示数据的过拟合，并在分布偏移时表现脆弱。强化学习（RL）直接优化任务奖励，从而解决了这种不对齐问题，但真实机器人交互成本高昂，且传统仿真器难以构建和迁移。我们通过学习的世界模型和针对基于流的动作头设计的RL流程，解决了VLA后训练中的数据效率和优化稳定性问题。具体而言，我们提出了Prophet，一种在异构大规模机器人数据上预训练的统一动作到视频机器人驱动模型，用于学习可重用的动作-结果动态。它能够通过少量样本适应新的机器人、物体和环境，生成可直接用于仿真的环境。基于Prophet，我们通过Flow-action-GRPO（FA-GRPO）强化动作策略，该方法将Flow-GRPO适配至VLA动作，并结合FlowScale（一种逐步重加权方法，用于重新调整流头中每步梯度）。Prophet、FA-GRPO和FlowScale共同构成了ProphRL，为VLA后训练提供了一条实用、数据与计算高效的路径。实验表明，在不同VLA变体上，公开基准测试的成功率提升5-17%，真实机器人任务中提升24-30%。