EXPO-FT: Sample-Efficient RL Finetuning for Vision-Language-Action Models
EXPO-FT is a novel system designed for stable and sample-efficient reinforcement learning (RL) fine-tuning of pretrained vision-language-action (VLA) policies. It tackles the reliability issues that hinder the deployment of pretrained robotic policies in real-world scenarios. In contrast to current approaches that either start from scratch or fine-tune with inadequate sample efficiency, EXPO-FT utilizes pretrained knowledge to ensure both stability and high success rates. This system was tested on complex manipulation tasks, including routing string lights and plugging them in, pocketing a pool ball, and placing a flower into a wine bottle. The research, available on arXiv under 2605.25477 with a replace-cross announcement, posits that RL fine-tuning is a viable method for efficiently acquiring new tasks, bridging the gap between pretrained VLA models and their practical applications.
Key facts
- EXPO-FT is a system for stable, sample-efficient RL finetuning of pretrained VLA policies.
- VLA models demonstrate strong generalization across diverse manipulation tasks.
- Pretrained policies fall short of the reliability required for real-world deployment.
- Existing RL fine-tuning approaches either train from scratch or fine-tune without sufficient sample efficiency.
- EXPO-FT closes the gap between pretrained VLAs and practical deployment.
- The system solves routing string lights and inserting the plug to light it up.
- It solves striking a pool ball into a pocket.
- It solves inserting a flower into a wine bottle.
Entities
Institutions
- arXiv