ART Framework Enhances VLA Models with On-the-fly Tool-use

The Agentic Robot with Tool-use (ART) framework introduces a novel integration of end-to-end Visual-Language-Action (VLA) models with agentic tool-use, allowing robots to utilize readily available tool modules for enhanced efficiency. As outlined in arXiv paper 2608.14047, this framework simplifies the action solution space in comparison to traditional VLA models, resulting in improved generalizability across various tasks and decreased reliance on extensive data. To showcase these benefits, researchers compiled a dataset featuring 30K tool-use trajectories and action demonstrations, which is notably smaller than those employed by conventional methods. Their training regimen focuses on long-trajectory tool-use reasoning in complex settings. Findings reveal that ART surpasses baseline methods by achieving a 20% higher success rate. The paper was submitted as a cross-type entry on arXiv, suggesting potential conference presentation. This framework aims to serve as a tool-injection system, optimizing any VLA model and enhancing capabilities in low-level vision, high-level affordance, and embodiment. This strategy could minimize the necessity for extensive data gathering in robot learning, making VLA models more applicable in real-world scenarios.