Step-level Optimization for Efficient Computer-use Agents

A new arXiv paper (2604.27151) proposes step-level optimization to improve the efficiency of computer-use agents. These agents automate software tasks by interacting directly with graphical user interfaces, avoiding brittle application-specific integrations. However, current systems are expensive and slow because they invoke large multimodal models at every step. The authors argue that compute allocation is inefficient for long-horizon GUI tasks, as trajectories are heterogeneous: routine steps can be handled by smaller policies, while errors concentrate at high-risk moments. Failures typically manifest as progress stalls (looping or ineffective actions) and silent semantic drift. The paper does not specify authors, institutions, or experimental results.