IntentScore: AI Reward Model Boosts Computer-Use Agents by 6.9%

A new reward model called IntentScore has been created by researchers to assess the quality of actions taken by Computer-Use Agents (CUAs). These agents, which utilize large language models, execute GUI tasks in desktop settings but frequently encounter irreversible mistakes. IntentScore is trained on 398,000 offline GUI interaction steps from three different operating systems, employing contrastive alignment and margin ranking to evaluate potential actions. By incorporating planning intent into the action encoder, it achieves a pairwise discrimination accuracy of 97.5% on a separate evaluation set. When implemented as a re-ranker for Agent S3 in the unseen OSWorld environment, IntentScore enhanced the task success rate by 6.9%. The findings are published in arXiv:2604.05157.