Location Preference Optimization Introduced for Enhanced GUI Agent Spatial Localization
A novel method called Location Preference Optimization (LPO) has been developed to improve how autonomous agents interact with Graphical User Interfaces using natural language. Current approaches, primarily relying on Supervised Fine-Tuning for spatial localization, struggle with accurately perceiving positional data. Reinforcement learning techniques also frequently fall short in evaluating positional accuracy. LPO addresses these limitations by utilizing locational data to optimize interaction preferences. The approach employs information entropy to predict interaction positions, concentrating on information-rich zones. Additionally, it incorporates a dynamic location reward function based on physical distance, which accounts for the differing significance of various interaction positions. This research, documented in arXiv:2506.09373v3, represents a technical advancement in the field of autonomous agents and human-computer interaction.
Key facts
- Location Preference Optimization (LPO) is a new method for GUI agent interaction.
- LPO uses locational data to optimize interaction preferences.
- It employs information entropy to predict interaction positions in information-rich zones.
- A dynamic location reward function based on physical distance is introduced.
- Current Supervised Fine-Tuning methods for spatial localization face challenges with positional data perception.
- Existing reinforcement learning strategies often fail to assess positional accuracy effectively.
- The research is documented in arXiv:2506.09373v3.
- The method aims to enhance autonomous agents' interactions with Graphical User Interfaces via natural language.
Entities
—