Novel ML Method Trains Computer Use Agent for GUI Usability Assessment
Researchers present uxCUA, a computer use agent trained via a novel machine learning method to assess graphical user interface (GUI) usability. The method operationalizes a computational definition of usability by prioritizing important interaction flows, executing human-like interactions, and predicting a numerical usability score. uxCUA is trained on a large-scale dataset of fully interactive UIs paired with usability labels and human preferences, outperforming larger models. This work addresses the high cost and time of traditional usability testing with experts and users.
Key facts
- Usability testing with experts and users remains costly and time-intensive.
- Prior generative agents struggle to provide accurate usability assessments.
- The method prioritizes important interaction flows, executes human-like interactions, and predicts a numerical usability score.
- uxCUA is trained on a large-scale dataset of fully interactive UIs with usability labels and human preferences.
- uxCUA outperforms larger models.
- The work is published on arXiv with ID 2604.26020.
- The method is a novel machine learning approach.
- The goal is to assess GUI effectiveness, efficiency, and user satisfaction.
Entities
Institutions
- arXiv