CoAdapt-GUI: Test-Time Adaptation for Unseen Mobile GUI Applications
A new study has introduced CoAdapt-GUI, which is designed to improve how mobile GUI agents adapt to new applications at test time. This framework adjusts both the workflow context and the agent's policy by using its own rollouts and rewards from the target app. It keeps transferable processes and rules while discarding specific source app details, allowing adaptation based on workflow knowledge. For updating the policy, it employs a method that optimizes a LoRA adapter within a fixed vision-language model. In evaluations on two benchmarks with unfamiliar apps, CoAdapt-GUI achieved a score of 45.0% on AndroidWorld-Generalization, surpassing the 37.5% of the Policy-Only TTA baseline, and also improved results on AndroidWorld Plus. The paper is available on arXiv with the identifier 2608.11588.
Key facts
- CoAdapt-GUI is a test-time adaptation framework for mobile GUI agents.
- It jointly adapts workflow context and policy from target-app rollouts and rewards.
- Workflow context retains transferable procedures, failure modes, and verification rules.
- Policy adaptation uses task-context-matched group-relative optimization on a LoRA adapter.
- Achieves 45.0% on AndroidWorld-Generalization vs 37.5% for Policy-Only TTA baseline.
- Also improves performance on AndroidWorld Plus.
- Paper available on arXiv with ID 2608.11588.
- Aims to improve generalization to unseen GUI applications without target demonstrations.
Entities
Institutions
- arXiv