Faithful-Agent: A Framework for Reliable GUI Agents
Researchers propose Faithful-Agent, a framework to improve faithfulness in vision-language model-based GUI agents. These agents often rely on shortcuts rather than screen evidence or user instructions. Faithful-Agent uses a two-stage pipeline: a faithfulness-oriented supervised fine-tuning (SFT) stage to teach abstainment under evidence perturbations, and a reinforcement fine-tuning (RFT) stage with a guided advantage estimator (GuAE) to prevent advantage collapse under sparse rewards. The approach aims to enhance evidence groundedness and internal consistency.
Key facts
- Faithful-Agent addresses unfaithful behavior in GUI agents.
- It uses a two-stage pipeline: SFT and RFT.
- The SFT stage instills abstainment behaviors under evidence perturbations.
- The RFT stage uses guided advantage estimator (GuAE).
- GuAE is based on GRPO and prevents advantage collapse.
- A thought-action consistency reward is used.
- The framework prioritizes evidence groundedness and internal consistency.
- The paper is available on arXiv.
Entities
Institutions
- arXiv