Persona Policies Improve LLM Agent Evaluation Realism
Researchers introduce Persona Policies (PPol), a control layer for LLM-based user simulators that generates realistic behavioral variation. Current simulators are cooperative and homogeneous, causing agents to fail with real users. PPol uses LLM-driven evolutionary program search to create diverse personas without hand-crafting, improving evaluation robustness.
Key facts
- LLM agents interact with diverse users including unclear, impatient, or reluctant individuals.
- Real interaction data collection is expensive.
- Existing LLM simulators are cooperative and homogeneous.
- Agents strong in simulation often fail with real users.
- Persona Policies (PPol) is a plug-and-play control layer.
- PPol induces realistic behavioral variation while preserving task goals.
- Persona generation uses LLM-driven evolutionary program search.
- The approach optimizes a Python generator to discover behaviors.
Entities
Institutions
- arXiv