Research Proposes Framework for Optimizing Human-LLM Survey Allocation
A new research paper introduces a framework for optimally allocating human respondents in surveys augmented by Large Language Models. The study addresses the challenge of unpredictable LLM accuracy across different survey questions. Researchers developed a method combining three components: characterizing question-specific rectification difficulty, deriving a closed-form optimal allocation rule, and proposing a meta-learning approach for new surveys. The framework directs more human labels to tasks where LLMs are least reliable, maximizing efficiency within fixed budgets. This approach extends to general M-estimation problems, including regression coefficient estimation. The paper was published on arXiv under identifier 2604.17267v1.
Key facts
- Large Language Models can generate synthetic survey responses at low cost
- LLM accuracy varies unpredictably across different survey questions
- Researchers study allocation of fixed budget of human respondents across estimation tasks
- Framework characterizes question-specific rectification difficulty
- Optimal allocation rule directs more human labels to tasks where LLM is least reliable
- Meta-learning approach predicts rectification difficulty for new tasks without pilot data
- Framework extends to general M-estimation problems
- Paper published on arXiv with identifier 2604.17267v1
Entities
Institutions
- arXiv