LLMs Tested for Survey Research in Disaster Preparedness

A new study explores how large language models (LLMs) can address common problems in survey research, such as low response rates, biased samples, and fake submissions. The researchers put forward a five-step framework for incorporating LLMs, covering aspects like designing questionnaires, selecting samples, conducting pilot tests, handling missing data, and analyzing results afterward. They used the 2024 Hurricane Milton preparedness survey, which involved 946 residents from Florida, as a case study. The team developed a knowledge graph based on Protection Motivation Theory (PMT) and tested seven different LLM configurations, including zero-shot inference and theory-based methods like the Anchored Margin approach. This comprehensive evaluation is available on arXiv (2605.19229v1).