LLM Agents Predict Social Media Reactions with 70.7% Accuracy in Benchmark Study
A recent study evaluates how well LLM-powered agents can anticipate individual responses to social media posts. The research analyzed more than 120,000 combinations of agents and personas, based on data from 1,511 participants in Serbia and 27 large language models. It revealed that the agents achieved an overall accuracy of 70.7% in predicting user reactions such as like, dislike, comment, share, or no reaction. The choice of LLM resulted in a performance variation of 13 percentage points. In Study 2, a binary forced-choice assessment was conducted using chance-corrected metrics. This research underscores both the capabilities and limitations of AI agents in mimicking human behavior on social media platforms.
Key facts
- LLM agents achieved 70.7% overall accuracy in predicting social media reactions
- Study used 120,000+ agent-persona combinations from 1,511 Serbian participants
- 27 large language models were benchmarked
- LLM choice produced a 13 percentage-point performance spread
- Study 2 employed binary forced-choice (like/dislike) evaluation
- Research focuses on predicting individual reactions to specific content
- Previous work only replicated aggregate survey responses
- Study addresses platform governance and democratic resilience
Entities
Locations
- Serbia