Study Finds LLMs Fail to Mimic Human Belief Dynamics
A recent study in computer science investigates the ability of large language models (LLMs) to mimic human belief formation and alteration within social networks. By replicating a prior study on belief dynamics, researchers assessed 12 LLMs from various model families and sizes. The results indicate a definitive failure; LLMs do not accurately represent initial human belief distributions and generally exhibit greater conformity than humans, adjusting their responses to match those around them. Additionally, they demonstrate a complex understanding of human homophily in networks. These results underscore critical aspects of LLM behavior and serve as a strong caution against using LLMs as substitutes for humans in social simulations.
Key facts
- Study tests whether LLMs can emulate human belief dynamics in social networks.
- Replicates an established study on belief dynamics.
- Evaluates 12 LLMs across multiple model families and parameter sizes.
- LLMs fail to capture initial human belief distributions.
- LLMs are more conformist than humans.
- LLMs shift responses to align with others.
- LLMs take a nuanced approach to homophily.
- Findings warn against using LLMs as human proxies in social simulations.
Entities
Institutions
- arXiv