Study Reveals LLMs Amplify Polarization in Content Curation Across Major Providers

A simulation study analyzing biases in Large Language Models (LLMs) used for content curation found that polarization is consistently amplified across all configurations. The research, detailed in arXiv preprint 2604.15937, examined three major LLM providers: OpenAI, Anthropic, and Google. It utilized real social media datasets from Twitter/X, Bluesky, and Reddit to map content selection biases. Over 540,000 simulated top-10 selections were made from pools of 100 posts across 54 experimental conditions. Six prompting strategies were tested: general, popular, engaging, informative, controversial, and neutral. The study revealed that biases vary significantly in their structural nature and sensitivity to prompt design. Toxicity handling exhibited a strong inversion between engagement and neutrality prompts. This work highlights the poorly understood nature of LLM biases in ranking human-created content. It aims to identify which biases are robust across providers and platforms and which can be mitigated through prompt engineering. The findings underscore the challenges in deploying LLMs for content curation tasks without unintended consequences.