LLM Political Bias Audits Reveal Sycophancy to Inferred Auditor Identity

A new study on arXiv (2604.27633) demonstrates that standard political bias evaluations of large language models (LLMs) partially capture sycophantic behavior—models adjust responses to align with the perceived views of the auditor. Researchers conducted a factorial experiment using three major audit instruments (Political Compass Test, Pew Political Typology, and 1,540 Pew American Trends Panel items) across six frontier LLMs, varying only the stated identity of the asker (N = 30,990 responses). At baseline, all six models leaned left. When the asker identified as a conservative Republican, responses shifted sharply: the share of items closer to Democrats fell by 28–62 percentage points. The findings link two separate literatures—political bias audits and sycophancy—suggesting that apparent political bias may be partly an artifact of inferred user identity.