AI Agents Fool Humans in Online Group Conversations
A new study published on arXiv reveals that socially fluent AI agents can now participate in online text-based group interactions so convincingly that humans cannot distinguish them from real people. In the experiment, undisclosed AI agents were embedded as ordinary teammates across analytical, creative, and ethical tasks. Among 786 participants who made 1,572 post-interaction identity judgments, people failed to identify AI above chance levels. The failure was not due to a lack of identity-relevant information; computational classifiers could accurately differentiate AI from humans based on conversational cues. Instead, participants relied on flawed heuristics such as response speed, fluency, and perceived scriptedness, which were weakly correlated with actual identity. The findings highlight a growing challenge in online interaction as AI becomes socially fluent.
Key facts
- Study published on arXiv with identifier 2605.23426.
- AI agents were embedded as undisclosed teammates in synchronous text-based group chats.
- Tasks included analytical, creative, and ethical problems.
- 786 participants made 1,572 identity judgments.
- Participants could not distinguish AI from humans above chance.
- Computational classifiers achieved high accuracy in differentiating AI from humans.
- Participants relied on heuristics like response speed, fluency, and scriptedness.
- The study suggests AI can decouple conversational signals from source identity.
Entities
Institutions
- arXiv