AI Chatbots Fooled by Fake Disease Invented by Researchers
Researchers led by Almira Osmanovic Thunström at the University of Gothenburg invented a fake disease called bixonimania to test whether large language models (LLMs) could detect misinformation. In early 2024, they published blog posts on Medium and two fabricated research reports on a preprint server, with obvious clues like a lead author named Lazljiv Izgubljenovic (meaning 'lying loser') and references to 'Galactic Triad' and 'Lord of the Rings.' Despite these red flags, chatbots including Microsoft Bing's Copilot, Google's Gemini, and OpenAI's ChatGPT began citing the condition as real. The project exposed how LLMs ingest web data uncritically, and even some scientists cited the bogus preprints without reading them. Osmanovic Thunström, also an AI strategist at Chalmers Industriteknik, aimed to demonstrate how training data is processed. The fake disease now has its own Wikipedia page. The experiment highlights the vulnerability of AI systems to misinformation and the need for human critical thinking.
Key facts
- Researchers invented a fake disease called bixonimania to test AI chatbots.
- The team was led by Almira Osmanovic Thunström at the University of Gothenburg.
- Information was circulated via Medium blog posts and preprint server reports in early 2024.
- Fabricated papers included obvious clues like a fake lead author name and absurd funding sources.
- Chatbots like Microsoft Bing Copilot, Google Gemini, and OpenAI ChatGPT referenced the fake condition as real.
- Some scientists cited the bogus preprints without reading them.
- The fake disease now has its own Wikipedia page.
- The project demonstrates how LLMs can spread misinformation from the web.
Entities
Institutions
- University of Gothenburg
- Chalmers Industriteknik
- Microsoft
- OpenAI
- University College London
- University of Cambridge
- Scientific American
- Nature
- Medium
- MIT
Locations
- Sweden