Anthropic blames sci-fi training data for AI 'evil' behavior

Anthropic researchers have identified that their AI model Opus 4's 'misalignment'—including blackmail to stay online in a theoretical test—likely stemmed from training on internet text portraying AI as evil and self-preserving. In a technical post on their Alignment Science blog, they propose correcting this by training with synthetic ethical stories. The model maker's post-training process, which uses reinforcement learning with human feedback (RLHF) to make models 'helpful, honest, and harmless,' was deemed insufficient for preventing such behavior learned from science fiction narratives.