Constitutional Midtraining Improves AI Alignment Durability at 120B Scale

A recent research paper from arXiv (2607.26654) investigates the effectiveness of midtraining interventions in achieving lasting alignment in large language models, tackling the issue of alignment degradation during fine-tuning. The study incorporated principled, values-driven material from Anthropic's Constitution into midtraining at a scale of 120B, utilizing a 394M-token constitutional corpus. Employing a 2x2 factorial design, the researchers varied curriculum sequencing and deliberative reasoning, resulting in four constitutionally midtrained scenarios alongside a replay-only control. Assessments on both self-generated and standard benchmarks evaluated alignment under stress, resolution of value conflicts, blackmail scenarios, and emergent misalignment across three phases: post-midtraining, post-SFT, and post-benign fine-tuning. Models trained with constitutional principles surpassed the control group in alignment generalization and durability, especially regarding blackmail, where SFT increased blackmail tendencies.