Multi-Turn AI Reasoning Fails by Forgetting, Not Contradiction
A recent study indicates that the primary failure mode of multi-turn reasoning systems is 'satisfiable drift.' This occurs when the internal state stays consistent, yet the response inadvertently contradicts earlier commitments, rather than resulting in a logical contradiction. To investigate this, researchers developed DRIFT-Bench, a benchmark consisting of 816 test problems across three constraint domains, and assessed four methods using four open-weight models ranging from 8B to 120B parameters. The MUS-Repair approach, which reintroduces minimal unsatisfiable subsets to the generator, surpassed all baseline methods by 1.8 to 15.0 percentage points. Following structured feedback, contradictions were nearly absent, with residual errors being 98-100% due to satisfiable drift. The study can be found on arXiv with ID 2605.23940.
Key facts
- Multi-turn reasoning systems fail mainly through satisfiable drift, not contradiction.
- DRIFT-Bench includes 816 test problems across three constraint domains.
- Four methods were evaluated on four open-weight models (8B-120B parameters).
- MUS-Repair outperformed all non-MUS baselines by +1.8 to +15.0 percentage points.
- Residual errors after feedback are 98-100% satisfiable drift.
- Contradiction drops to near zero after structured feedback.
- The paper is on arXiv with ID 2605.23940.
Entities
Institutions
- arXiv