DRIFT: New AI Framework for Self-Improving Language Models
Researchers have unveiled DRIFT, a framework for online self-evolution policy optimization tailored for large language models. This innovative system facilitates stable self-enhancement without the need for external expert guidance. By monitoring learning progress at the problem level, DRIFT tackles the complexities of reasoning tasks and adjusts its optimization methods accordingly. It employs Difficulty Routing to assess the model's learning state and to strategically distribute self-distillation and reinforcement learning signals, while Rhythm Gating fine-tunes policy updates at the token level. This methodology aims to mitigate over-optimization of simpler problems, inadequate supervision from more challenging ones, and limited exploration of edge cases. The study can be found on arXiv with the identifier 2606.30345.
Key facts
- DRIFT is an online self-evolution policy optimization framework for large language models.
- It enables stable self-improvement without external expert supervision.
- Difficulty Routing allocates self-distillation and reinforcement learning signals at the problem level.
- Rhythm Gating refines policy updates at the token level.
- The framework addresses over-optimization of easy problems and weak supervision from hard problems.
- It aims to improve exploration of borderline cases.
- The paper is available on arXiv with identifier 2606.30345.
- The announcement type is replace-cross.
Entities
Institutions
- arXiv