PS-OPSD: New Method Improves Reasoning in Self-Distilled LLMs

A recent study published on arXiv (2608.01589) presents Problem-Space-Guided On-Policy Self-Distillation (PS-OPSD), a novel technique aimed at improving reasoning within large language models. Instead of relying on complete reference solutions, this approach utilizes trajectory-based guidance that outlines the initial state, goal conditions, constraints, and a chosen state-transition path. This method addresses the limitations of teacher-provided token-level targets, which often depend on reference-specific data that is not accessible during inference. The student rollout and OPSD objective remain unchanged. In tests across three mathematical reasoning benchmarks and model sizes ranging from 1.7B to 8B, PS-OPSD outperformed other methods in aggregate question-only accuracy. Controlled experiments reveal that the relevance of guidance and coherence of paths significantly enhance performance, emphasizing the importance of representing privileged information. The research team has made this work available on arXiv, contributing to advancements in artificial intelligence, particularly in enhancing language models' reasoning abilities through self-distillation methods.