Semantic Flow Regularization Improves LLM Output Diversity

Researchers have developed a new method called Semantic Flow Regularization (SFR) to address Cross-Style Collapse in large language models. This issue happens when models fine-tuned for specific styles end up producing less diverse outputs. The underlying cause is the cross-entropy objective that restricts variation in shared representations. SFR serves as a simple additional goal, guiding the model with continuous sentence-encoder embeddings for future segments via conditional flow matching. It keeps multi-modality intact, and the flow-matching head is removed during inference, meaning no extra costs for deployment. Tests on the Qwen3-32B dataset, featuring 9 personas, show that SFR improves output diversity, style consistency, and response quality. Validation on LiveCodeBench-v5 confirms these enhancements. You can check out the study on arXiv, identifier 2605.27971.