MT-GRPO: Balancing Multi-Task LLM Reasoning
A new algorithm called Multi-Task GRPO (MT-GRPO) has been developed by researchers to enhance the dependability of large language models (LLMs) across various reasoning challenges. Conventional reinforcement learning methods utilizing GRPO often result in uneven performance, where certain tasks overshadow others, causing stagnation. Furthermore, the frequency with which prompts yield no advantages can distort the optimization signal's effectiveness. MT-GRPO tackles these challenges by adjusting task weights dynamically to improve the performance of the worst task and ensure balanced advancement. It also employs a ratio-preserving sampler to align task-specific policy gradients with these adjusted weights. Experiments conducted in both 3-task and 9-task scenarios demonstrate that MT-GRPO consistently surpasses existing baseline approaches. The research paper can be found on arXiv with the identifier 2602.05547.
Key facts
- MT-GRPO is a novel algorithm for multi-task reinforcement learning in LLMs.
- It dynamically adapts task weights to optimize worst-task performance.
- It introduces a ratio-preserving sampler for task-wise policy gradients.
- Experiments were conducted on 3-task and 9-task settings.
- MT-GRPO consistently outperforms baseline methods.
- The paper is available on arXiv (2602.05547).
- The algorithm addresses imbalanced outcomes in multi-task GRPO.
- It handles varying zero-advantage rates across tasks.
Entities
Institutions
- arXiv