Mutual Reinforcement Learning Framework for Heterogeneous LLMs

A novel framework known as Mutual Reinforcement Learning has been developed, enabling distinct families of large language models (LLMs) to collaboratively learn post-training despite differing objectives and configurations. Key components of this system include features such as Shared Experience Exchange (SEE) and Multi-Worker Resource Allocation (MWRA), along with a Tokenizer Heterogeneity Layer (THL) for efficient retokenization. Additionally, three innovative tools inspired by Generalized Randomized Play Optimization (GRPO) have been introduced: Peer Rollout Pooling (PRP), Cross-Policy GRPO Advantage Sharing (XGRPO), and Success-Gated Transfer (SGT). Research indicates these methods require a careful balance between stability and support.