Flexer: Adaptive Mixing of Neural and MCTS Policies for Efficient Reinforcement Learning

A new reinforcement learning architecture called Flexer has been introduced, which adaptively mixes policies from neural networks and Monte Carlo tree search (MCTS) to reduce computational cost while maintaining performance. The approach, detailed in a paper submitted to arXiv, addresses the inefficiency of performing search to a fixed depth at regular intervals. Instead, Flexer dynamically adjusts the search depth based on the quality of the policy network's priors, reducing search effort when the network is confident. The mixing factor favors the MCTS policy when the network's imitation error and the environment model's variance increase. In experiments on three toy symbolic problems, Flexer outperformed versions of AlphaZero, DQN, and ADP. The paper is categorized under Computer Science > Artificial Intelligence and was submitted on arXiv with the identifier 2608.15700. The work highlights a potential improvement in sample efficiency and computational efficiency for reinforcement learning agents, particularly in domains where planning is expensive.