G-Boost: Adaptive Edge-Cloud Framework Enhances Small Language Models
A novel framework named G-Boost has been introduced by researchers, as outlined in a recent arXiv publication. This innovative system aims to enhance the capabilities of small language models (SLMs) operating on edge devices without requiring modifications to their parameters. G-Boost allows SLMs to work alongside more robust cloud-based large language models (LLMs) while maintaining data privacy. Utilizing a Monte Carlo tree search supported by a reward model, G-Boost has demonstrated superior performance on benchmark tests, achieving increases of 8.6 percentage points over MCTS and 10.7 percentage points compared to Proxy-Tuning on evaluations like GSM8K and MATH-500.
Key facts
- G-Boost is an adaptive edge-cloud framework for improving SLM performance.
- It does not require parameter updates to the deployed SLM.
- It uses tree search to decide between SLM-only inference and SLM-LLM logit fusion.
- The fusion transfers domain knowledge without exposing private data.
- A process reward model guides Monte Carlo tree search.
- Evaluated on GSM8K and MATH-500 with Qwen2.5 and LLaMA2.
- Outperforms SLM alone, static fusion, and fine-tuned baselines.
- Gains up to 8.6 and 10.7 percentage points over MCTS and Proxy-Tuning.
- Paper announced as replace-cross on arXiv (2503.10367).
Entities
Institutions
- arXiv