FutureBridge: New Method for Collaborative LLM-SLM Token Selection
A recent paper published on arXiv (2608.06819) presents FutureBridge, a technique designed to enhance token-level cooperation between large language models (LLMs) and small language models (SLMs). This method tackles a shortcoming in current collaborative decoding strategies, which depend on the LLM's local preferences for token selection, potentially hindering the SLM's reasoning. FutureBridge evaluates combined LLM-SLM token options based on their support for the SLM's reasoning. During training, the LLM trajectory, which is answer-verified, establishes a consistent shared future, while the SLM assesses each candidate within this framework. The resulting counterfactual scores guide a lightweight token reranker that focuses solely on the current state and candidate token. At inference, FutureBridge employs the LLM to broaden the candidate pool, chooses one token, and returns the generation to the SLM without creating or adding extra tokens. This method, developed by researchers, aims to improve the efficiency and effectiveness of collaborative decoding in natural language processing.
Key facts
- FutureBridge is a method for token-level collaboration between LLMs and SLMs.
- It ranks joint LLM-SLM token candidates based on support for SLM's subsequent reasoning.
- Training uses an answer-verified LLM trajectory as a fixed shared future.
- A frozen SLM evaluates candidates under the common context.
- Counterfactual scores supervise a lightweight token reranker.
- At inference, LLM expands the candidate pool, then SLM generates.
- The paper is available on arXiv with ID 2608.06819.
- It was announced as a cross-type submission.
Entities
Institutions
- arXiv