Bicameral Model: Neural Interface Between Parallel Language Models
A team of researchers has rolled out the Bicameral Model, which combines two static language models via a trainable neural interface that zeroes in on hidden states. In this setup, both models work together seamlessly: the main model tackles the core task, while the secondary model deals with tools, constraints, or code execution. They communicate through a translation network and a learned suppression gate, which only makes up about 1% of all parameters. This gate creates a selective communication method based on task loss, without any preset format. When tested with arithmetic, connecting two 0.5B models to a calculator raised accuracy dramatically from 36% to 96%. You can check out their findings on arXiv (2605.11167).
Key facts
- Two frozen language models are coupled through a trainable neural interface on intermediate hidden states.
- Both models run in lockstep at every generation step.
- A primary model drives the task while an auxiliary model operates tools, solves constraints, or executes code.
- Conditioning on each other's activations via a translation network and a learned suppression gate.
- The suppression gate uses ~1% of combined parameters.
- The gate learns a selective communication protocol from task loss alone.
- On arithmetic, coupling two 0.5B models with a calculator raises accuracy from 36% to 96%.
- Paper published on arXiv with ID 2605.11167.
Entities
Institutions
- arXiv