Universal Activation Bus: A Shared Interface for Language Models
The Universal Activation Bus, a newly introduced framework, establishes a unified activation interface for compatible language models, facilitating the sharing of activation-based tools. This approach derives a collective dense space from a limited number of source models, utilizing a lightweight linear encoder-decoder adapter for each model. Once training is complete, the interface remains fixed, allowing new models to integrate by adjusting only their adapter pairs on unlabeled matched texts. Consequently, tools such as probes, sparse autoencoders (SAEs), and natural-language interpreters can be reused across different models. Experiments conducted with five models demonstrate that semantically related texts consistently cluster in the shared space, enabling an onboarded model to utilize tools developed for another model. The paper can be found on arXiv under ID 2608.09521.
Key facts
- The Universal Activation Bus provides a common activation interface across compatible language models.
- It uses a small set of source models to learn a shared dense space.
- Each model has one lightweight linear encoder-decoder adapter pair.
- After source training, the interface is frozen; new models fit only their adapter pair on unlabeled matched text.
- Activation-based tools such as probes, SAE features, and natural-language interpreters can be shared.
- Tests across five models show consistent neighborhoods for semantically related texts.
- An onboarded model can reuse tools originally trained for a different model.
- The paper is available on arXiv with ID 2608.09521.
Entities
Institutions
- arXiv