MACRO: Markov Chain Routing for Efficient Transformer Layer Execution
A novel framework called MACRO (Markov Chain Routing of Transformer Layers) has been introduced to enhance the performance of Large Language Models (LLMs) through dynamic layer routing. Unlike traditional LLMs that process layers in a linear fashion, MACRO enables the learning of task-specific pathways without altering the model's parameters. It conceptualizes layer routing as a context-sensitive Markov policy, influenced by layer indices, computation budget phases, directional changes, and operator context, facilitating operations like skip, repeat, and residual hidden-state addition. The Markov route distribution is refined using feedback from training data and is decoded with a top-k Viterbi algorithm to pinpoint high-probability routes. This method seeks to boost efficiency without necessitating weight updates, costly per-instance search loops, or ground-truth labels during inference. The research can be found on arXiv with the identifier 2608.05872.
Key facts
- MACRO stands for Markov Chain Routing of Transformer Layers.
- It is a framework for dynamic layer routing in LLMs.
- It does not modify underlying model parameters.
- It supports skip, repeat, and residual hidden-state addition operations.
- The routing is modeled as a context-dependent Markov policy.
- It uses a top-k Viterbi algorithm for decoding.
- The paper is on arXiv with ID 2608.05872.
- The approach avoids updating weights, expensive search loops, and ground-truth labels during inference.
Entities
Institutions
- arXiv