LogFloor: Small Models Guide Large-Model Skill Bottleneck Order
A new study shared on arXiv (ID 2608.14936) introduces LogFloor, a closed-loop controller that uses small proxy models to tackle skill bottlenecks in larger models during their training phase. This method, termed 'first-passage skill training,' tracks each skill to meet a target floor, aiming to cut down the tokens needed for all floors. LogFloor hones in on existing bottlenecks with each training round, creating efficient resolution paths. Tests on Qwen2.5-1.5B across five bAbI skill segments demonstrated a 56.2% average reduction in token costs. When moving from a 70M to a 12B model, a three-round replay of a 70M scout path accomplished all floors in eight target runs, achieving savings of 30.9% by pair mean and more.
Key facts
- Paper ID: arXiv:2608.14936
- LogFloor is a closed-loop controller for skill bottleneck ordering.
- First-passage skill training minimizes tokens to reach target floors.
- Qwen2.5-1.5B experiments show 56.2% average token cost reduction.
- 70M-to-12B transfer: three-round replay reaches all floors in eight runs.
- Savings: 30.9% pair mean, 39.4% pooled tokens, 37.6% source-cost.
- MMLU-control: frozen scout path succeeds across all eight 12B runs.
- Published on arXiv.
Entities
Institutions
- arXiv