HodgeCover: Topological Compression for Sparse Mixture-of-Experts
A new method called HodgeCover addresses a fundamental obstruction in compressing Sparse Mixture-of-Experts (MoE) layers. The obstruction arises when three experts are pairwise compatible but form an irreducible cycle when merged, making pairwise ranking blind to triple compatibility. The authors identify this as the harmonic kernel of the simplicial Laplacian on a 2-complex, where vertices are experts, edges carry KL merge barriers, and faces carry triplet barriers. HodgeCover greedily selects edges and triangles to cover harmonic-critical structures, enabling effective compression without retraining. The paper is available on arXiv under identifier 2605.13997.
Key facts
- HodgeCover is a learning-free compression method for Sparse Mixture-of-Experts layers.
- It addresses a topological obstruction where three experts are pairwise compatible but not jointly mergeable.
- The obstruction is formalized as the harmonic kernel of the simplicial Laplacian on a 2-complex.
- The method greedily covers harmonic-critical edges and triplet-critical triangles.
- The paper is published on arXiv with ID 2605.13997.
- The approach reduces inference cost without retraining.
Entities
Institutions
- arXiv