DeaMoE: Efficient MoE Architecture for Fast Small-Batch Decoding

A new paper on arXiv (2608.14385) introduces DeaMoE, a decoding-efficient Mixture-of-Experts (MoE) architecture designed to address the memory-bound bottleneck in small-batch decoding, which is critical for real-time interactive applications like coding assistants and audio-video interaction systems. The authors note that existing solutions, such as post-training weight compression or fine-grained expert design, either degrade accuracy or add overhead. DeaMoE groups experts into departments, where experts in the same department share most parameters due to their professional field overlap. The paper is announced as a cross-type submission, indicating it may have been presented elsewhere. The research aims to improve inference speed without sacrificing model quality, potentially benefiting latency-sensitive applications.