Pruning 75% of Experts from MoE LLMs for Translation with Minimal Loss

A new method aggressively prunes experts from mixture-of-experts large language models to create efficient translation specialists. The approach exploits expert specialization and separable multilingual capabilities to identify and remove translation-irrelevant experts without retraining. Pruning half of all experts yields negligible degradation, 70% pruning causes only minor losses, and 75% pruning with short supervised fine-tuning recovers baseline performance. This drastically reduces memory and compute requirements for translation tasks.