PALS: Power-Aware LLM Serving for Mixture-of-Experts Models

Researchers have introduced a novel system named PALS (Power-Aware LLM Serving), which utilizes GPU power limits as a manageable asset to enhance energy efficiency during large language model inference. Integrated into the vLLM framework, PALS merges simple offline power-performance models with a feedback-based controller to choose settings that achieve throughput goals while reducing energy use. This system does not necessitate any model retraining or alterations to APIs and has been validated on multi-GPU setups using both dense and mixture-of-experts (MoE) models, showing significant gains in energy efficiency.