Personalized Federated Sparse Adaptation of Time-Series Foundation Models

A novel framework has been introduced for the personalized federated learning of time-series foundation models (TSFMs), specifically aimed at forecasting energy usage in buildings. This approach tackles the issue of modifying pretrained TSFMs for distributed, private, and non-IID meter data from various buildings. It features a heterogeneous temporal mixture-of-experts (MoE) adapter positioned after the pretrained TSFM representation, utilizing a sequence-level router that assigns each 168-hour context window to a select group of top-k experts focused on periodicity, long-range interactions, local variations, trend-residual structures, and multi-resolution behaviors. The research evaluates global federated learning (FL), local training, and personalized FL methods with either globally shared or client-specific expert banks, based on experiments conducted in 50 buildings, showcasing the success of the personalized sparse adaptation technique.