FedSLM: A New Framework for Federated Fine-Tuning of Foundation Models with Heterogeneous Clients

A recent paper on arXiv (2607.29071) presents FedSLM, a framework focused on parameters to tackle the issue of resource asymmetry in federated learning for foundation models. The primary challenge is that organizations with the most valuable domain-specific datasets often lack the capacity to host models with billions of parameters. Current heterogeneous federated methods, including parameter-efficient tuning, model pruning, and knowledge distillation, compromise essential attributes such as full-model memory efficiency, architectural self-sufficiency, or representational accuracy. FedSLM utilizes SVD-based decomposition to develop self-contained client models with low-rank subspaces that create nested manifolds, facilitating structural compatibility for aggregation. It implements a two-stage protocol to synchronize lightweight adapters and merge full-rank reconstructions. This paper suggests it may have been shared or published previously. The framework seeks to allow effective federated fine-tuning while maintaining model quality and architectural integrity.