FraQ: Efficient Coordinate-Space Recompression for Federated LoRA

There’s a new study on arXiv (2608.03605) that introduces a method called FraQ, which helps with coordinate-space recompression in federated Low-Rank Adaptation (LoRA). This is a big deal because it allows for the collaborative fine-tuning of Large Language Models (LLMs) without needing to gather sensitive data in one place. However, LoRA’s two-factor setup can create problems when clients try to aggregate their data, as just averaging the factors doesn’t reflect the true updates. To resolve this, FraQ calculates the exact aggregate in the full weight space before recompressing it. It starts with stacked factors, factorizes them, picks the necessary rank based on a specific energy threshold, and converts those back into LoRA factors for better efficiency in federated learning.