Federated Nested Learning Framework for Test-Time Adaptation

A novel machine learning framework named Federated Nested Learning (FedNL) has been introduced to tackle the challenges posed by non-IID client data in federated learning. This framework reinterprets federated learning as a three-tier nested optimization model, incorporating Titans-based linear attention to facilitate lightweight, zero-shot test-time adaptation through a delta rule as an online gradient step. Tests conducted on non-IID MMLU and long-context benchmarks demonstrate strong performance in short-context reasoning, enhanced long-context retrieval, and consistent streaming cross-entropy, along with stable inference memory. The research paper can be accessed on arXiv.