Unsupervised Adaptation of PDE Foundation Models via NSLoRA

A new arXiv preprint (2608.07053) introduces an unsupervised finetuning framework for partial differential equation (PDE) foundation models, enabling adaptation to unseen PDE systems without requiring ground-truth solution data. The method first pretrains a neighborhood attention Transformer on diverse time-dependent PDEs across varying spatial scales, learning transferable representations. During adaptation, a physics-based objective using PDE residuals and boundary conditions is constructed, and the model is finetuned via low-rank adaptation (LoRA). To address uneven learning across physical quantities, the authors propose NSLoRA, a Newton-Schulz orthogonalized variant that rebalances adaptation. The approach achieves performance comparable to supervised methods, as reported in the abstract. The work addresses the high cost of dense solution data, offering a more practical path for applying PDE foundation models to new equations.