Model Merging: Combining Neural Networks in Weight Space

A new thesis on arXiv (2605.01580) proposes model merging as an alternative to training separate neural networks. The approach combines independently trained networks directly in weight space without requiring original training data or extensive optimization. In the single-task setting, the thesis introduces C$^2$M$^3$, a cycle-consistent merging algorithm based on Frank-Wolfe optimization that aligns multiple networks into a shared parameter space. For multi-task settings, where models are fine-tuned from a common initialization, a theoretical framework is developed. The work challenges the conventional paradigm of treating models as isolated artifacts.