CRDTMergeState: A Two-Layer Architecture for Conflict-Free Neural Network Merging

A recent study published on arXiv (2605.19373v1) indicates that none of the 26 neural network merging techniques assessed—including weight averaging, SLERP, TIES, DARE, Fisher merging, and evolutionary methods—fulfill the necessary algebraic properties (commutativity, associativity, idempotency) for conflict-free distributed operations. The researchers demonstrate that this issue is structural, as normalization-based merges cannot satisfy all three properties simultaneously. To resolve this, they introduce CRDTMergeState, a dual-layer framework that encapsulates any merging strategy within a Conflict-Free Replicated Data Type (CRDT) layer. The first layer utilizes OR-Set CRDT semantics for managing contributions, ensuring trivial commutativity, associativity, and idempotency through set union. The second layer employs deterministic pure functions for merging over a canonically ordered contribution set, with randomness derived from the Merkle root. The authors establish that this separation ensures Strong Eventual Consistency, allowing all replicas receiving identical updates to converge on the same state. This research lays a formal groundwork for the merging of distributed neural network models.