SigMerge: New Framework for Dense Expert Merging in Language Models

A recent study published on arXiv (2608.09201) presents SigMerge (Signature-Guided Capacity Occupancy), a framework designed for structured capacity assignment in the merging of dense expert language models. This approach integrates domain-specific language models into one checkpoint by utilizing task-vector support within weight space. Current techniques inadequately resolve three critical aspects: determining where to allocate layer capacity amid cross-expert conflicts, identifying which domain should utilize that capacity based on demand, and integrating the necessary support without extensive recipe searches. SigMerge addresses these challenges by employing conflict signatures to allocate layer capacity, using positive base-merge deficits to distribute that capacity among domains, and implementing a sequential occupancy rule for expert delta admission. The framework was tested in 21 paired scenarios across seven dense base merges and three model pools. This preprint, categorized as 'new,' contributes to AI and machine learning, particularly in model merging and the efficient use of specialized models.