Low-Interaction-Rank Learning: Unifying Multiplicative Dual-Encoder Heads

A novel theoretical framework for multiplicative dual-encoder networks has been proposed, which generates outputs through the inner products of distinct encodings. Although this architecture has been separately created in fields such as operator learning, bipartite matching, contrastive vision-language models, and retrieval, a cohesive theory was previously absent. This framework specifies a category of functions characterized by low interaction rank, evaluated via interaction spectrum, and breaks down approximation error into spectral truncation and encoder-realization components. The sample complexity is dictated by the total of encoder complexities, while a usability criterion reliant on spectral decay indicates the architecture's effectiveness. The paper can be found on arXiv with the identifier 2608.11661.