Mass-Aware Attention: New Mechanism for Evidence Accumulation in Neural Networks

A recent study published on arXiv (2607.22781) presents Mass-Aware Attention (MAA), a novel approach that enhances traditional attention mechanisms by incorporating the quantity of accumulated evidence within neural network representations. The researchers highlight a flaw in conventional attention's weighted averaging: when evidence patterns recur, both the numerator and denominator increase at the same pace, leading to identical aggregates for inputs with varying evidence counts. MAA extends L1 normalization into the Lp family, allowing the numerator and denominator to scale differently upon repetition, thereby maintaining the effective number of inputs contributing to the representation's magnitude. This method does not introduce any additional supervision, parameters, hidden dimensions, or explicit counting features, and reverts to standard attention when p=1. The paper is classified as a cross-type announcement.