Neural Network Feature Ownership Controlled by Scaling Gauge

A recent study published on arXiv (2608.06766) examines the regulation of feature specialization in overparameterized ReLU networks through a parameter that is not visible to the initial predictor. Utilizing a Gaussian teacher-student framework, the researchers maintained the complete initial function while varying a positive-homogeneous scaling gauge. Their findings revealed that different gauges lead to unique feature trajectories and a significant Θ(D²) difference in specialization time, which cannot be accounted for by any global time alteration. When starting with identical students, designating the advantageous gauge to a single neuron deterministically identifies it as the owner, reducing the functional contribution of others to zero. This research introduces 'feature ownership' and indicates that a straightforward gauge selection can influence which neuron adopts a teacher feature, offering insights into the manipulation of neural network internal representations.