Vector Network: A New Architecture for Compositional Learning
A new hierarchical recurrent framework called the Vector Network (VN) has been developed by researchers, which substitutes fixed weight matrices in deep learning models with collections of reusable rank-1 weight atoms. For each input, VN optimizes a layer-specific energy to determine a sparse selection of active weight atoms and their corresponding coefficients, adhering to constraints from bottom-up input reconstruction and top-down feedback consistency. These coefficients form a low-rank weight matrix tailored to the input. Following convergence, only the chosen weight atoms are updated through slow learning, utilizing local residual signals adjusted by the inferred coefficients. The VN has been tested across four compositional benchmarks involving 1D signals and 2D data. The paper can be found on arXiv with the identifier 2605.28007.
Key facts
- Vector Network (VN) is a hierarchical recurrent architecture.
- Each layer has a library of reusable rank-1 weight atoms.
- VN minimizes a layer-local energy to infer active weight atoms and coefficients.
- Constraints include bottom-up input reconstruction and top-down feedback consistency.
- Inferred coefficients compose an input-specific low-rank weight matrix.
- Slow learning updates only selected weight atoms.
- Updates use local residual signals scaled by inferred coefficients.
- Evaluated on four compositional benchmarks (1D signals, 2D data).
Entities
Institutions
- arXiv