Hyperspherical Forward-Forward Algorithm Boosts Speed 40x
The Hyperspherical Forward-Forward (HFF) algorithm has been unveiled by researchers as a new version of the Forward-Forward (FF) algorithm, addressing its costly inference process. The traditional FF algorithm necessitated an individual forward pass for each class during inference, leading to inefficiency. In contrast, HFF transforms the local objective of every layer from a binary goodness-of-fit challenge to a multi-class classification task within a hyperspherical feature space. It develops class-specific, unit-norm prototypes that serve as geometric anchors and implicit negatives. This advancement maintains local training advantages while allowing for weight updates and inference in a single forward pass, resulting in over a 40x speed enhancement compared to the original FF algorithm. The method is easy to implement and scalable, with the research published on arXiv under identifier 2605.00082.
Key facts
- HFF reformulates the Forward-Forward algorithm.
- Original FF required a separate forward pass per class during inference.
- HFF uses a hyperspherical feature space for multi-class classification.
- Class-specific prototypes serve as geometric anchors.
- HFF enables single-pass weight update and inference.
- Speed improvement is over 40x compared to original FF.
- Method is simple to implement and scalable.
- Published on arXiv with ID 2605.00082.
Entities
Institutions
- arXiv