Fast LapSum: Exact Differentiable Top-k at Million Scale
A recent publication on arXiv presents Fast LapSum, a soft top-k primitive that adheres to exact-budget requirements for extensive sparse computations. The top-k function is crucial in contemporary machine learning applications, including tasks like token routing, expert activation, memory selection, and attention pruning. Traditional hard top-k methods obstruct gradients, while current continuous relaxations are costly for large-scale models. Fast LapSum overcomes this challenge by offering a GPU solver that operates in linear time post-sorting. Distinct from earlier linear-time approaches like DFTopK, which relax normalization constraints, Fast LapSum uniquely maintains an exact selection mass of k and is fully differentiable throughout. The solver utilizes linear-time threshold calculations and an analytical vector-Jacobian product, employing probabilistic bracketing for sorting uncertain kernel-noised scores. The overhead remains minimal, efficiently handling up to 10^7 elements. The paper can be found on arXiv under ID 2608.06912.
Key facts
- Fast LapSum is an exact-budget soft top-k primitive.
- It runs in linear time after sorting on GPU.
- It is fully differentiable end-to-end.
- It preserves an exact selection mass of k.
- It uses a linear-time threshold computation and analytical vector-Jacobian product.
- For extreme scales, it uses probabilistic bracketing to sort only the uncertain middle band.
- The solver processes 10^6 and 10^7 elements with negligible overhead.
- The paper is on arXiv with ID 2608.06912.
Entities
Institutions
- arXiv