torch-sla: Differentiable Sparse Linear Algebra Library for PyTorch
There's a new open-source library called torch-sla that just came out, and it's designed to bring differentiable sparse linear algebra to PyTorch. This tool has an autograd-aware API, which makes it easy to use various solvers, including iterative, nonlinear, and eigenvalue types, across five backends: SciPy, Eigen for CPUs, and cuDSS, CuPy, or even a native PyTorch solver for GPUs. It allows for batch solving with either shared or unique sparsity patterns and can run across multiple GPUs using domain decomposition. Plus, it features a scalable O(1)-graph adjoint differentiation system and an autograd-compatible distributed halo-exchange layer. You can check it out on arXiv under the reference 2601.13994.
Key facts
- torch-sla is an open-source library for differentiable sparse linear algebra in PyTorch
- It provides a single autograd-aware API for multiple solver types
- Supports five interchangeable backends: SciPy, Eigen, cuDSS, CuPy, and PyTorch-native iterative solver
- Automatic dispatch by device and problem size
- Batched solves over shared or distinct sparsity patterns
- Distributed multi-GPU execution via domain decomposition with halo exchange
- O(1)-graph adjoint differentiation framework for scalability
- Autograd-compatible distributed halo-exchange layer
Entities
Institutions
- PyTorch
- SciPy
- Eigen
- cuDSS
- CuPy
- arXiv