Sutra: A Programming Language That Compiles to PyTorch Neural Networks
Researchers have created a new programming language called Sutra. It's a typed and purely functional language that compiles into a PyTorch neural network during its forward pass. The compiler integrates the whole program—including its primitives and control flow—into a single tensor-op graph built on a fixed embedding substrate. Operations like rotation binding, unbind, and polynomial Kleene logic are converted into tensor operations, with Kleene connectives shown as Lagrange-interpolated polynomials that accurately reflect the {-1, 0, +1} truth grid. To validate its effectiveness, two tests were conducted: one involved running the same program on four frozen embeddings across two modalities, achieving 100% accuracy with width k=8, and the other confirmed PyTorch autograd processed the computations correctly.
Key facts
- Sutra is a typed, purely functional programming language.
- Its compiled forward pass is a PyTorch neural network.
- The compiler beta-reduces the whole program to one fused tensor-op graph.
- Operations include rotation binding, unbind, bundle, polynomial Kleene three-valued logic, and tail-recursive loops.
- Kleene connectives are Lagrange-interpolated polynomials exact on the {-1, 0, +1} truth grid.
- Validation tested the same program on four frozen embeddings: nomic-embed-text, all-minilm, mxbai-embed-large, and ESM-2.
- Decoding achieved 100% accuracy through width k=8 on every substrate.
- Hadamard product collapsed to 2.5% on mxbai-embed-large and 7.5% on all-minilm.
Entities
Institutions
- arXiv