Transformers' Turing-Completeness Depends on Context Management
A new paper on arXiv (2605.19514) clarifies that claims of Transformers being Turing-complete often conflate two settings: a fixed-system setting where a single autoregressive Transformer uses a fixed context-management method, and a scaling-family setting where different models handle varying input lengths. The authors argue that real-world LLM deployment aligns with the fixed-system setting, while existing proofs of Turing-completeness typically apply to the scaling-family setting. They formalize the fixed-system setting to characterize how LLMs actually operate, noting that results from the scaling-family setting provide limited theoretical support for the fixed-system case.
Key facts
- arXiv paper 2605.19514 distinguishes two settings for Transformer Turing-completeness.
- Fixed-system setting: a single autoregressive Transformer with fixed context management.
- Scaling-family setting: a family of models with varying context windows or precision.
- Existing proofs of Turing-completeness often rely on the scaling-family setting.
- Real-world LLM deployment corresponds to the fixed-system setting.
- The paper formalizes the fixed-system setting for LLM operation.
- Results from scaling-family setting provide limited support for fixed-system claims.
- The paper argues that the conflation of settings leads to misleading claims.
Entities
Institutions
- arXiv