Capability Convergence Hypothesis: Access Structure, Not Scale
A recent study published on arXiv introduces the Capability Convergence Hypothesis (CCH), which questions the Platonic Representation Hypothesis (PRH) that suggests similar AI representations lead to identical capabilities. Researchers indicate that capabilities relate to an "access-complete hybrid" category, necessitating a compressive O(1)-state channel alongside a scalable verbatim-index channel. Using the Newton's-apple problem as a case study, the authors identify three significant barriers—Shannon wall, horizon wall, and circuit wall—affecting various AI architectures. They argue that addressing these barriers through hybrid structures can enhance capabilities, highlighting the critical importance of design choices over mere scaling in AI development.
Key facts
- The paper is titled 'The Capability Convergence Hypothesis: Capability from Access Structure, Not Scale' and is available on arXiv with ID 2607.14144.
- It proposes the Capability Convergence Hypothesis (CCH) as a sequel to the Platonic Representation Hypothesis (PRH).
- CCH states that under a fixed per-token inference budget, representational convergence does not entail capability convergence.
- Capability converges toward a class called the access-complete hybrid, which requires both a compressive O(1)-state channel and a scalable verbatim-index channel.
- The hypothesis is anchored on a witness task: the Newton's-apple problem in an infinite stream.
- Three resource walls are identified: a Shannon wall, a horizon wall, and a circuit wall.
- The Shannon wall bars any o(Nb)-state architecture, the horizon wall bars any fixed window, and the circuit wall bars fixed-depth attention-only composition (conditional on TC0 != NC1).
- Under an explicit separability assumption, a hybrid architecture can cross all three walls by paying each wall's price, indicating that capability is strict.
Entities
Institutions
- arXiv