LLMs Collapse Reading and Writing into a Single Entangled Code
A recent investigation published on arXiv (2607.24797) explores how decoder-only large language models (LLMs) merge the distinct human cognitive functions of reading and writing into a unified mechanism. In humans, these functions operate as two separable systems: a ventral decoding pathway (affected in pure alexia) and a fronto-parietal encoding pathway (impaired in pure agraphia), both sharing a common orthographic foundation. In contrast, LLMs utilize a single autoregressive pathway optimized for text, a relatively new cultural construct. The study contrasts an input-side 'reading code' W_E with an output-side 'writing code' W_U through an entanglement index E ranging from [0,1], employing methods like CKA and Procrustes residual. Analyzing nine probes across models such as GPT-2, OPT, Pythia (14M–1.4B), T5, and BERT/RoBERTa reveals that two complementary levels align in direction. In the weights, untied models exhibit one coupled but sub...
Key facts
- Study on arXiv 2607.24797 examines reading and writing in LLMs vs. humans
- Human reading and writing are doubly dissociable: ventral decoding route (pure alexia) and fronto-parietal encoding route (pure agraphia)
- Decoder-only LLMs use a single autoregressive path for both reading and writing
- Entanglement index E ∈ [0,1] measured via CKA, Procrustes residual, mutual k-NN
- Calibrated against independent-init floor and tied ceiling
- Nine probes on GPT-2, OPT, Pythia (14M–1.4B), T5, BERT/RoBERTa
- Six probes consolidate established results, three introduce read/write analysis
- Two complementary levels agree in direction
Entities
Institutions
- arXiv