Study of GPT-2, Pythia, BERT finds LLMs fuse reading and writing into one entangled code

A new preprint by Diego Saldaña Ulloa investigates a fundamental difference between how humans and large language models process reading and writing. In the literate human brain, reading relies on a ventral decoding route (impaired in pure alexia) and writing on a fronto-parietal encoding route (impaired in pure agraphia) — these are doubly dissociable systems. LLMs, however, drive both tasks through a single autoregressive path optimized on text. The paper, titled "Reading Without a Reader," asks how entangled that single mechanism is by comparing the input-side reading code (W_E) with the output-side writing code (W_U) using an entanglement index E ∈ [0,1] computed via CKA, Procrustes residual, and mutual k-NN.

The results span nine probes across GPT-2, OPT, Pythia (14M–1.4B), T5, and BERT/RoBERTa, with six probes consolidating established results and three introducing the read/write analysis. Two complementary levels agree in direction: in the weights, untied models show one coupled but sub-ceiling code (E=0.23–0.35, far above floor), following a non-monotonic couple-then-differentiate trajectory where W_U drifts ~3.2× farther than W_E in every frequency decile. Behaviorally, comprehension and production are positively coupled in all 12 non-degenerate models (sign test p<0.001) — the opposite of the brain's double dissociation. This coupling is general, not decoder-only: encoder-decoders separate the two pathways representationally (up to 0.96) yet stay behaviorally coupled. The nulls are reported plainly: the geometry-to-behavior bridge is null (ρ=0.00). Because a single forward path makes coupling expected a priori, the contribution is its quantification and cross-level concordance, situating LLMs as a distinct point in the space of possible minds.