LLM theory-to-code trial: renderer format has no effect on programs

Andre Panossian's arXiv paper (2608.10314) asks whether the way a theory is written—structured contracts versus connected prose—systematically changes the model an LLM builds from it. The study used a prospective preregistered design with 16 independent assignment bits allocating 32 paired renderer slots. Two pinned LLM snapshots translated five anonymous theoretical accounts into 320 single-shot programs in a frozen sparse quadratic language, and a deterministic evaluator measured finite-difference and interaction responses. The primary endpoints were cross-model matched-distance reduction and same-account identifiability.

Results were decisive: both hypotheses returned NOT_SUPPORTED, with only 19 of 108 preregistered criteria passing. Same-account identifiability hovered near chance (AUC 0.469–0.523, against a registered 0.80 threshold), and while one H2 matched-distance endpoint survived multiplicity correction in the signed-linear pipeline, it missed the effect-size floor in magnitude-rank analysis. This places a concrete boundary on how much renderer format influences LLM theory-to-program translation, and the author released fully auditable datasets and software for future work.