Academic Proof-of-Work in the Age of LLMs

A viral LessWrong post by user LawrenceC diagnoses a foundational crisis in academic publishing, particularly in fields like machine learning. The author argues that the system has long relied on 'proof-of-work'—costly, visible effort like meticulously formatted papers, extensive codebases, and polished figures—as a proxy for research quality and seriousness. This acted as a filter, as only researchers confident in their ideas would invest the 30-40+ hours required. The process of writing a paper itself also improved the research. This system, while wasteful, allowed unknown researchers to break in based on demonstrated effort.

The core problem is that Large Language Models (LLMs) like GPT-4 and Claude have demolished the cost of this proof-of-work. What was once expensive—coherent writing, generating complex code for experiments, creating matplotlib figures, and citing literature—is now cheap and fast. The author provides a concrete example: a 'crackpot' paper with dense, potentially incorrect mathematics passed peer review because its AI-generated codebase was expansive and superficially convincing, leading reviewers to blame their own understanding rather than the paper's validity. This collapse of reliable effort signals threatens to flood academia with plausible but low-quality work, potentially forcing a shift to a 'proof-of-stake' model based solely on existing reputation, which could stifle new entrants.