LessWrong Contest Tests if Human Prompting Still Matters in an ASI World
A $1,000 essay contest probes whether skilled prompters can outperform domain experts with AI.
A provocative LessWrong post by user wp grapples with a question looming over the AI field: what role is left for intelligent humans when artificial superintelligence (ASI) arrives? Using chess as an analogy, wp notes that superhuman AIs like Stockfish have turned human players into passive consumers of intelligence, with little benefit from incremental AI improvements. In programming, the picture is more nuanced: experienced programmers still extract superior results from LLMs, acting as a multiplier for AI skill. For mathematics, evidence is mixed — some breakthroughs come from nonspecialists with deep pockets (e.g., Anthropic's AES attacks), while others require domain expertise (e.g., OpenAI's cycle double cover conjecture).
To cut through the speculation, wp designed a concrete experiment: a $1,000 contest challenging participants to produce an academic humanities essay on AI safety and Kazuo Ishiguro's *Klara and the Sun*. The scoring uses a hand-crafted 176-item binary rubric, auto-scored for reproducibility. The contest aims to answer whether skilled LLM operators without humanities expertise can surpass domain experts, or if deep subject knowledge remains essential. Baseline LLM essays are provided for comparison. The results could illuminate how much human guidance adds as AI capabilities grow — a crucial data point for professionals navigating the post-AGI transition.
- The contest uses a 176-item binary rubric to auto-score academic essays on *Klara and the Sun*, with a $1,000 prize.
- Chess analogy: superhuman AIs have turned humans into intelligence consumers, not producers.
- In programming, experienced humans act as 'multipliers' for LLMs, suggesting a continuing role even under ASI.
Why It Matters
Professionals must reassess their value-add as AI surpasses human ability; this experiment tests where guidance still pays off.