FirstResearch framework makes LLM scientific questions auditable with certificates

Yufeng Wang's FirstResearch framework addresses a critical gap in LLM-driven scientific discovery: the inability to audit the first research question an agent proposes. The system's core artifact is a structured Research Question Certificate that explicitly records primitive definitions, key assumptions, a mechanism model, a tension or contradiction, a falsifiable hypothesis, a minimal decisive test, and a failure update rule. This makes the reasoning behind any proposed question fully inspectable before downstream execution—whether literature synthesis, experiment planning, or report generation.

In rigorous evaluations using both a primary DeepSeek-blind-judge protocol and a Gemini-2.5-Flash independent rescore across 40 baseline packages from AI co-scientist, Agent Laboratory, and AI Scientist-v2, FirstResearch achieved the highest score of 4.86/5 versus 4.38/5 for the best baseline. The judges showed strong agreement (Pearson r=0.865). Ablation studies further confirmed the certificate is the strongest component: removing it crashed scores below 1/5 under both judges. While results are preliminary and use LLM judges rather than human domain experts, the framework demonstrates that explicit derivation constraints are a promising mechanism for making LLM-generated scientific questions more auditable.