Measuring Mid-2025 LLM-Assistance on Novice Performance in Biology
New research shows AI assistance provides a 1.4x boost in some tasks but doesn't guarantee complex procedure completion.
Deep Dive
A research team led by Shen Zhou Hong published a pre-registered, randomized controlled trial (n=153) testing if mid-2025 LLMs help novices complete a viral reverse genetics lab workflow. The study found no significant difference in overall completion rates (5.2% LLM vs. 6.6% Internet). However, Bayesian modeling estimated a 1.4-fold increase in success for a typical task, with notable improvement in cell culture (68.8% vs. 55.3%).
Why It Matters
This reveals a critical gap between AI benchmark performance and real-world utility, impacting biosecurity risk assessments.