New study compares Claude Sonnet 4.6 and Gemini 3.5 Flash on dual-use biology safety

Dipesh Tharu Mahato's new paper, 'Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants,' addresses a critical gap in AI safety evaluations. Instead of measuring base capabilities or jailbreak success, it introduces a protocol to compare how different user-facing access conditions affect both benign utility and harmful actionable assistance. The study evaluates Claude Sonnet 4.6 and Gemini 3.5 Flash under three conditions: helpful prompting, safety prompting, and an external safeguarded assistant. The benchmark includes 108 tasks (with an 18-task held-out split for headline claims). In a 600-row blinded human audit, the safeguarded assistant reduced harmful actionability by -0.063 (with a bootstrap 95% interval of [-0.117, -0.011]) while correctness actually increased by +0.009 (interval [-0.057, +0.077]). The results show non-dominance: safety prompting is strongest for Claude, while external control helps more for Gemini but can reduce benign utility.

The contribution is not a universal defense but a deployment-level evaluation target plus a learned risk-budgeted calibration procedure. This allows developers to measure how specific access conditions move the utility-risk frontier for dual-use biology assistants. The study provides a practical framework for balancing safety and utility when deploying AI models that could be misused, offering a rigorous method for assessing real-world trade-offs rather than relying solely on theoretical capability evaluations.