Research & Papers

Can we automatize scientific discovery in the cognitive sciences?

A new framework uses LLMs to automate every step of scientific discovery, from designing experiments to generating theories.

Deep Dive

A team of researchers including Akshay K. Jagadish, Milena Rmus, and Eric Schulz has proposed a radical new framework to fully automate scientific discovery in cognitive science. Published in the paper "Can we automatize scientific discovery in the cognitive sciences?" (arXiv:2603.20988), the system aims to replace the slow, intuition-bound manual research pipeline. Traditionally, cognitive scientists develop paradigms, collect data, and test predefined model classes in a cycle constrained by human pace and background knowledge.

The proposed framework implements every stage of the discovery cycle using Large Language Models (LLMs). First, an LLM directly samples experimental paradigms that explore conceptually meaningful task structures. Next, high-fidelity behavioral data is simulated using foundation models of cognition, bypassing the need for initial human subject studies. The tedious process of handcrafting cognitive models is replaced by LLM-based program synthesis, which performs a high-throughput search over a vast landscape of algorithmic hypotheses.

Finally, the discovery loop is closed by optimizing for 'interestingness'—a metric of conceptual yield evaluated by an LLM-critic. This creates a self-contained, automated engine that rapidly surfaces informative experiments and potential cognitive mechanisms. The output is designed for subsequent validation in real human populations, positioning the system as a powerful hypothesis-generation tool rather than a final arbiter of truth. This represents a significant step toward scalable, in-silico theory development that could dramatically accelerate progress in understanding intelligence.

Key Points
  • Replaces the entire manual research pipeline (paradigm design, data simulation, model synthesis) with LLM-driven automation.
  • Uses LLM-based program synthesis for high-throughput search over algorithmic hypotheses, moving beyond predefined model classes.
  • Closes the discovery loop by optimizing for 'interestingness' via an LLM-critic, creating a self-contained in-silico engine.

Why It Matters

This could dramatically accelerate theory development in cognitive science by automating hypothesis generation before costly human experiments.