Meet Dude: An AI That Checks If Research Papers Really Match Their Code

Research papers are supposed to include code so others can verify results. But with submissions growing faster than reviewers can read them, many papers slip through with code that doesn't actually do what the paper claims. That matters because bad papers can send other scientists down dead ends or even influence policies based on shaky evidence.

That's the problem Dude tackles. Dude is an AI system that compares a paper's written claims with its actual code, line by line. Instead of using one AI reviewer, Dude runs several specialized agents — each looks at the problem from a different angle, like a team of detectives. This approach lets the system spot mismatches that a single AI would miss.

The tricky part is avoiding false alarms. A paper describes ideas in broad language, while code deals in rigid details. That mismatch can make an AI think there's a problem when there isn't. Dude solves this with a "negotiation" step, where agents talk it out and agree on what's truly wrong, plus a filter that removes low-confidence issues.

In testing on real paper-code pairs, Dude found 22.8% more true discrepancies than existing methods while also reducing false positives. What does that mean for the average person? Trust in published science takes a hit every time a paper can't be reproduced. Tools like Dude could help restore that trust by making sure results are honestly reported — so the next breakthrough you read about is more likely to be real.