AI That Solves Physics Can Be Tricked Into Giving Plausibly Wrong Answers

Scientists increasingly rely on neural PDE operators — AI models that predict how fluids, heat, and forces behave without running slow traditional simulations. These models are trained on vast, shared archives of solutions, and researchers usually trust them if their everyday predictions look accurate. A new paper introduces a disturbing attack: by slipping poisoned examples into those archives, an attacker can plant a hidden trigger. When the trigger appears, the AI silently switches to a solution that is physically plausible but corresponds to the wrong settings — imagine a weather model that calmly "predicts" sunshine when a storm is actually coming.

This attack is called a wrong-physics backdoor. It works because the attacker relabels training data so the model learns to associate a specially crafted input with a valid solution from a different scenario. The researchers tested this across famous equation types like Burgers' equation and the Navier-Stokes equations used in airplane design and ocean modeling. Their method achieved a stunning 100% success rate on two of the hardest cases, while the models still passed routine accuracy checks and looked completely healthy to standard validation.

The danger is that these AI models are already used as fast stand-ins for expensive physics simulations. If a malicious actor — or even an accidental data mix-up — plants such a backdoor, engineers might make decisions based on confident, physically realistic, yet completely wrong outputs. The paper's authors argue that today's validation methods, which check smoothness or average error, are not enough. You must verify the exact physical conditions the AI believes it is solving for, not just that the output looks plausible.

For everyday readers, this is a reminder that "AI for science" carries serious trust risks. It's not just about biased language models or fake images; core infrastructure like climate models, medical device simulations, and bridge safety analyses could be quietly compromised. The research offers no simple fix yet, but it shines a light on a hidden weakness in the growing world of AI-powered science.