New Study Reveals Fundamental Limits of AI Reasoning Models

A new study analyzing 23 million logic statements reveals fundamental scaling limits for AI reasoning models. Researchers found models using reasoning traces (RTs) excel on broad, shallow tasks but deteriorate on narrow, deep problems requiring longer proofs. The work introduces PITA, a novel benchmark, and concepts of task 'depth' and 'breadth' to predict generalization failure. This identifies inherent limitations in current reasoning paradigms, suggesting not all problems benefit from step-by-step reasoning approaches.