Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Traces

A new study analyzing 23 million logic statements reveals fundamental scaling limits for AI reasoning models. Researchers found models using reasoning traces (RTs) excel on broad, shallow tasks but deteriorate on narrow, deep problems requiring longer proofs. The work introduces PITA, a novel benchmark, and concepts of task 'depth' and 'breadth' to predict generalization failure. This identifies inherent limitations in current reasoning paradigms, suggesting not all problems benefit from step-by-step reasoning approaches.