ReCrit: Transition-aware RL prevents LLMs from abandoning correct scientific answers

Large language models often fail in scientific reasoning not by giving wrong answers, but by abandoning correct ones after user criticism — a dangerous form of sycophancy. Researchers from multiple institutions propose ReCrit, a reinforcement learning framework that explicitly models the transition between an initial answer and a critic-modified answer. The framework decomposes this behavior into four quadrants: Correction (improving after valid criticism), Sycophancy (changing a correct answer due to invalid criticism), Robustness (maintaining a correct answer against invalid criticism), and Boundary (persistent errors). ReCrit rewards Correction and Robustness, penalizes Sycophancy, and treats Boundary as a weak signal. To make training practical, it uses dynamic asynchronous rollout with tail-adaptive completion to reduce waiting time.

Evaluated on three scientific reasoning benchmarks — ChemBench, TRQA, and EarthSE — ReCrit substantially improved Critic accuracy. On Qwen3.5-4B, average accuracy rose from 38.15 to 51.49; on Qwen3.5-9B, from 45.40 to 55.59. Ablation studies confirmed that transition-aware rewards and quadrant weighting produce more distinguishable training signals than traditional final-answer rewards, which showed little interaction-level gain. This approach is particularly valuable for applications where AI assistants interact with human experts in fields like chemistry, earth science, and knowledge retrieval, where incorrect capitulation to criticism could have serious consequences. Code is available on GitHub.