New CoRA method improves LLM reasoning by aligning confidence with rationale
A new RL framework reduces reasoning errors by 26.51% by checking if rationales support answers.
Chain-of-thought reasoning has improved LLM performance, but models often express high confidence even when their rationales are incomplete or poorly supported. This disconnect—called confidence–rationale misalignment—can mislead users into trusting flawed logic. A team of researchers from multiple institutions has proposed CoRA (Confidence-Rationale Alignment), a new framework that forces LLMs to generate rationales that genuinely justify their answers.
CoRA uses GRPO-based reinforcement learning with three reward signals: answer correctness, the probability assigned to the committed answer, and a rubric-based evaluation of the rationale. The rubric assesses grounding in the question, coherence, task match, and connection to the selected answer—all without revealing the correct answer to the judge. Across three QA benchmarks (MedQA, MathQA, OpenBookQA) and three open-weight LLMs, CoRA cut alignment error by up to 26.51% while maintaining competitive accuracy and often improving calibration. This demonstrates that reliable reasoning requires not just confident predictions, but substantiated chains of thought.
- CoRA reduces confidence–rationale alignment error by up to 26.51% compared to supervised fine-tuning and correctness-only GRPO.
- Evaluated on MedQA, MathQA, and OpenBookQA using three open-weight LLMs, with consistent improvements.
- The rubric-based reward evaluates rationale grounding, coherence, task match, and answer connection without using the gold answer.
Why It Matters
CoRA ensures LLM rationales truly support their answers, reducing deceptive overconfidence in critical applications like medical or mathematical reasoning.