DS@GT ARC's LLM Verifier Outperforms Reward Model for Numerical Claims

Researchers from DS@GT ARC (Georgia Tech) tackled automated verification of numerical claims in English and Arabic as part of the CLEF 2026 CheckThat! Task 2. They developed two approaches: an LLM-based verifier fine-tuned with LoRA (Low-Rank Adaptation) that scores each reasoning trace independently via binary classification, then selects the final verdict using Best-of-N selection. They also experimented with adaptive sub-claim decomposition to break complex claims into simpler parts. The second approach uses a lightweight TF-IDF reward model with handcrafted features for numeric and temporal overlap to score traces, then aggregates scores by verdict group to determine the final prediction.

Results show the LLM-based approach outperforms the reward model on most metrics, particularly Recall@5 (a measure of how often the correct trace appears in the top five). However, the reward model performed better on the Conflicting claim class. Sub-claim decomposition did not improve performance—claim splitting introduced noise rather than aiding reasoning. For Arabic, the team compared a general multilingual model against AraBERT, a language-specific model pretrained on Arabic text. AraBERT outperformed the multilingual baseline across most metrics, highlighting the value of domain-specific pretraining. The paper was accepted at CLEF 2026 CheckThat!.