New GRPO method boosts heart QA accuracy by 39% on small LLMs

A new paper from researchers at Oklahoma State University and the University of Oklahoma introduces a post-training method for improving heart-focused medical question answering in LLMs. Using Group Relative Policy Optimization (GRPO) with a Variance-Aware Reward Framework, they fine-tuned the 14B-parameter Qwen3-14B model on a heart-related subset of HealthBench. The key innovation replaces traditional binary or Likert-style rubric scoring with continuous analytical reward functions derived from criterion-level rubric outcomes. This approach provides richer optimization signals for sparse, multi-criteria feedback and enables more stable on-policy reinforcement learning.

The results are striking: the best GRPO variant boosted accuracy from 0.362 to 0.502 and F1 from 0.532 to 0.668 relative to the base Qwen3-14B model. These numbers are competitive with the 120B-parameter GPT-OSS model, which achieved 0.508 accuracy and 0.674 F1—showing that a carefully fine-tuned small model can match a much larger one in specialized medical reasoning. The work, published on arXiv (2606.05174), suggests that rubric-based rewards with variance awareness offer a practical path for improving medical QA in smaller, more deployable LLMs, with potential to extend to other rubric-based tasks.