SD-GRPO: New method improves long-form vision-language AI with segment-level rewards

Group Relative Policy Optimization (GRPO) and its variants have shown success in multimodal LLMs, but they rely on a single scalar advantage, which underfits vision-language tasks requiring long-form, image-grounded responses. Researchers from South Korea introduce SD-GRPO, which exploits the natural segmentation of long-form outputs by applying z-normalized per-segment rewards across the rollout group. This yields a vector of per-segment advantages, solving the coarse-grained credit assignment problem. The method is evaluated across three settings: controlled multi-panel dense captioning (DOCCI), multi-chart VQA (MultiChartQA), and real-world scientific figure captioning (MMSci). On DOCCI, SD-GRPO consistently outperforms GRPO, with larger gains at higher segment counts. On MultiChartQA, it theoretically and empirically demonstrates that rollout-level rewards suffer from cross-segment misattribution that scales with output length. On MMSci, blending holistic and per-segment rewards further improves results when segments are semantically entangled.

SD-GRPO is also integrated into the existing Dr. GRPO framework, confirming its plug-and-play nature with minimal implementation overhead. The method is particularly impactful for tasks where long, structured output is essential—such as summarizing multi-panel figures, complex chart analysis, or scientific diagrams. By replacing a single scalar with a vector of segment-level advantages, SD-GRPO enables more precise reinforcement learning for vision-language models, paving the way for higher-quality, detail-rich AI descriptions in professional and research contexts.