GoodPoint: Learning Constructive Scientific Paper Feedback from Author Responses

A research team from Carnegie Mellon University and KAIST has introduced GoodPoint, a novel AI training framework designed to generate constructive feedback for scientific papers. The system addresses a critical gap in current LLM applications by focusing on feedback that is both valid (correct and relevant) and actionable (likely to lead to author improvements). To achieve this, the team first curated GoodPoint-ICLR, a dataset of 19,000 papers from the ICLR conference where reviewer comments were annotated based on the authors' subsequent responses. This dataset provides a unique success signal, showing which feedback actually prompted meaningful revisions.

The core innovation is the GoodPoint training recipe, which combines fine-tuning on valid and actionable feedback examples with preference optimization using both real and synthetic data pairs. When applied to the Qwen3-8B model, the results were striking: evaluation on a benchmark of 1,200 ICLR papers showed an 83.7% improvement in predicted feedback success rate over the base model. Remarkably, the enhanced model set a new state-of-the-art among similarly sized LLMs and even surpassed the much larger Gemini-3-flash in precision on a golden set of human feedback. An expert human study further confirmed that GoodPoint-generated feedback was consistently rated as having higher practical value by authors, validating the real-world utility of this approach for augmenting, rather than automating, the scientific review process.