Asking What Matters: Reward-Driven Clarification for Software Engineering Tasks
An 8B-parameter module matches GPT-5's success rate by learning which clarifications actually matter.
A team of researchers from Carnegie Mellon University and the University of Illinois Urbana-Champaign has published a paper introducing CLARITI, a novel AI system designed to solve a persistent problem in software engineering: incomplete task specifications. The core challenge is that when users ask AI assistants to write or fix code, they often leave out crucial details. Current models like GPT-5 tend to ask a barrage of clarifying questions, many of which are irrelevant or unanswerable, frustrating users and wasting time. The researchers tackled this by first empirically analyzing what makes a good question. Using methods like Shapley attribution, they identified two key properties: 'task relevance' (does the missing information actually predict task success?) and 'user answerability' (can a user realistically provide this information?).
They then operationalized these insights by turning them into multi-stage reinforcement learning rewards to train CLARITI, a specialized 8-billion-parameter clarification module. Instead of asking about every possible ambiguity, CLARITI learns to prioritize questions that are both impactful and practical. The results are striking: on real, underspecified software engineering tasks, CLARITI achieved the same issue resolution rate as the far larger and more general GPT-5 model, but did so while generating 41% fewer questions. This represents a major leap in human-AI interaction efficiency, moving from a scattershot interrogation to a targeted, intelligent dialogue.
The research, detailed in the arXiv preprint 'Asking What Matters: Reward-Driven Clarification for Software Engineering Tasks,' demonstrates that grounding AI reward design in empirical human factors leads to more usable and effective systems. This approach of quantifying information value and user capability could be applied beyond coding to any domain where AI assistants must collaborate with humans on complex, underspecified problems, from data analysis to creative writing.
- CLARITI is an 8B-parameter AI module trained with RL rewards based on empirical analysis of 'task relevance' and 'user answerability'.
- It matches GPT-5's success rate on vague coding tasks while asking 41% fewer clarifying questions, drastically improving interaction efficiency.
- The method uses Shapley attribution to quantify which missing information most affects task success, informing smarter question selection.
Why It Matters
This makes AI coding assistants less verbose and more effective, saving developer time and reducing frustration during collaborative task specification.