Quality-Driven Agentic Reasoning for LLM-Assisted Software Design: Questions-of-Thoughts (QoT) as a Time-Series Self-QA Chain

Researchers Yen-Ku Liu and Yun-Cheng Tsai have introduced a novel framework called Questions-of-Thoughts (QoT) to tackle persistent quality issues in AI-assisted software development. The method acts as an inference-time scaffold that transforms a user's goal into a structured, ordered sequence of engineering steps. Crucially, it incorporates a time-series self-questioning chain, where the LLM verifies constraints and reduces omission errors at each step, maintaining a lightweight reasoning record to stabilize subsequent design decisions. This approach directly targets common failure modes like incomplete implementations, weak modularization, and inconsistent security practices that plague current LLM-generated code.

The team rigorously evaluated QoT across three complex backend engineering domains: API Design, Data Communication, and File Systems. They scored the AI-generated artifacts using a custom, ISO/IEC-inspired quality rubric measuring Scalability, Completeness, Modularity, and Security. Results showed the improvements are capacity-dependent; larger, more capable models like GPT-4 and Claude 3.5 Opus saw consistent and significant quality gains when using QoT, with domain-wise score improvements often exceeding 40%. However, smaller models sometimes faced trade-offs due to tight context windows and planning budgets. The researchers have released their full artifact—including prompts, scoring guidelines, and reproduction scripts—to support further applied AI and software engineering research.