From What to How: Bridging User Requirements with Software Development Using Large Language Models

A new research paper introduces DesBench, a design-aware benchmark evaluating 7 top LLMs on software design tasks. The study found LLMs struggle significantly with high-level design, object-oriented modeling, and generating correct code from requirements alone. While they can identify classes, they fail at defining operations and relationships. The benchmark includes 30 Java projects, 194 classes, and 737 test cases, testing models like GPT, DeepSeek R1, and Qwen2.5.