DrawingVQA benchmark reveals huge gap in AI's construction drawing skills

A team of researchers from the University of Illinois at Urbana-Champaign has introduced DrawingVQA, the first benchmark designed to rigorously evaluate multimodal large language models (MLLMs) on real-world construction drawings. Unlike natural images or simple floor plans, these drawings combine abstract geometry, symbolic notation, tabular data, annotations, and domain-specific text—creating a uniquely complex visual-textual domain that is core to architecture, civil engineering, and other engineering workflows.

The benchmark comprises 33 professional "Issued for Construction" drawings and 92 carefully crafted question-answer pairs spanning three reasoning depths: perceptual understanding (e.g., identifying symbols), contextual interpretation (e.g., understanding relationships), and domain-expert reasoning (e.g., applying engineering codes). The team also developed a dual categorization framework to map AI reasoning competencies to seven construction-engineering dimensions. Evaluations of state-of-the-art MLLMs (including GPT-4o and other top models) showed a substantial performance gap compared to expert human annotators, particularly at higher reasoning depths, highlighting the need for domain-specialized multimodal AI. The work was accepted as a Findings paper at CVPR 2026.