Researchers' SafeBuild-Bench exposes MLLMs' construction safety gap with 60% top score

SafeBuild-Bench, presented at KDD 2026 by a research team including Yi Cui and Hui Xiong, tackles a critical blind spot in vision-language AI: construction safety assessment. Unlike curated object recognition datasets, the benchmark is mined from over 100,000 real industrial image-text records spanning multiple construction sites and months. It contains 3,314 task instances drawn from more than 3,000 expert-verified images, with tasks covering both multiple-choice hazard identification and free-form hazard description. The metadata-driven design explicitly tests temporal robustness—whether models can handle variations in site conditions, equipment placement, and worker activity over time.

To build the dataset efficiently, the authors developed GEMS (graph-enhanced multimodal selection), a pipeline that uses a proxy-model confusion signal combined with graph-based diversity to select the most informative candidates from redundant data streams. This approach also proves effective on public instruction-tuning data, preserving robustness-oriented performance even with small data budgets. Benchmark results show current multimodal large language models (MLLMs) achieve a best overall score of only about 60%, indicating significant gaps in reliable construction-safety understanding. The authors have released the benchmark, evaluation scripts, and GEMS codebase, aiming to drive progress toward deployable AI safety systems.