Experimental study on surveillance video-based indoor occupancy measurement with occupant-centric control
Researchers combine YOLOv8 and DeepSeek-V2 to create a smart building system that slashes energy consumption.
A new research paper demonstrates a significant leap in using AI for smart building efficiency. The study, led by Irfan Qaisar, Kailai Sun, and colleagues, tested a novel pipeline that refines computer vision-based occupancy detection with a large language model. They compared three methods using real surveillance footage from a lab: a basic detection-only approach, a more stable tracking-based method, and their new LLM-enhanced pipeline. The winning combination used YOLOv8 for initial object detection and the open-source DeepSeek-V2 model to analyze and correct the results, achieving a high accuracy of 0.8824 and an F1-score of 0.9320. This system drastically reduced false 'unoccupied' predictions, providing a stable and reliable count of people in a room.
The critical innovation was integrating this precise occupancy data into a building's control systems. The researchers plugged their YOLOv8+DeepSeek pipeline into a Model Predictive Control (MPC) framework within the OpenStudio-EnergyPlus building simulation software. This created a closed-loop, occupant-centric control (OCC) system where HVAC operations dynamically respond to the actual, real-time number of people present. The experimental results were striking: this AI-driven approach enabled the HVAC system to achieve a 17.94% reduction in energy consumption. The study provides a concrete methodology and performance benchmark, proving that modern vision models and LLMs can move beyond theory to deliver major operational savings and support global Net Zero emissions goals.
- The system combines YOLOv8 for detection with DeepSeek-V2 for refinement, achieving an F1-score of 0.9320 for occupancy counting.
- Integrated into an EnergyPlus simulation, the AI pipeline enabled a Model Predictive Control system that cut HVAC energy use by 17.94%.
- The research provides a validated, practical framework for using open-source AI models to create efficient, occupant-centric smart buildings.
Why It Matters
This proves AI can deliver major, quantifiable energy savings in real-world infrastructure, directly supporting sustainability and cost-reduction goals.