Toward Scalable Co-located Practical Learning: Assisting with Computer Vision and Multimodal Analytics

A research team led by Xinyu Li and Linxuan Zhao has demonstrated that scalable, non-intrusive AI monitoring of co-located teamwork is possible with minimal hardware. Their study, "Toward Scalable Co-located Practical Learning," used just one ceiling-mounted camera to capture video from 52 undergraduate nursing simulation sessions. Teachers first identified seven key observable behavior categories, which were then used to train a YOLO-based object detection model. The system achieved strong performance on a held-out test set, with a precision of 0.789, recall of 0.784, and a mean Average Precision (mAP@0.5) score of 0.827.

The core finding was that behavior frequency alone did not distinguish team performance. The breakthrough came from multimodal analytics—combining the detected behavior labels with spatial context data. This analysis revealed clear, actionable patterns: higher-performing teams spent significantly more time interacting with the patient in the primary work area. In contrast, lower-performing teams exhibited more phone-related activity and were more frequently observed in secondary, non-critical areas of the simulation space. This proves that *where* an action occurs is as informative as *what* the action is for assessing task engagement and collaboration quality.

The research provides a blueprint for affordable, scalable assessment in hands-on educational and professional training environments. By relying on a single camera instead of wearable sensors or complex multi-camera setups, the method lowers the barrier to implementation. It offers educators and trainers an objective tool to provide feedback on teamwork dynamics, potentially improving outcomes in fields like healthcare, aviation, and emergency response where effective co-located collaboration is critical.