Actionable Real-Time Modeling of Surgical Team Dynamics via Time-Expanded Interaction Graphs

Current surgical AI systems focus heavily on visual workflow signals, such as tool detection and phase recognition, but largely ignore the complex interpersonal dynamics that drive team performance. Non-technical skills like communication, coordination, and leadership play a critical role in surgical outcomes, yet they remain poorly modeled in automated systems. To address this gap, researchers introduce a real-time actionable framework that represents the operating room team as a time-expanded interaction graph. In this graph, each team member appears as a node at every time step, and communication exchanges (e.g., verbal commands, acknowledgments) define directed edges. This structure captures the spatio-temporal evolution of team interactions while allowing efficient inference using a static graph neural network (GNN). The model is trained to predict procedural efficiency as the normalized deviation from the expected duration of the surgery, giving a real-time risk score for prolonged interventions.

The system goes beyond simple prediction by integrating counterfactual analysis to offer actionable explanations. It identifies the minimal changes in communication structure—such as increasing the frequency of closed-loop confirmations or adjusting information flow between specific roles—that would lead to improved predicted outcomes. Tested on recorded surgical procedures, the model demonstrated improved early identification of prolonged interventions compared to baseline approaches that ignore team dynamics. The interpretable insights generated by the counterfactuals provide surgeons and OR managers with concrete behavioral levers to improve team coordination in real time. This work, accepted at HHAI 2026, marks a significant step toward surgical AI that is not only vision-aware but also team-aware, enabling decision support that accounts for the human factors at the core of surgical performance.