HiLSVA puts humans back in control of AI for scientific visualization

Researchers from the University of Notre Dame have introduced HiLSVA, a novel human-in-the-loop agentic system designed for scientific visualization (SciVis). Unlike prior LLM-powered systems that prioritize full autonomy, HiLSVA keeps humans firmly in the loop through a plan-first multi-agent architecture. It integrates explicit human oversight at every step, stepwise provenance tracking to trace decisions, and a learn-at-test-time mechanism that adapts to user feedback in real time. The system supports fluid handoffs between humans and agents via natural language or direct manipulation of visualizations, and all executions run in a sandboxed environment to ensure safety and reproducibility.

A controlled user study with 12 participants of varying expertise tested HiLSVA across multiple autonomy settings. Results showed that mixed-initiative interaction significantly improved task completion rates, user control, and workflow transparency compared to fully autonomous or fully manual modes. However, the study also revealed a clear tradeoff: adding human oversight reduced execution speed. These findings underscore the importance of human-centered design in agentic SciVis, reframing the process as a collaborative partnership rather than a replacement for human analytical reasoning. The team has released their demo video, case studies, and source code for further exploration.