Sketch Bug lets devs debug code with pen strokes instead of mouse clicks

University of Waterloo researchers Helen Weixu Chen and Daniel Vogel have published a paper at Graphics Interface 2026 introducing Sketch Bug, a novel debugging interface that uses sketch-like pen input for execution control. Instead of clicking to set breakpoints or pressing keyboard shortcuts for step-into, programmers draw lightweight marks on their code to set breakpoints, use symbolic strokes to control step-wise execution, and extend those strokes into spirals to repeat traversal actions. The prototype combines gesture recognition with Python execution tracing inside a conventional editor.

In a controlled study with 24 programmers, Sketch Bug was compared against standard mouse-and-keyboard debugging for tasks requiring breakpoint placement, stepping, and runtime state inspection. Results show the sketch interface can effectively support these tasks, but introduces challenges in precision (e.g., accidentally triggering breakpoints), gesture recognition errors, and users forgetting available gestures. The authors conclude that pen input is most promising where debugger interactions benefit from spatial grounding or continuous movement—such as tracing execution paths—rather than as a wholesale replacement for traditional controls.