Refactoring for Novices in Java: An Eye Tracking Study on the Extract vs. Inline Methods

A team of researchers from multiple universities has published a novel study on arXiv that uses eye tracking to analyze how Java novices comprehend code after Extract Method versus Inline Method refactorings. The study moves beyond static code metrics to measure the human factors of comprehension and navigation dynamically.

The controlled experiment involved 32 novice Java developers solving eight simple tasks across four programs, each presented in both inlined and extracted versions. Researchers tracked visual effort using metrics like fixation duration, count, regressions (backward eye movements), and revisits, alongside task completion time and attempts. For complementary data, they also surveyed 58 additional novices. The key finding is that the impact of method extraction is highly dependent on task complexity. For two more difficult tasks, extraction significantly improved performance, reducing completion time by up to 78.8% and cutting regressions by 84.6%. However, for simpler tasks (like calculating a square's area), extraction had a negative effect, increasing time by up to 166.9% and regressions by 200%.

The data revealed that even with meaningful method names, novices frequently switched back and forth between call sites and extracted methods, increasing navigation and cognitive load for simple logic. While survey preferences often favored extraction for perceived readability and reuse, these preferences did not always align with the measured performance outcomes.

This research provides crucial, evidence-based guidance for computer science education. It suggests that instructors should be cautious about encouraging premature modularization for beginners, as it can inadvertently hinder comprehension of straightforward code. The study also validates eye tracking as a powerful tool for obtaining nuanced, human-centric data that static code analysis alone cannot capture.