A Visualization for Comparative Analysis of Regression Models
Researchers propose a 2D visualization method that moves beyond aggregate metrics like MAE and R-squared to compare regression models.
A team of researchers from the ICube laboratory (Nassime Mountasir, Baptiste Lafabregue, Bruno Albert, Nicolas Lachiche) has introduced a new visualization technique designed to solve a common problem in machine learning: the oversimplification of model comparison. Currently, practitioners rely heavily on aggregate metrics like Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R-squared (R²) to choose between regression models. While these numbers indicate overall performance, they collapse all error information into a single value, potentially hiding critical patterns in how and where models fail.
The proposed method, detailed in the arXiv preprint "A Visualization for Comparative Analysis of Regression Models," offers a more nuanced, graphical alternative. Its core innovation is plotting the residuals (prediction errors) from two different models against each other in a 2D space. This scatter plot is then enhanced by using the Mahalanobis distance—a statistical measure that accounts for correlations and differing scales within the dataset—to better understand the error distribution. Finally, a colormap based on error percentiles is applied, making it easy to visually identify clusters of high-density errors and outliers.
This visual framework allows data scientists to move beyond asking "which model has a lower RMSE?" to understanding questions like "does Model A consistently underestimate high values while Model B's errors are more random?" or "where in the feature space do both models struggle?" By revealing the correlation structure between two models' errors and the spatial distribution of those errors, the tool facilitates a deeper, more interpretable comparison, ultimately leading to more informed model selection and diagnosis for real-world applications.
- Plots residuals from two models in 2D space for direct visual comparison, moving beyond single-number metrics.
- Uses Mahalanobis distance to properly account for data correlations and scale, providing a statistically sound visualization.
- Applies a percentile-based colormap to highlight dense error regions and outliers, revealing patterns hidden by MAE or R².
Why It Matters
Enables data scientists to make better model choices by visually diagnosing error patterns that aggregate metrics miss, improving real-world prediction reliability.