Visual Decoding Operators: Towards a Compositional Theory of Visualization Perception

A team of researchers from leading institutions has published a paper titled 'Visual Decoding Operators: Towards a Compositional Theory of Visualization Perception' on arXiv. The work, led by Sheng Long with co-authors Remco Chang, Eugene Wu, Alex Kale, and Matthew Kay, addresses a fundamental limitation in data visualization research. Current methods for assessing a chart's perceptual effectiveness rely on decomposing it into basic channels (like angle or length) and ranking them. However, these rankings lack a computational structure to predict how people will perform on new, unseen combinations of charts and analytical tasks, forcing researchers to run new experiments for each variation.

The team proposes a novel unit of analysis: 'visual decoding operators.' This approach operationalizes the act of reading a quantitative chart as a sequence of smaller, composable perceptual operations. Using PDF and CDF charts, they show how chart-specific tasks can be broken down into these reusable, chart-agnostic operations. They then characterize the error profiles of these operators using hierarchical Bayesian modeling. Crucially, they tested the generalizability of their learned operators by composing them to predict performance on a completely different experiment from prior literature: Moritz et al.'s scatterplot mean-estimation task. With a pre-registered analysis, they evaluated six candidate composition strategies against the empirical data without fitting parameters to it. Only one strategy successfully captured both the bias and variance of human responses, while the other five failed in identifiable ways.

This research lays a new foundation for building predictive, generative models of visualization interpretation. Instead of just describing which charts are better, this framework aims to model the distribution of likely human interpretations. It opens the door to predicting how errors might arise under new viewing conditions, with novel chart types, or for unfamiliar analytical tasks, moving the field from descriptive rankings to computational prediction.