Unpacking Interpretability: Human-Centered Criteria for Optimal Combinatorial Solutions
New study reveals why some optimal solutions are easier to understand than others, enabling interpretability-aware AI.
A team of researchers led by Dominik Pegler has published a significant paper titled 'Unpacking Interpretability: Human-Centered Criteria for Optimal Combinatorial Solutions' on arXiv. The study tackles a core problem in human-AI collaboration: algorithmic systems often return optimal solutions that are technically correct but difficult for humans to comprehend. When multiple equally optimal solutions exist, humans must choose one, but the criteria for what makes a solution 'interpretable' have remained vague. The researchers developed an experimental paradigm where participants chose which of two equally optimal solutions for a classic 'bin packing' problem was easier to understand.
Their findings, detailed in the 66-page paper, show that human preferences reliably track three specific, quantifiable structural properties of a solution. The strongest associations were for solutions with an ordered visual representation and those that aligned with a simple, step-by-step 'greedy heuristic' a human might use. A third property, simple within-bin composition (grouping similar items together), also showed a consistent positive association. Interestingly, reaction-time data was mixed, and aggregate gaze-tracking from webcams did not reliably correlate with perceived complexity.
This research provides the first concrete, feature-based framework for measuring and designing for interpretability in combinatorial optimization tasks. By identifying these actionable properties—heuristic alignment, compositional simplicity, and ordered representation—the work enables a new approach: 'interpretability-aware optimization.' Developers can now explicitly code for these features or present AI-generated solutions in ways that highlight them, directly addressing the trade-off between raw optimality and human usability. This outlines a clear path to building AI systems that don't just solve problems correctly, but do so in a way that fosters effective human collaboration and trust in real-world allocation, scheduling, and design tasks.
- Identified three quantifiable properties for interpretability: alignment with a greedy heuristic, simple within-bin composition, and ordered visual representation.
- Based on experiments where participants chose between equally optimal 'bin packing' solutions, with the strongest effects for ordered representation and heuristic alignment.
- Enables 'interpretability-aware optimization,' allowing AI systems to be designed or tuned to produce solutions humans find easier to understand and trust.
Why It Matters
Provides a concrete framework for building AI that collaborates effectively with humans, moving beyond pure accuracy to include usability and trust.