New study defines user trust with Perceived System Predictability
Researchers quantify 'predictability' as a 6-item scale to measure user trust in AI systems.
Researchers Hendrik Schuff, Heike Adel, and Ngoc Thang Vu from the University of Stuttgart have introduced Perceived System Predictability (PSP), a user-centered construct designed to quantify how predictable users find interactive systems, particularly AI models. Published on arXiv as *Perceived System Predictability: Scale Development and Application*, this work addresses a critical gap in human-computer interaction (HCI) by distinguishing between epistemic, aleatory, and effective predictability—three dimensions of user perception that influence trust and reliance on AI systems.
The team developed a 6-item PSP scale, refined from an initial 60-item pool through expert reviews and cognitive interviews. The scale was validated in two separate studies involving 400 participants. In a shape-classifier study, PSP was found to predict user prediction correctness, while explanations increased perceived predictability without improving actual accuracy. Conversely, increased stochasticity in system outputs degraded correctness without reducing perceived predictability, highlighting a divergence between user perception and system behavior. These findings suggest PSP offers a more principled foundation for designing transparent and trustworthy AI systems than existing subjective or objective measures.
- PSP is a 6-item scale that quantifies how predictable users perceive AI systems to be, validated across 400 participants.
- Findings show explanations can boost perceived predictability but don’t always improve user prediction accuracy.
- Increased stochasticity in AI outputs degrades prediction correctness but doesn’t necessarily reduce perceived predictability.
Why It Matters
PSP provides a measurable way to design AI systems that users can trust and understand, bridging the gap between perception and reality.