New HCI tools enable cross-sensory AI design

Albrecht Kurze and Klaus Stephan from the Human-Computer Interaction (HCI) field have introduced novel co-design tools, the 'Loaded Dice' and 'Wheel of Plush,' designed to facilitate cross-sensory interactions in AI systems. These devices integrate multiple sensors and actuators, allowing users to map any sensor input (e.g., temperature, sound) to any actuator output (e.g., color, vibration) through intuitive interaction schemes. The tools are grounded in the principle of 'technical synesthesia,' a concept enabling seamless translation between senses, which could revolutionize affective computing by allowing AI to interpret and respond to multi-modal emotional cues more naturally.

Presented at the CHI 2026 workshop 'Cross-Sensory Futures: Rewiring Perception in HCI,' these tools were developed over the past decade to support activities like sensory sketching, scenario-driven ideation, and exploration of multi-modal interactions. The research highlights how these devices can aid designers in prototyping interactions that blend sensory inputs, such as associating the 'temperature of a loud cry' or the 'color of a hand-clap'—bridging abstract sensory experiences into tangible design outcomes. The work opens new avenues for creating emotionally intelligent AI systems by making cross-sensory co-design accessible and practical.