ProgramAT lets blind users build custom AI tools with natural language

A new study from University of Michigan and Carnegie Mellon researchers introduces ProgramAT, an agentic programming tool that empowers blind and low-vision (BLV) users to build custom assistive technologies using natural language commands. In a longitudinal co-design study with five tech-savvy BLV participants, the team observed the creation of 37 distinct camera-based tools over multiple sessions. These tools addressed accessibility gaps not covered by existing commercial solutions, such as identifying specific Uber rides by license plate and interpreting hand gestures during conversations.

The study, set to be presented at ASSETS '26, uncovered several key findings. BLV users employed unique strategies for iterating on their tools, often relying on auditory feedback from ProgramAT to debug and refine their code. However, participants faced significant barriers including model capability limits, specification conflicts (when natural language instructions produced contradictory code), and difficulties in sharing or reusing tools. The authors recommend adding conversational scaffolding, community tool-sharing features, and support for specialized models and personal datasets to future agentic programming environments. This research demonstrates that lowering technical barriers through natural-language-based programming can enable BLV individuals to create highly personalized visual assistance, potentially reshaping how assistive technology is designed and deployed.