New AI Makes Nepali Voice Payments Easier for Blind Users

Imagine trying to use your phone to send money, but you can’t see the screen. That’s the reality for many blind users in Nepal, where mobile payments are growing fast but rely entirely on visual interfaces. A new AI system called SpeakPay is changing that by letting people use voice commands — like “send 500 rupees to Ram” — to complete financial tasks without ever touching the screen.

The breakthrough comes from training a speech recognition model not on general speech, but on 403 specific Nepali money-related phrases (think “load wallet”, “check balance”, or “send 1,234”). This focused training cut errors dramatically — for example, the AI now recognizes Nepali numbers correctly 74% of the time, up from 0% before. More importantly, it made actual payments work: the chance of a transaction succeeding jumped from just 2% to 33%. Even with just 100 training examples, the AI cut errors in half — showing you don’t need massive datasets to make a real difference.

SpeakPay uses a technique called domain adaptation (training AI on the exact kind of speech it will hear) and a method called LoRA (a way to fine-tune large AI models without huge computing power). It’s built on Whisper, a popular speech recognition model, and is now live as a free web app anyone can use. While it’s tailored for Nepali, the same idea could help bring voice-first finance to other low-resource languages where typing or seeing screens is a barrier.

The team behind it hopes this shows how AI can solve practical problems — not just flashy tech — by focusing on real user needs. But they also found that most remaining errors come from confusing numbers or adding extra digits, proving even smart AI still needs a human touch in sensitive tasks like money.