AI Can Spot Quran Recitation Slips — But Can't Agree What Counts

Anyone who has memorized a holy text knows the routine: you recite, a teacher listens, and they tell you where you slipped. Researchers now want software to handle part of that job. A new study — posted online, not yet peer-reviewed — asked a deceptively simple question. When an AI listens to a Quran memorization session, what exactly should it count as a mistake?

To find out, five researchers hand-annotated 100 real recording cases, marking 348 scored units and 162 specific events. Crucially, they marked not just errors but also repetitions, self-corrections, opening formulas, and spelling differences that are perfectly acceptable. A plain word-by-word comparison — the simplest possible method — scored about 0.53 on labeling and 0.83 on locating events. The team then ran eight 20-minute tests using three different AI coding tools and eight models. Results swung wildly: from 0.14 to 0.89, with seven runs well above the simple baseline and one falling below it, only because a routine text-cleaning step was left out.

The most revealing finding came after the scores. Across six of the runs, 970 of 972 known events were flagged by the AI. So spotting something is not the hard part. The hard part is deciding whether what was spotted is genuinely a mistake. Seven events defeated every single run — and five of those come down to one rule about spelling that a human judge stipulated, not something visible in the text itself.

The authors are upfront about a major limitation: no run was allowed to annotate before building its system, so this pilot tests only the algorithmic half of the job. Still, the lesson travels well beyond Quran study. Every AI grading tool, language-learning app, or transcription service hits the same wall. The machine can hear the difference, but a person has to say which differences actually matter.