Ancient Palm-Leaf Texts Stump AI — Best Model Gets 58% Right

Researchers have built a new standardized test — think of it as a final exam — for AI that looks at photos of ancient palm-leaf manuscripts. These are fragile, hand-written documents from South and Southeast Asia, some centuries old, made by scratching text into dried palm leaves. The test, called PalmLeaf-VQA, contains 923 photographed manuscripts and 7,384 question-and-answer pairs drawn from eight different traditions, including Tamil, Khmer, Balinese, Kannada and Sundanese.

The questions go beyond simply reading words. They ask about the physical object: Is the leaf cracked? Where are the binding holes? What is written in the margins? Are there drawings or coating marks? That matters because libraries, museums and universities hold thousands of these documents, and many are decaying from humidity, insects and time. If AI could reliably catalog, describe and translate them, archivists could protect and share cultural history far faster than people working page by page.

But the results are sobering. The strongest AI model tested got just 58% of answers exactly right — and "exactly right" is a strict standard, not a generous one. Imagine a translator who gets roughly four words in ten wrong. The reason is simple: today's AI learned mostly from modern, Latin-script web text, so faded handwriting, unusual letter shapes and old page layouts are unfamiliar territory. The dataset is also small, with under a thousand images, so the test isn't the final word on AI's abilities.

The takeaway isn't that AI is useless here — it's that researchers have finally built a measuring stick for a problem most people never noticed. As models improve, the payoff could be real: searchable, translated archives, faster preservation decisions, and digitized history that anyone with a phone can explore, instead of a handful of scholars in a locked room.