That 'Accessible' PDF May Still Be Unusable, Study Finds

A researcher at arXiv has published a study with a mouthful of a name — PDFa11yMut — but a simple warning underneath: the software that tells you whether a PDF is accessible often isn't telling the whole story. PDFs are everywhere: your bank statements, utility bills, HR handbooks, school permission slips, government forms. Accessibility checkers (tools built into Adobe Acrobat, plus free ones called PAC and veraPDF) scan those files and issue a pass-or-fail report. Companies and agencies rely on those reports to claim they're following disability-access laws.

The study tested the checkers by deliberately breaking PDFs in 69 different ways and seeing which breaks got caught. For 30 structural problems — think missing labels that tell a screen reader "this is a heading" or "this is a table" — the checkers did well. PAC and veraPDF each flagged all 30, and Acrobat flagged 23 directly. But for 39 subtler problems about meaning and how content is presented to assistive technology, not a single checker caught anything automatically. A separate hands-on test with NVDA, a popular free screen reader used by blind people, found that 8 out of 9 broken PDFs actually sounded different to a real user.

In plain terms: the checkers spot whether the scaffolding is in place. They don't check whether the building is usable. A PDF can pass with flying colors and still read out loud as garbled nonsense — out-of-order paragraphs, images described as "image," or tables that jumble into a stream of disconnected numbers.

So what should you do? Treat a green checkmark as a starting point, not a finish line. If you publish PDFs for the public or for employees, the honest test is having someone who uses a screen reader try to actually use the document. The researcher is careful to note this isn't a universal scorecard for every tool in every setting — it's a scoped study. But the practical lesson holds: passing the automated check doesn't prove your document is readable by the people it's supposed to serve.