Kimi K2.5 patches 80% of accessibility errors, but full fixes rare

Researchers evaluated Kimi K2.5 for automated web accessibility repair, comparing it to rule-based tools. For detection, the LLM reached a macro F1 of 0.65, matching rule-based methods overall, but excelled at semantic understanding (F1 0.83) while lagging on syntactic and layout violations. When called upon to fix issues, the model produced syntactically valid code in over 99.7% of cases and improved compliance in 80.2% of instances, reducing average violations per file from 3.98 to 1.7. However, full resolution occurred in fewer than 26% of cases, and about 30% of patches introduced structural changes to the page.

Crucially, attempting to improve fixes through iterative agent-based refinement backfired: it increased computational cost by 52% and API calls by 1.64x without boosting remediation quality. The findings suggest that while LLMs can significantly aid accessibility work—especially for semantic issues—they are not yet a complete replacement for manual or rule-based methods. The authors advocate for hybrid systems that combine the LLM's semantic strengths with deterministic validation and constraint-aware correction to achieve scalable, reliable web accessibility.