Khondo benchmark reveals MLLMs fail at page order in Bangla forms

A team of researchers from Bangladesh and the US has introduced Khondo (Bangla for split/segment), the first benchmark specifically designed for document packet splitting of low-resource language forms. Document packets—multiple documents concatenated into a single file—are common in government and administrative workflows, but splitting them reliably is difficult, especially for languages like Bangla. Khondo focuses on Bangladeshi government forms and is bilingual (Bangla-English) and vision-native, meaning models operate directly on page images rather than OCR text. The benchmark spans five concatenation schemes (from sequential to fully shuffled) across 14 administrative domains, providing ground-truth boundaries, domain types, and page order.

Zero-shot evaluation of state-of-the-art multimodal large language models (MLLMs) revealed that while they can cluster pages into their source documents fairly well, they struggle significantly with restoring original page order after shuffling. Controlled tests showed that explicit page-order instructions are necessary but insufficient, and that English packets are ordered more reliably than Bangla ones. The authors conclude that page-order reconstruction is a key open problem in vision-based, low-resource document understanding. Khondo establishes a controlled benchmark for measuring progress toward solving this challenge. The dataset and code are publicly available.