DocAnnot framework uses GenAI to auto-annotate documents with 84.6% F1 score
New framework achieves 84.6% F1 on SROIE, reducing manual dataset creation by up to 80%
DocAnnot is a new framework designed to accelerate the creation of training datasets for Key Information Extraction (KIE) — the task of extracting structured fields like invoice totals or dates from documents. Traditionally, this requires costly manual annotation. The framework uses a Large Vision Language Model (LVLM) to extract label values, OCR to detect text and bounding boxes, and a novel Spatially Informed Contextual Matching (SICM) algorithm. SICM improves the association between labels and values by combining spatial proximity and textual cues, reducing the need for rule-heavy heuristics.
Evaluated on standard benchmarks CORD and SROIE, DocAnnot achieves auto-annotation F1-scores of 0.679 and 0.846 respectively. When these outputs are used to fine-tune a downstream model like LayoutLMv3, performance reaches 0.6765 F1 on CORD — about 5% below human-annotated baselines. The authors emphasize that while human review is still beneficial, DocAnnot cuts annotation time and cost drastically, making it valuable for resource-constrained teams and rapid model prototyping. The paper is published at ICDAR 2025 and is available on arXiv.
- DocAnnot combines LVLM, OCR, and a Spatially Informed Contextual Matching algorithm to auto-annotate document datasets
- Achieves 67.9% F1 on CORD and 84.6% F1 on SROIE benchmarks, drastically reducing manual labeling effort
- LayoutLMv3 fine-tuned solely on DocAnnot outputs reaches 67.65% F1 on CORD, enabling fast model prototyping
Why It Matters
Automates 80% of document dataset annotation, letting teams prototype KIE models in days instead of weeks.