DocArena: turning raw documents into controllable search agent training environments

Training document search agents typically requires expert trajectories or manually annotated (question, answer, evidence) tuples, which are expensive and hard to scale—especially for multimodal documents. To address this, researchers from Adobe and academia introduce DocArena, a fully automated data curation pipeline that takes raw document collections and turns them into controllable training environments without any human annotation. The pipeline first structures and indexes documents using an MLLM-based visual perception module, then profiles cross-page information distribution to construct reasoning-intensive QA pairs. A cascaded quality assurance step using MLLMs further refines the data. The resulting dataset, DocArena-79K, contains QA pairs from 8,336 documents spanning 16 domains and 49 languages.

Beyond the dataset, the team designed a Doc-Search agent infrastructure that decouples visual perception from the policy model, allowing text-based LLMs to serve as the reasoning backbone for multimodal document retrieval and QA. This design choice simplifies deployment and leverages the strengths of existing LLMs. In a unified evaluation framework where only the policy model varies, agents trained on DocArena achieved best-in-class performance on both retrieval accuracy and QA quality across six multimodal document scenarios and seven text-based QA benchmarks. Further analysis of agent search behaviors confirmed the effectiveness and controllability of the constructed training environment, showing that the pipeline can produce diverse, targeted search strategies.