AnnoIndex lets you run SQL-like queries on unstructured documents with 0.87 F1

Unstructured documents make up most enterprise and web data, yet traditional retrieval methods like vector similarity are fuzzy and unreliable for precise analytical queries. AnnoIndex, a new system from researchers Teng Lin, Yuyu Luo, and Nan Tang (arXiv:2608.13384), tackles this problem by first converting raw text into a structured, queryable index before answering questions. Its first core component, the Annotation Index, uses a module called SchemaLoop to automatically build hierarchical annotation schemas from a corpus. Lightweight language models then extract attribute values, materializing scattered text into a structured index that supports low-cost filtering and avoids the black-box matching of vector similarity.

The second innovation is a Structured Query Engine that compiles user questions into execution plans based on SQL extensions. It starts by using the Annotation Index to precisely filter documents, then applies extraction operations in ascending order of cost. LLMs are only invoked for the small remaining fraction of data that truly requires deep semantic understanding, and any extracted attributions are merged back into the index to reduce future query costs. In experiments on three real-world datasets, AnnoIndex consistently outperformed state-of-the-art baselines, achieving the highest average F1 score of 0.87, while maintaining robust performance on complex multi-hop joins and progressive reasoning queries. The result is a scalable, transparent alternative to fuzzy retrieval—making analytical queries over unstructured documents as reliable as querying a database.