New RAG method uses ToC-guided page retrieval with zero LLM cost, boosting answer quality by 41%

Retrieval-augmented generation (RAG) systems typically chunk documents, destroying structural information. Existing heading-based retrieval methods require multiple LLM calls per document and return sub-chunks within matched sections. In a new arXiv paper, researcher Ng S. T. Chong presents ToC-guided page retrieval, which infers headings directly from visual formatting (e.g., fonts, sizes) using a zero-LLM-cost algorithm. Headings are embedded as a parallel index, and the system loads full page sections rather than sub-chunks. This approach preserves document hierarchy and eliminates the overhead of LLM-based heading extraction.

Tested across 1,280 conditions on 8 enterprise documents ranging from 5 to 195 pages, the method yields a statistically significant main effect on answer quality (Cohen's d = +0.41, p = 0.031), with the largest improvements in completeness and usefulness (+0.40 each). Combined with answer-side verification, it outperforms query-side decomposition plus verification (d = +0.32, p = 0.036). Despite adding only 2.9 pages per query, ToC contributes 20% of all citations. Gains are directionally larger on longer documents (up to +1.50 on a 118-page document), though the trend does not reach significance with 8 documents. A 480-condition sensitivity analysis confirms that default parameters are near-optimal (all p > 0.38). The work provides factorial evidence that document-side, query-side, and answer-side enhancements are complementary—a three-way interaction not previously studied.