ChunkGroupSHAP: New method explains AI rankings by grouping semantic chunks

Dense embedding rankers score documents using contextual sentence- and passage-level representations, but existing listwise explanation methods still attribute rankings to isolated words. This feature-unit mismatch leaves word-level features too fragmented for dense semantic ranking. To solve this, researchers from an anonymous institution (authors Hyunkyu Kim, Yeeun Yoo, Youngjun Kwak) propose ChunkGroupSHAP, a novel Shapley-based method that clusters semantically related chunks into shared cross-document features. By masking a group, it perturbs all documents with related evidence, attributing rankings at a granularity closer to dense representations while preserving the listwise setup.

The team evaluated ChunkGroupSHAP across four datasets—MS MARCO, FinanceBench, AILACaseDocs, and FinQA—using E5 dense rankers and BM25 lexical baseline. Their key finding: the best explanation unit is highly setting-dependent. For lexical BM25, word-level features work best. For dense rankers, corpus-level semantic groups are most effective. For heterogeneous web retrieval, query-local grouping shines. This implies explanation methods must adapt to both the ranker's representational granularity and the structure of the retrieved corpus. The work, published on arXiv with 17 pages, 5 figures, and 4 tables, offers a practical blueprint for more transparent information retrieval systems.