GreekBarRetrieval benchmark tackles Greek legal AI with 6K+ statutes
New benchmark GreekBarRetrieval tests AI's ability to parse 6,308 Greek legal articles...
Researchers introduce GreekBarRetrieval, a public retrieval benchmark derived from and complementing GreekBarBench, which previously did not include retrieval. The benchmark includes 283 bar-exam questions, each with the facts of the case it refers to, and 6,308 candidate statutory articles to retrieve from. In experiments, vanilla dense retrieval far outperformed vanilla sparse retrieval in Recall@100, but LLM-based query reformulation helped BM25 close that gap while also boosting dense retrieval. A ten-round ReAct-like reformulation loop improved BM25 further in Recall@100 and gave it the best nDCG and MAP scores among all tested retrievers. Query reformulation also beat pseudo-relevance feedback, sparse-dense fusion, and English translation.
- GreekBarRetrieval is the first public benchmark for Greek statutory retrieval with 283 questions and 6,308 legal articles
- Dense retrievers beat sparse methods in Recall@100, but LLM query reformulation boosts BM25 to top scores
- Ten-round ReAct-like reformulation loop outperforms translation, pseudo-relevance feedback, and sparse-dense fusion
Why It Matters
Enables AI-powered legal research for Greek statutes, improving access to justice and reducing research time for lawyers