BGE-M3 beats Google Translate for Sinhala-Tamil e-government RAG, hitting 96% recall

A new arXiv paper (2608.12820) from Dharshi Balasubramaniyam and Tiroshan Madushanka tackles a real-world pain point: how do citizens query English government documents in Sinhala or Tamil? The researchers built a human-verified benchmark of 500 question-answer pairs derived from 1,699 segmented contexts from Sri Lanka's Government Information Center (GIC). They then systematically compared two cross-lingual information retrieval (CLIR) paradigms: Query Translation (QT) using Google Translate, NLLB, and mBART50; and Cross-Lingual Embeddings (CLE) using LaBSE, multilingual E5, and BGE-M3. Monolingual English retrieval served as the baseline, and all methods were evaluated using Recall@k (k = 1, 3, 5, 10, 15).

The results are striking. Monolingual English retrieval performed poorly, with Recall@15 under 10%, confirming that native-language queries need dedicated cross-lingual processing. All CLIR approaches dramatically improved accuracy, but BGE-M3 stood out: it achieved the highest Recall@15 of 96.2% for Sinhala-English and 95.6% for Tamil-English. That outperforms the best QT approach, Google Translate, which reached 92.4% and 93.0% respectively. BGE-M3 also avoids translation overhead entirely, meaning no extra latency or cost from running a neural MT pipeline. The authors conclude that multilingual embedding models offer a more effective and scalable solution for cross-lingual retrieval-augmented generation (RAG) in low-resource government domains. The practical takeaway: for public-sector AI serving multilingual populations, embedding-based retrieval like BGE-M3 is a strong, production-ready alternative to classic translate-then-retrieve pipelines—especially when infrastructure and budgets are constrained.