RAG improves reading recommendations by up to 35% across LLMs

A new arXiv paper by Sooyeon Kim and Piotr Maciąg presents a system that combines Retrieval-Augmented Generation (RAG) with three modern LLMs — Meta LLaMA 4 Scout, LLaMA 3.1 8B Instant, and Google Gemma2 9B — to produce personalized reading content. The architecture consists of four modules: Input, RAG, Generation, and Judging. Users provide a question and a target reading complexity level; the RAG module then retrieves relevant information from the web to ground the generated text. Three prompting strategies (Chain-of-Thought, zero-shot, few-shot) are tested, and an automatic LLM-as-a-Judge evaluates answer quality and alignment with the desired readability.

The experimental results are striking: RAG consistently improved performance across all models and prompting techniques, increasing relevance and particularly groundedness by 26–35 percentage points compared to non-RAG baselines. This means the system can deliver reading recommendations that are both factually accurate and tailored to the user's needs. The findings underscore the importance of grounding LLM outputs with external knowledge and demonstrate a scalable approach for educational platforms, content recommendation engines, and research assistants seeking to provide context-aware, personalized reading materials.