v2.26.0
The open-source RAG framework now supports Jinja2 templating for agents and introduces LLMRanker for semantic document reranking.
Deepset has launched Haystack v2.26, a significant update to their popular open-source framework for building production-ready retrieval-augmented generation (RAG) and AI agent applications. The release introduces several powerful features that enhance developer flexibility and system performance, with the standout addition being dynamic system prompts for agents using Jinja2 templating. This allows developers to inject runtime parameters and conditional logic directly into agent system messages, enabling adaptable behavior based on context, language requirements, or time-sensitive responses without redefining prompts from scratch.
The update also brings LLMRanker, a novel reranking component that leverages large language models to evaluate document relevance as a semantic reasoning task rather than traditional similarity scoring. This approach can yield superior results for complex or multi-step queries compared to conventional cross-encoders, while also filtering out irrelevant or duplicate documents to maintain lean context windows in RAG pipelines. Additionally, developers gain programmatic access to supported models through new SUPPORTED_MODELS class variables in components like OpenAIChatGenerator and AzureOpenAIResponsesChatGenerator, simplifying model selection and integration workflows.
Other notable improvements include enhanced input connection flexibility for components with union-typed inputs, better serialization support for the new LLMRanker, and expanded compatibility with the latest OpenAI models including GPT-5-mini and GPT-4o. These updates collectively strengthen Haystack's position as a comprehensive framework for building sophisticated AI applications that require dynamic behavior, high-quality retrieval, and seamless integration with various LLM providers.
- Agent component now supports Jinja2 templating in system_prompt for runtime parameter injection and conditional logic
- New LLMRanker component uses LLMs for semantic reranking, treating relevance as reasoning rather than similarity scoring
- Chat generator components expose SUPPORTED_MODELS class variables for programmatic model discovery
Why It Matters
Enables more dynamic, context-aware AI agents and improves RAG quality through semantic reranking for production applications.