Haystack 3.0 launches production-grade agents with hooks and async
New hooks system allows human-in-the-loop and dynamic tool selection at runtime.
deepset has officially released Haystack 3.0, a landmark version of their open-source AI framework that shifts focus from simple pipelines to production-grade agents. The headline feature is a revamped Agent with a general-purpose hooks system that lets developers inject custom logic at key lifecycle points: before_run, before_llm, before_tool, after_tool, on_exit, and after_run. This enables guardrails, audit logging, and human-in-the-loop checkpoints without modifying the agent's internals. Skills are now first-class citizens with progressive discovery—the model only sees tool names and one-line descriptions until it loads a skill, keeping context efficient. Tools can be dynamically selected at runtime by passing `tools=...` to `run()` or `run_async()`, and native async tool functions are supported via `@tool` decorators. The standalone ToolInvoker has been removed, with tool execution fully owned by the Agent.
For serving workloads, Haystack 3.0 unifies Pipeline and AsyncPipeline into a single class that exposes `run()`, `run_async()`, `run_async_generator()`, and `stream()`. Concurrent tool calls and token-by-token streaming come standard, and a symmetric lifecycle (`warm_up`/`close`) ensures long-running services don't leak connections or GPU memory. Built-in introspection exposes `step_count`, `token_usage`, and `tool_call_counts` as state variables, while tracing emits dedicated spans (e.g., `haystack.agent.step` with nested `.llm`/`.tool` children) tagged with actual tool usage. The core is leaner: legacy Generators are removed, `haystack-experimental` is no longer a core dependency, and 30 components (e.g., Sentence Transformers, Hugging Face, Whisper, Azure OCR) now live in independently released packages. Migration is straightforward with a dedicated guide.
- New hooks system (before_run, before_llm, before_tool, etc.) enables custom guardrails and human-in-the-loop without modifying internals
- Unified Pipeline class supporting synchronous run, async run_async, streaming, and concurrent tool calls for serving
- Built-in introspection exposes step_count, token_usage, and tool_call_counts; tracing emits dedicated spans with tool names
Why It Matters
Enables developers to build scalable, observable, and flexible AI agents with full control and production readiness.