Haystack's New Update Makes Long-Running AI Assistants Cheaper and Less Forgetful

Haystack, a free toolkit that developers use to build AI assistants, shipped an early test version called v3.2.0-rc1. The project has more than 26,000 stars on GitHub (a rough popularity score) and is made by deepset. The update tackles three of the most common complaints about "agents" — AI that can take actions like searching the web or filing a request on your behalf.

The first fix is about money. AI providers charge by "tokens" (chunks of text, roughly three-quarters of a word each). The new TokenBudgetHook lets a developer set a ceiling — say, 100,000 tokens — for a single task. When the ceiling is hit, the assistant stops cleanly before its next paid request, explains why it stopped, and hands back whatever work it finished. It can also stop after a specific action, like five web searches. Think of it as a prepaid phone plan that shuts off instead of letting you run up charges.

The second fix is about memory. Assistants can only hold so much conversation at once, and long tasks previously meant either dumping the oldest messages or paying to re-read everything. The new SummarizationCompactor replaces old exchanges with short AI-written summaries, starting with the oldest material and leaving the most recent steps untouched. A small, cheap model does the summarizing. It's like a note-taker who condenses the first hour of a meeting while keeping the last five minutes word for word.

The third improvement is convenience: building a workflow now takes far fewer lines of code, which speeds up development. The catch is that this is a release candidate — an unfinished test version — and its two headline features are labeled experimental. Some existing setups will also break when upgrading, meaning older projects need manual fixes. So the benefits will reach you through apps built on Haystack, not by you installing anything.