Free AI Search Tool Behind Many Chatbots Just Got Faster and Smarter

Haystack is a free building block that software teams use to make AI chatbots smarter. The trick it enables is called RAG (letting AI look things up before answering). Instead of the AI guessing from memory, it searches your company's own documents first, then writes an answer. About 26,600 developers have starred the project on GitHub, so a change here quietly reaches a lot of workplace chatbots.

The headline improvement is speed. Before, when an AI pulled ten relevant passages from a document store, it asked the store ten separate times. Now it asks once and gets everything back. Think of it like a librarian fetching all your books in one trip instead of ten. The results are identical — just faster and less work for the system. The best part: existing setups get faster with no code changes at all.

Two other fixes matter. First, search in Chinese, Japanese and Korean was broken, because those languages don't put spaces between words — the tool was chopping sentences in the wrong places. Now it splits characters correctly, so these users can actually find things. Second, a security flaw in a supporting piece of software (called anyio) was patched; older versions were vulnerable, so teams should update.

The catch: this is a 'release candidate' — a test run before the real release, so bugs are possible. Also, one search method now returns fewer results than before, returning nothing if no word matches. Any team using a fixed score cutoff for filtering should re-check it, or they may suddenly get empty answers.