Vosti: The AI Engine That Answers the Same Way Every Time
Same question, same answer, every time — this could make AI trustworthy for banks and hospitals.
Ask an AI chatbot the same question twice and you may get a slightly different answer. Some of that is the randomness setting you can switch off. But even with that off, results can still shift because of invisible behind-the-scenes plumbing: how the computer batches requests, splits up your prompt, and reuses memory. Those decisions should not change the answer. In practice, they do.
The researchers tested two widely used production tools, vLLM in its "batch-invariant" mode and SGLang in its "deterministic" mode. Both claim to solve this problem. The team found that under some ways of running the system, these modes still produce different internal numbers — the raw scores a model assigns to each possible next word before it picks one.
So they built their own engine, Vosti, designed so plumbing never leaks into results. It chooses its computation routines without peeking at the state of the running system, and it ties saved memory to the exact chunk of text it came from. Then they proved it: a machine-checked mathematical proof (a computer-verified receipt for their logic) shows that scheduling and memory sharing leave outputs untouched, while a separate analyzer confirms the chosen routines produce identical numbers across different batch sizes, question lengths, and memory layouts. In every test, Vosti returned results identical down to the last digit of the computer's precision — not just similar text — while matching vLLM's speed on typical workloads.
Why does byte-for-byte repeatability matter? If a bug appears, you can reproduce it instead of chasing a ghost. Banks, hospitals, and insurers that must show consistent decisions get a paper trail. Teams comparing two models can finally make a fair test. And when someone asks "why did the AI say that?", the answer is checkable. The catch: this is early research, not an app you can install today, and strict determinism can limit some speed tricks in other settings.
- Ask an AI the same question twice and you may get different numbers, even with randomness turned off, because of invisible behind-the-scenes juggling.
- Two popular tools, vLLM and SGLang, both claim to fix this — but researchers found they still differ in some cases.
- Vosti uses computer-checked math proofs to guarantee identical results, at speeds on par with today's best.
Why It Matters
Repeatable AI means reliable tests, fair audits, and trustworthy answers in finance, medicine, and law.