New Open-Source Trick Makes AI Agents Finish Jobs Twice as Fast

AI agents are AI that doesn't just answer a question — it takes a series of actions for you. Ask one to research a topic and it might search the web, read five pages, write notes, then draft a report. Each step depends on the last, so you can end up waiting minutes. Most AI systems today are tuned for the wrong thing: how fast the first word appears. The PipeSwift team argues that for agents, only the total finishing time really matters.

Their fix is a mix of smarter scheduling and "pipeline parallelism" — splitting the AI's work across many chips like an assembly line instead of making one chip do everything in order. They found that the fastest setup isn't the one that prioritizes either reading or writing, but a balance in between. The team tested two enormous AI models (each over 360 billion parameters) on 64 Nvidia H800 chips, using replays of real coding and web-search jobs.

The results: PipeSwift finished those jobs up to 1.45 times faster than SGLang, today's leading open-source serving setup, 2.33 times faster than vLLM, and 1.54 times faster than the current best "disaggregated" approach. It's open source, so any company can use it free. The catch: these numbers come from lab replays, not live customer traffic, and the setup needs dozens of expensive data-center chips. You won't run this on a laptop.

So what? Faster, cheaper agent runs mean AI companies can offer you longer, more complex tasks — deep research, multi-file coding, automated shopping — at the same price, or drop prices to keep you. When your AI assistant stops making you stare at a spinner, this paper is part of why.