New Free Tool Makes AI Run Twice as Fast on Your Mac

If you use AI tools like ChatGPT on your Mac, you know the frustration of waiting for answers. A new open-source project called Kapitsa promises to cut that wait time dramatically. Developed by a small team, Kapitsa is a free engine that speeds up AI models on Apple's M-series chips. In benchmarks released this week, it outperformed established tools like Ollama and LM Studio by a wide margin.

The secret is a technique called speculative decoding. Normally, AI generates text one word at a time, which is slow. Kapitsa uses a smaller, faster model to guess the next several words, then the big model checks those guesses all at once. A new method called KFlash improves this by checking multiple guesses in a tree-like structure, so even if the first guess is wrong, others might be right. This results in decoding speeds of up to 201 tokens per second on a high-end MacBook Pro, compared to 15-32 for some competitors.

For everyday users, this means AI assistants could feel almost instant. Imagine asking for a Python script and getting it in a second instead of five. The tool works on all Apple Silicon Macs, from M1 to M6, and supports popular AI models like Qwen. It also provides a simple command-line interface and can serve as an API for other apps.

However, Kapitsa is still in early development. The developer admits it may have bugs and encourages users to report issues. Also, it requires some technical know-how to install and run. But for those willing to try, it's a free way to make your Mac's AI capabilities much faster. The team is asking for benchmark submissions from users to improve the tool further.