A Free Hack Makes Old Gaming PCs Run AI Twice as Fast

A Reddit user has published a free, modified version of llama.cpp — think of it as an engine that lets your own computer run an AI chatbot instead of renting one from OpenAI or Google. This version is specially tuned for Nvidia's RTX 30-series graphics cards, the beefy gaming chips sold from 2020 onward (the 3090 is the famous one). The creator says it pushes out more than 90 tokens per second, and a token is roughly three-quarters of a word — so about 65 words a second, far faster than you can read.

The bigger surprise is stamina. Every AI has a "context window" — the amount of text it can hold in mind at once. This setup claims to handle up to 240,000 tokens, roughly two full novels, normally a size that slows a home machine to a crawl. The creator says it keeps about 80% of its speed even at 200,000 tokens, while comparable free setups drop off sharply. That matters because slow, forgetful AI is annoying; fast, long-memoried AI is genuinely useful for reading documents, writing code, or sorting through a pile of email.

Why should you care? Running AI on your own hardware means no monthly subscription, no internet connection needed, and nothing you type leaves your computer. For anyone handling sensitive material — medical notes, legal contracts, financial records — that privacy is the whole point. A used 3090 costs several hundred dollars, but it's a one-time purchase that can replace years of AI subscription fees.

The honest catch: this is a community hobby project, not a company product. The speed numbers are self-reported and haven't been independently verified, the setup involves command-line tools and multi-gigabyte downloads, and it only works if you already own a capable Nvidia card with plenty of memory. If you're not comfortable tinkering, wait — improvements like this tend to get folded into friendlier, mainstream apps within months.