New AI Math Trick Makes Chatbots Smarter Without Extra Cost

models like ChatGPT are huge and expensive to run. To save money, companies compress them using a technique called quantization, which reduces the precision of numbers. But this can make the AI dumber—it might give wrong answers or write buggy code. ResServe fixes that by adding a small 'residue' that captures the lost information, using spare computing power that's normally wasted.

The key insight is that during certain operations, the computer's hardware isn't fully used. ResServe takes advantage of that idle capacity to do extra calculations that improve accuracy. It's like using the downtime of a machine to double-check its work. On NVIDIA's latest B200 chip, this approach kept speed nearly the same as uncompressed models while boosting accuracy significantly.

In tests across five AI models, ResServe improved code-generation accuracy by 15% on average for one format and 54% for another. That means fewer bugs and better suggestions when you ask an AI to write code. And it did this while reducing latency by up to 53% compared to a higher-precision alternative. So not only is it smarter, it's also faster.

For everyday users, this could mean AI assistants that make fewer mistakes and respond quicker. For companies, it could lower the cost of running AI services, potentially making them cheaper or more powerful. The catch? It requires specific hardware (like NVIDIA's B200) and is currently focused on code generation, though the principle could apply to other tasks.