CREDIT: New Software Makes Nvidia GPUs Run Up to 46% Faster

Think of a GPU as a giant team of workers processing data together. To cooperate, they often need to share information quickly. Nvidia chips offer a new way to do that called distributed shared memory, or DSMEM — sort of like a shared whiteboard that workers can read and write from anywhere in the room. But using that whiteboard has a cost: workers must pause to sync up, and write access isn't always faster than the old method.

CREDIT is a new software framework that decides when the whiteboard is actually worth using. It studies your specific task first, estimates the cost of coordination, and only switches to the shared-memory shortcut if it predicts a real speed boost. This is like a delivery route planner that knows when taking a shortcut will save time versus when sticking to the highway is better. It removes the guesswork for programmers.

The results are significant. On a top-end consumer Nvidia RTX 5090 GPU, CREDIT delivered an average speedup of 1.466x — nearly 50% faster. On Nvidia's powerful data-center H100 chip, it achieved 1.318x, or about 32% faster. It outperformed existing programming tools like CUTLASS and Triton across all six test workloads. And it correctly predicted when DSMEM would help or hurt 91.7% of the time.

For everyday users, this means AI chatbots, image generators, and scientific simulations could run faster and at lower cost — because data centers don't have to buy new chips to get more performance. The CREDIT source code is public, so developers can start integrating it now. The tradeoff? It only works with Nvidia hardware so far, and the speedup varies depending on the task. But as AI demands grow, squeezing more out of existing silicon is a win for both companies and consumers.