New AI Trick Keeps Apps Fast Without Buying Expensive Memory

Every data center faces a simple money problem: the fastest memory is small and pricey, and the bigger, cheaper memory is slow. So computers "tier" their data — keeping the stuff you're actively using in the fast lane and parking the rest in the slow lane. The catch is deciding what belongs where. The current software uses fixed rules of thumb, and those rules fall behind when your work suddenly shifts — say, a burst of traffic or a new phase of a job.

A team of researchers from the University of Wisconsin–Madison and their collaborators built MANTA, a small AI model that watches how memory is being used right now and predicts which pieces will be useful next. Think of it as a librarian who notices what you're reading and pre-pulls the next three books before you ask, instead of waiting for you to walk to the stacks.

The results are real but uneven. Across eight test workloads running on emulated "CXL" memory (a newer way to plug extra memory into servers), MANTA averaged about 12% faster than the existing method. On Intel Optane memory, a now-discontinued but common technology, it averaged 69% faster and hit 5.6 times faster on one workload. That's the difference between a web page loading instantly and you watching a spinner.

The catch: these are lab simulations on specific hardware, and the average gains are modest. Intel stopped making Optane, so the biggest wins apply to machines already in service. Still, the pattern matters — as AI models choke on memory, software that squeezes more out of the hardware you already own is a cheaper path than buying new chips.