Amazon's New Trick Makes Big AI Start Up in Seconds

Every time you use an AI chatbot or app, the AI model has to be running on a computer somewhere. Picture a huge software file — the 'model weights,' which are the AI's learned knowledge. For a top model like DeepSeek-R1, that file is over 600 GB, roughly the size of 150 downloaded movies. Right now, when a company needs to start that AI on a new machine, the machine must download the whole thing plus a big chunk of software. That takes about 30 minutes. During those 30 minutes, the AI simply isn't available.

That's a real problem when lots of people suddenly show up. Say a news story goes viral and thousands of new users hit an AI app at once. The app's system automatically asks for five more machines to handle the traffic. The system responds instantly — but each new machine still has to download everything. So users wait, or get error messages, even though the app's code did the right thing.

Amazon's fix is called model caching. Instead of downloading on demand, Amazon quietly copies the AI files and the software onto each machine in advance, storing them on fast local drives. When a new machine is needed, it reads from that local drive at around 7 GB per second — like grabbing a book off your own shelf instead of ordering it online and waiting for delivery. Amazon says the AI can then start serving requests in seconds rather than tens of minutes. The files stay put even if the AI restarts, so the benefit repeats.

For you, this mostly shows up as a better experience: AI features that feel instant instead of broken, apps that survive sudden popularity without falling over, and companies spending less on idle computing — savings that could eventually show up in lower prices. The catch is that this only works on Amazon's specialized AI hosting service, and companies have to turn it on deliberately. It isn't automatic.