Amazon's New AI Tool Makes Business Decisions in Milliseconds

Amazon Web Services (AWS) now offers a managed service on SageMaker AI that helps businesses deploy AI models which make decisions rather than generate text. Using an open-source tool called llama.cpp, companies can run these models on regular CPUs or GPUs. The AI looks at a situation—like a customer's message—and returns probabilities for a set of predefined options, such as 'high priority' or 'low priority'. This avoids the slower process of generating full sentences.

For you, this could mean faster and more consistent automated decisions behind the scenes. For example, when you contact customer support, an AI might quickly route your issue to the right department. Or when you apply for a loan, the system could instantly score your application. Because these models can run on cheaper hardware, companies save money—savings that might eventually reach you through lower prices or better service.

The catch is that this only works for decisions with a fixed set of answers. You can't have a free-form conversation with it. Also, the AI still needs to process all the information, so it's not instant. And businesses must test it carefully to make sure the probabilities are reliable before trusting it with important choices.

Overall, this is part of a growing trend: AI that doesn't just chat, but actually takes actions or makes judgments. As these tools become easier to deploy, you'll likely interact with more AI-powered systems in your daily life—for better or worse.