OpenAI's New AI Makes Super-Fast Decisions for Apps

OpenAI has launched a new tool called the Decisions API, which helps software make fast, simple choices—like a super-fast classifier. Given a set of options, it picks one and gives a confidence score, all in a fraction of a second. This is different from large language models (LLMs) that generate text, which are slower and more expensive. The new tool is already being used by companies like Box and Datadog to speed up their apps.

For you, this means apps could become more responsive and cost less to run. For example, Speak, an AI language tutor, cut its response time from 1.4 seconds to just 0.13 seconds. That's like going from waiting for a webpage to load to it appearing instantly. Faster decisions also open up new possibilities, like real-time video analysis or smoother voice assistants. And because it's cheaper, companies might pass those savings on to you.

The catch is that these decision models aren't always better than LLMs for every task. They're great for quick choices but might struggle with complex reasoning. Also, there's no standard way to measure which decision model is best—it depends on what you're using it for. So while OpenAI's version is fast, competitors like TypeSafe's Jev model still lead in some benchmarks. You might not see a huge difference unless the apps you use adopt this technology.

Overall, OpenAI's move shows that AI is getting faster and more specialized. As these models get quicker—possibly down to 60 milliseconds—we could see them used in more everyday applications, making your digital life smoother and more efficient.