Near-free AI intelligence forces data system redesign for agents
Inference costs dropped 50x median - now under $0.10 per million tokens
The cost of AI inference is plummeting. GPT-4-class capabilities that cost $30 per million tokens in early 2023 now run under $1, and some providers push below $0.10. Across benchmarks, prices have fallen between 9x and 900x per year, with a median decline near 50x. Frontier models are getting cheaper each generation, and open-source models follow closely. This trend points to an era of virtually free intelligence sufficient for most knowledge work.
UC Berkeley's Aditya Parameswaran and collaborators argue this necessitates rethinking data systems along three axes: For Agents, Of Agents, and By Agents. Agents will soon dominate workload patterns, performing 'agentic speculation' — high-volume heterogeneous queries where 80-90% of sub-plans are duplicate across multiple agent attempts. Data systems can exploit multi-query optimization and approximate query processing to reuse results. Meanwhile, managing long-running swarms of agents requires new substrates for state, coordination, and failure handling. Finally, agents capable of synthesizing entire data systems pose verification challenges.
- Inference costs dropped from $30 to under $1 per million tokens (some below $0.10), median decline 50x per year
- Agentic speculation leads to 80-90% duplicate sub-queries across multiple agent attempts on the same task
- Three research directions: data systems for agent workloads, of agent swarms, and by agent-driven synthesis
Why It Matters
Near-free intelligence will supercharge AI agents, demanding entirely new data infrastructure architectures within the next few years.