Agentic Inference Shift: AI's Future Compute Favors China and Space

Ben Thompson's 'The Inference Shift' draws a critical distinction between two types of AI inference: 'answer inference' (human-in-the-loop, speed-sensitive) and 'agentic inference' (fully autonomous, speed-irrelevant). He argues that agentic inference will dominate future market size, driving demand for fundamentally different compute architectures. This shift is good news for China's domestic chip ecosystem and space-based data centers (e.g., SpaceX's Starlink), which can provide massive, latency-tolerant compute—but potentially bad for Nvidia, whose strength lies in low-latency GPU interconnects built for training and answer inference.

Separately, Anthropic's deal to secure compute from xAI underscores market pragmatism: even rival labs collaborate when GPU supply is tight. For Elon Musk, the deal raises strategic questions—whether he'll lean into being a compute supplier (SpaceXAI) or compete directly with OpenAI. Meanwhile, OpenAI itself is forming a new 'deployment company' to operationalize AI at scale, and Apple has economic reasons to work with Intel for on-device AI. The broader theme: AI infrastructure is fragmenting along new lines of speed vs. scale, autonomy vs. interactivity.