Intel's agentic AI study: plan by agent density, not count, for enterprise readiness

Intel conducted thousands of experiments on agentic AI workloads to guide enterprise infrastructure planning. Their findings highlight that agentic AI is fundamentally a systems problem—far beyond LLM inference. They offer five practical lessons: treat agentic AI as a larger orchestration challenge, use agent density (agents per vCPU) as the primary capacity metric, monitor P95 task latency rather than average CPU utilization (since agents work in bursts), default to scale-out architectures for hosting agents, and reserve scale-up only for workloads with heavier per-agent compute needs. They also propose a balanced set of six metrics for platform teams: task success rate, cost per task, time per task, task throughput, agent density, and latency.

To measure real-world performance, Intel extended the open-source Terminal-Bench benchmark with profiling, telemetry, and replay capabilities. This allows teams to trace where agents spend time beyond inference, using deterministic record-replay of LLM responses to eliminate variability. The benchmark included diverse enterprise-like tasks (compilation, testing, database ops, ray tracing, etc.). The study advances a three-phase deployment approach: plan capacity by normalized agent density (e.g., 10 agents on 8 vCPUs vs. 20 on 16 scale similarly), implement new observability centered on P95 latency, and decide scaling strategy based on workload type—interactive copilots need low density, while batch IT workflows can handle higher density. This framework answers practical questions about system capacity, scaling, and cost.