More cores hurt throughput: vector DB scaling paradox on HPC systems

Researchers from Argonne National Laboratory and the University of Chicago have published a large-scale evaluation of three state-of-the-art vector databases—Qdrant, Milvus, and Weaviate—on two production supercomputers, scaling to 256 distributed workers across 64 compute nodes. The study, now on arXiv, tested representative HPC workload patterns (mixed read/write and write-then-read) using popular benchmarks, multimodal embeddings, and a novel real-world scientific dataset. The results reveal a stark scaling paradox: adding more CPU cores can actually reduce query throughput by up to 30.67% in certain configurations, and scaling from 16 to 256 workers (a 16x increase) yields only a 5.46x improvement in performance.

This fundamental mismatch stems from the cloud-oriented design of current vector databases—optimized for low latency and elastic scaling—which fails to leverage HPC-specific features like high-bandwidth interconnects, shared memory, and NUMA-aware scheduling. The findings highlight a critical gap for emerging scientific AI workloads such as molecular search, meteorological trajectory detection, and literature-driven hypothesis generation, which demand efficient execution on supercomputers. The authors call for new HPC-aware vector database designs that can properly utilize the parallel architecture of modern large-scale systems.