RedNote's Helmsman slashes ANNS costs 90% with clustering at scale

RedNote (Xiaohongshu), a global social network, has long relied on graph-based approximate nearest neighbor search (ANNS) like HNSW to power its search, recommendation, and advertising services under strict SLA requirements. However, the platform’s explosive growth in users and content caused memory footprints to balloon, driving massive capital and operational expenses. The team explored alternatives and found that clustering-based ANNS on all-flash storage could be promising—but only after overcoming kernel I/O overhead, fixed pruning strategies, and slow index construction. Their solution is Helmsman, a system accepted at OSDI’26 that combines three key innovations: an ANNS-oriented userspace storage stack, a leveling-learned pruning module, and GPU-accelerated pipelines for fast index construction.

Helmsman’s impact is staggering: it saves over 90% of hardware costs while enabling billion-scale index rebuilds within hours. In production, 40 machines now handle the same ANNS workloads that previously required roughly 35,000 CPU cores and 0.35 petabytes of DRAM. The system has been operating stably for several months, delivering high throughput and low latency without sacrificing accuracy. By ditching the memory-heavy HNSW approach and embracing clustering with flash storage, Helmsman demonstrates a cost-effective path to scaling vector similarity search for hyperscale platforms—a lesson that could reshape how social networks and recommender systems architect their infrastructure.