Don't Surrender to Low QPS/$: Fast and Cost-Efficient ANNS with TridentANN

The scale of vector data has been continuously growing. SSD-based approximate nearest neighbor search (ANNS) methods have become popular in handling billion-scale vectors with just one node. While delivering high performance in terms of query per second (QPS), they often fall short in the cost efficiency (i.e., QPS/$). In this paper, we present TridentanN, a highperformance yet cost-efficient system for large-scale ANNS with strengths of SSDs, low-end GPUs, and CPUs. We start by proposing a hybrid noise-clusters index for efficient I/O utilization and parallel search for decoupled noise and clusters. Second, we customize hardware-software architecture for GPU-SSD ANNS, which enables full bandwidth utilization when transferring data between GPUs and SSDs. Third, we orchestrate GPU-CPU-SSD task pipelines in parallel according to hardware characteristics to maximize system efficiency. Evaluations show that, compared to existing ANNS systems, TridentaNN performs 1.8-3.4× throughput and 21-70% of the latency. More importantly, it can be deployed on readily available and affordable devices, and achieves much higher cost efficiency (~2-4× QPS/$) than others.