Tree and Graph Based Two-Stages Routing for Approximate Nearest Neighbor Search

Jiannan Li, Zhenyu Zhang, Xiaoling Wang, Haoyang Li

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

With the expansion of extensive datasets of high-dimensional vectors, the approximate nearest neighbor search (ANNS) has become increasingly significant in data mining. Among the prevailing ANNS methods, graph-based methods achieve high recall at the cost of excessive memory usage, whereas tree-based methods are memory-efficient but suffer from low recall and increased search complexity. In this paper, we propose a hybrid index based on Ball tree and Hierarchical Navigable Small World(HNSW). Ball-HNSW employs a two-stages hybrid framework to accelerate the identification of approximate nearest neighbors. For datasets with different distributions, we utilize reinforcement learning to optimize parameter combinations by simulating decisions during the search process, thereby improving search results. Experiments confirm that Ball-HNSW outperforms other state-of-the-art methods on multiple datasets.

Original languageEnglish
Title of host publicationWeb and Big Data - 8th International Joint Conference, APWeb-WAIM 2024, Proceedings
EditorsWenjie Zhang, Zhengyi Yang, Xiaoyang Wang, Anthony Tung, Zhonglong Zheng, Hongjie Guo
PublisherSpringer Science and Business Media Deutschland GmbH
Pages376-390
Number of pages15
ISBN (Print)9789819772377
DOIs
StatePublished - 2024
Event8th Asia-Pacific Web and Web-Age Information Management Joint International Conference on Web and Big Data, APWeb-WAIM 2024 - Jinhua, China
Duration: 30 Aug 20241 Sep 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14963 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference8th Asia-Pacific Web and Web-Age Information Management Joint International Conference on Web and Big Data, APWeb-WAIM 2024
Country/TerritoryChina
CityJinhua
Period30/08/241/09/24

Keywords

  • Approximate nearest neighbor search
  • Graph Index
  • Hybrid Index
  • Reinforcement learning
  • Tree Index

Fingerprint

Dive into the research topics of 'Tree and Graph Based Two-Stages Routing for Approximate Nearest Neighbor Search'. Together they form a unique fingerprint.

Cite this