LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

Hang He; Chuhuai Yue; Chengqi Dong; Mingxue Tian; Hao Chen; Zhenfeng Liu; Jiajun Chai; Xiaohan Wang; Yufei Zhang; Qun Liao; Guojun Yin; Wei Lin; Chengcheng Wan; Haiying Sun; Ting Su

doi:10.1145/3770855.3817466

LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

Hang He
, Chuhuai Yue
, Chengqi Dong
, Mingxue Tian
, Hao Chen
, Zhenfeng Liu
, Jiajun Chai
, Xiaohan Wang
, Yufei Zhang
, Qun Liao
, Guojun Yin^*
, Wei Lin
, Chengcheng Wan^*
, Haiying Sun
, Ting Su^*

^*Corresponding author for this work

Software Engineering Institute

Meituan
East China Normal University
Beijing Institute of Technology
University of Science and Technology of China
Shanghai Jiao Tong University
North China University of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \bench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.

Original language	English
Title of host publication	KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
Publisher	Association for Computing Machinery
Pages	8989-9000
Number of pages	12
ISBN (Electronic)	9798400722592
DOIs	https://doi.org/10.1145/3770855.3817466
State	Published - 8 Aug 2026
Event	32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026 - Jeju Island, Korea, Republic of Duration: 9 Aug 2026 → 13 Aug 2026

Publication series

Name	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Volume	2-B
ISSN (Print)	2154-817X

Conference

Conference	32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026
Country/Territory	Korea, Republic of
City	Jeju Island
Period	9/08/26 → 13/08/26

Keywords

agentic search
benchmark
large reasoning model
local life services

Access to Document

10.1145/3770855.3817466

Cite this

He, H., Yue, C., Dong, C., Tian, M., Chen, H., Liu, Z., Chai, J., Wang, X., Zhang, Y., Liao, Q., Yin, G., Lin, W., Wan, C., Sun, H., & Su, T. (2026). LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. In KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (pp. 8989-9000). (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Vol. 2-B). Association for Computing Machinery. https://doi.org/10.1145/3770855.3817466

He, Hang ; Yue, Chuhuai ; Dong, Chengqi et al. / LocalSearchBench : Benchmarking Agentic Search in Real-World Local Life Services. KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Association for Computing Machinery, 2026. pp. 8989-9000 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

@inproceedings{67be43ac6a7f493d810f64b59e913eed,

title = "LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services",

abstract = "Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \textbackslash{}bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \textbackslash{}bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \textbackslash{}eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \textbackslash{}bench: the best model (DeepSeek-V3.2) achieves only 35.60\% correctness, and most models have issues with completeness (average 60.32\%) and faithfulness (average 30.72\%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.",

keywords = "agentic search, benchmark, large reasoning model, local life services",

author = "Hang He and Chuhuai Yue and Chengqi Dong and Mingxue Tian and Hao Chen and Zhenfeng Liu and Jiajun Chai and Xiaohan Wang and Yufei Zhang and Qun Liao and Guojun Yin and Wei Lin and Chengcheng Wan and Haiying Sun and Ting Su",

note = "Publisher Copyright: {\textcopyright} 2026 Owner/Author.; 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026 ; Conference date: 09-08-2026 Through 13-08-2026",

year = "2026",

month = aug,

day = "8",

doi = "10.1145/3770855.3817466",

language = "英语",

series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

publisher = "Association for Computing Machinery",

pages = "8989--9000",

booktitle = "KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2",

address = "美国",

}

He, H, Yue, C, Dong, C, Tian, M, Chen, H, Liu, Z, Chai, J, Wang, X, Zhang, Y, Liao, Q, Yin, G, Lin, W, Wan, C, Sun, H & Su, T 2026, LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. in KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 2-B, Association for Computing Machinery, pp. 8989-9000, 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026, Jeju Island, Korea, Republic of, 9/08/26. https://doi.org/10.1145/3770855.3817466

LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. / He, Hang; Yue, Chuhuai; Dong, Chengqi et al.
KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Association for Computing Machinery, 2026. p. 8989-9000 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Vol. 2-B).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - LocalSearchBench

T2 - 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026

AU - He, Hang

AU - Yue, Chuhuai

AU - Dong, Chengqi

AU - Tian, Mingxue

AU - Chen, Hao

AU - Liu, Zhenfeng

AU - Chai, Jiajun

AU - Wang, Xiaohan

AU - Zhang, Yufei

AU - Liao, Qun

AU - Yin, Guojun

AU - Lin, Wei

AU - Wan, Chengcheng

AU - Sun, Haiying

AU - Su, Ting

PY - 2026/8/8

Y1 - 2026/8/8

N2 - Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \bench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.

AB - Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \bench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.

KW - agentic search

KW - benchmark

KW - large reasoning model

KW - local life services

UR - https://www.scopus.com/pages/publications/105051031960

U2 - 10.1145/3770855.3817466

DO - 10.1145/3770855.3817466

M3 - 会议稿件

AN - SCOPUS:105051031960

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 8989

EP - 9000

BT - KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2

PB - Association for Computing Machinery

Y2 - 9 August 2026 through 13 August 2026

ER -

He H, Yue C, Dong C, Tian M, Chen H, Liu Z et al. LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. In KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Association for Computing Machinery. 2026. p. 8989-9000. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). doi: 10.1145/3770855.3817466

LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this