LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

Hang He; Chuhuai Yue; Chengqi Dong; Mingxue Tian; Hao Chen; Zhenfeng Liu; Jiajun Chai; Xiaohan Wang; Yufei Zhang; Qun Liao; Guojun Yin; Wei Lin; Chengcheng Wan; Haiying Sun; Ting Su

doi:10.1145/3770855.3817466

LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

Hang He
, Chuhuai Yue
, Chengqi Dong
, Mingxue Tian
, Hao Chen
, Zhenfeng Liu
, Jiajun Chai
, Xiaohan Wang
, Yufei Zhang
, Qun Liao
, Guojun Yin^*
, Wei Lin
, Chengcheng Wan^*
, Haiying Sun
, Ting Su^*

^*此作品的通讯作者

软件工程学院

Meituan
East China Normal University
Beijing Institute of Technology
University of Science and Technology of China
Shanghai Jiao Tong University
North China University of Technology

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \bench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.

源语言	英语
主期刊名	KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
出版商	Association for Computing Machinery
页	8989-9000
页数	12
ISBN（电子版）	9798400722592
DOI	https://doi.org/10.1145/3770855.3817466
出版状态	已出版 - 8 8月 2026
活动	32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026 - Jeju Island, 韩国期限: 9 8月 2026 → 13 8月 2026

丛书

姓名	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
卷	2-B
ISSN（印刷版）	2154-817X

会议

会议	32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026
国家/地区	韩国
市	Jeju Island
时期	9/08/26 → 13/08/26

访问文件

10.1145/3770855.3817466

其它文件与链接

链接到 Scopus 的出版物

引用此

He, H., Yue, C., Dong, C., Tian, M., Chen, H., Liu, Z., Chai, J., Wang, X., Zhang, Y., Liao, Q., Yin, G., Lin, W., Wan, C., Sun, H., & Su, T. (2026). LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. 在 KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (页码 8989-9000). (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 卷 2-B). Association for Computing Machinery. https://doi.org/10.1145/3770855.3817466

He, Hang ; Yue, Chuhuai ; Dong, Chengqi 等. / LocalSearchBench : Benchmarking Agentic Search in Real-World Local Life Services. KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Association for Computing Machinery, 2026. 页码 8989-9000 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

@inproceedings{67be43ac6a7f493d810f64b59e913eed,

title = "LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services",

abstract = "Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \textbackslash{}bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \textbackslash{}bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \textbackslash{}eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \textbackslash{}bench: the best model (DeepSeek-V3.2) achieves only 35.60\% correctness, and most models have issues with completeness (average 60.32\%) and faithfulness (average 30.72\%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.",

keywords = "agentic search, benchmark, large reasoning model, local life services",

author = "Hang He and Chuhuai Yue and Chengqi Dong and Mingxue Tian and Hao Chen and Zhenfeng Liu and Jiajun Chai and Xiaohan Wang and Yufei Zhang and Qun Liao and Guojun Yin and Wei Lin and Chengcheng Wan and Haiying Sun and Ting Su",

note = "Publisher Copyright: {\textcopyright} 2026 Owner/Author.; 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026 ; Conference date: 09-08-2026 Through 13-08-2026",

year = "2026",

month = aug,

day = "8",

doi = "10.1145/3770855.3817466",

language = "英语",

series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

publisher = "Association for Computing Machinery",

pages = "8989--9000",

booktitle = "KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2",

address = "美国",

}

He, H, Yue, C, Dong, C, Tian, M, Chen, H, Liu, Z, Chai, J, Wang, X, Zhang, Y, Liao, Q, Yin, G, Lin, W, Wan, C, Sun, H & Su, T 2026, LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. 在 KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 卷 2-B, Association for Computing Machinery, 页码 8989-9000, 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026, Jeju Island, 韩国, 9/08/26. https://doi.org/10.1145/3770855.3817466

LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. / He, Hang; Yue, Chuhuai; Dong, Chengqi 等.
KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Association for Computing Machinery, 2026. 页码 8989-9000 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 卷 2-B).

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

TY - GEN

T1 - LocalSearchBench

T2 - 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026

AU - He, Hang

AU - Yue, Chuhuai

AU - Dong, Chengqi

AU - Tian, Mingxue

AU - Chen, Hao

AU - Liu, Zhenfeng

AU - Chai, Jiajun

AU - Wang, Xiaohan

AU - Zhang, Yufei

AU - Liao, Qun

AU - Yin, Guojun

AU - Lin, Wei

AU - Wan, Chengcheng

AU - Sun, Haiying

AU - Su, Ting

PY - 2026/8/8

Y1 - 2026/8/8

N2 - Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \bench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.

AB - Recent advances in large reasoning models (LRM s) have enabled agentic search systems to perform complex multi-step reasoning across multiple sources. However, most studies focus on general information retrieval and rarely explore vertical domains with unique challenges. In this work, we focus on local life services and introduce \bench, which encompasses diverse and complex business scenarios. Real-world queries in this domain are often ambiguous and require multi-hop reasoning across merchants and products, remaining challenging and not fully addressed. As the first comprehensive benchmark for agentic search in local life services, \bench comprises a database of over 1.3M merchant entries across 6 service categories and 9 major cities, and 900 multi-hop QA tasks from real user queries that require multi-step reasoning. We also developed \eval, a unified environment integrating multiple tools for LRM s interaction. Experiments show that even state-of-the-art LRM s struggle on \bench: the best model (DeepSeek-V3.2) achieves only 35.60% correctness, and most models have issues with completeness (average 60.32%) and faithfulness (average 30.72%). This highlights the need for specialized benchmarks and domain-specific agent training in local life services.

KW - agentic search

KW - benchmark

KW - large reasoning model

KW - local life services

UR - https://www.scopus.com/pages/publications/105051031960

U2 - 10.1145/3770855.3817466

DO - 10.1145/3770855.3817466

M3 - 会议稿件

AN - SCOPUS:105051031960

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 8989

EP - 9000

BT - KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2

PB - Association for Computing Machinery

Y2 - 9 August 2026 through 13 August 2026

ER -

He H, Yue C, Dong C, Tian M, Chen H, Liu Z 等. LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services. 在 KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. Association for Computing Machinery. 2026. 页码 8989-9000. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). doi: 10.1145/3770855.3817466

LocalSearchBench: Benchmarking Agentic Search in Real-World Local Life Services

摘要

丛书

会议

访问文件

其它文件与链接

学术指纹

引用此