TY - GEN
T1 - ProphetAgent
T2 - 33rd ACM International Conference on the Foundations of Software Engineering, FSE Companion 2025
AU - Kong, Qichao
AU - Lv, Zhengwei
AU - Xiong, Yiheng
AU - Sun, Jingling
AU - Su, Ting
AU - Wang, Dingchun
AU - Li, Letao
AU - Yang, Xu
AU - Huo, Gang
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/7/28
Y1 - 2025/7/28
N2 - GUI tests is crucial for ensuring software quality and user satisfaction of mobile apps. In practice, companies often maintain extensive test cases written in natural language. Testers need to convert these test cases into executable scripts for regression and compatibility testing. Requirement changes or version updates often necessitate the addition and modification to these test cases. Thus, when faced with large volumes of test cases and regular updates, this process becomes costly, which is a common challenge across the industry. To address this issue, this paper proposes ProphetAgent that can automatically synthesize executable GUI tests from the test cases written in natural language. ProphetAgent first constructs a Clustered UI Transition Graph (CUTG) enriched with semantic information, then leverages large language models to generate the executable test case based on CUTG and test cases written in natural language. Experiment results show that ProphetAgent achieved a 78.1% success rate across 120 test cases in Douyin, Doubao, and six open-source apps, surpassing existing automated approaches (21.4% for AppAgent and 32.5% for AutoDroid). Additionally, statistical data from ByteDance’s testing platform show that ProphetAgent increased testers’ efficiency in synthesizing UI tests by 260%.
AB - GUI tests is crucial for ensuring software quality and user satisfaction of mobile apps. In practice, companies often maintain extensive test cases written in natural language. Testers need to convert these test cases into executable scripts for regression and compatibility testing. Requirement changes or version updates often necessitate the addition and modification to these test cases. Thus, when faced with large volumes of test cases and regular updates, this process becomes costly, which is a common challenge across the industry. To address this issue, this paper proposes ProphetAgent that can automatically synthesize executable GUI tests from the test cases written in natural language. ProphetAgent first constructs a Clustered UI Transition Graph (CUTG) enriched with semantic information, then leverages large language models to generate the executable test case based on CUTG and test cases written in natural language. Experiment results show that ProphetAgent achieved a 78.1% success rate across 120 test cases in Douyin, Doubao, and six open-source apps, surpassing existing automated approaches (21.4% for AppAgent and 32.5% for AutoDroid). Additionally, statistical data from ByteDance’s testing platform show that ProphetAgent increased testers’ efficiency in synthesizing UI tests by 260%.
KW - GUI Testing
KW - Large Language Model Agents
KW - Test Cases
UR - https://www.scopus.com/pages/publications/105013961463
U2 - 10.1145/3696630.3728543
DO - 10.1145/3696630.3728543
M3 - 会议稿件
AN - SCOPUS:105013961463
T3 - Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering
SP - 174
EP - 179
BT - FSE Companion 2025 - Companion Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering
A2 - Li, Jingyue
PB - Association for Computing Machinery
Y2 - 23 June 2025 through 27 June 2025
ER -