TY - GEN
T1 - GenCoG
T2 - 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023
AU - Wang, Zihan
AU - Nie, Pengbo
AU - Miao, Xinyuan
AU - Chen, Yuting
AU - Wan, Chengcheng
AU - Bu, Lei
AU - Zhao, Jianjun
N1 - Publisher Copyright:
© 2023 ACM.
PY - 2023/7/12
Y1 - 2023/7/12
N2 - TVM is a popular deep learning (DL) compiler. It is designed for compiling DL models, which are naturally computation graphs, and as well promoting the efficiency of DL computation. State-of-the-art methods, such as Muffin and NNSmith, allow developers to generate computation graphs for testing DL compilers. However, these techniques are inefficient - their generated computation graphs are either type-invalid or inexpressive, and hence not able to test the core functionalities of a DL compiler. To tackle this problem, we propose GenCoG, a DSL-based approach to generating computation graphs for TVM testing. GenCoG is composed of (1) GenCoGL, a domain-specific language for specifying type constraints of operators, and (2) an approach that concolically solves type constraints and incrementally generates computation graphs of high expressivity. We implement and evaluate GenCoG on TVM releases. Our results show that GenCoG is effective in generating valid and expressive computation graphs - all of the GenCoG-generated graphs pass type checking, a critical graph validation stage; letting the graphs' expressivity be measured by their vertex and edge diversities, GenCoG outperforms state-of-the-arts by achieving 1.65∼6.93× in vertex diversity and 1.06∼7.08× in edge diversity, respectively. Furthermore, GenCoG has detected 16 bugs in TVM v0.8 and v0.9, with 14 confirmed and 12 fixed.
AB - TVM is a popular deep learning (DL) compiler. It is designed for compiling DL models, which are naturally computation graphs, and as well promoting the efficiency of DL computation. State-of-the-art methods, such as Muffin and NNSmith, allow developers to generate computation graphs for testing DL compilers. However, these techniques are inefficient - their generated computation graphs are either type-invalid or inexpressive, and hence not able to test the core functionalities of a DL compiler. To tackle this problem, we propose GenCoG, a DSL-based approach to generating computation graphs for TVM testing. GenCoG is composed of (1) GenCoGL, a domain-specific language for specifying type constraints of operators, and (2) an approach that concolically solves type constraints and incrementally generates computation graphs of high expressivity. We implement and evaluate GenCoG on TVM releases. Our results show that GenCoG is effective in generating valid and expressive computation graphs - all of the GenCoG-generated graphs pass type checking, a critical graph validation stage; letting the graphs' expressivity be measured by their vertex and edge diversities, GenCoG outperforms state-of-the-arts by achieving 1.65∼6.93× in vertex diversity and 1.06∼7.08× in edge diversity, respectively. Furthermore, GenCoG has detected 16 bugs in TVM v0.8 and v0.9, with 14 confirmed and 12 fixed.
KW - Computation Graph Generation
KW - Constraint Solving
KW - Deep Learning Compiler
UR - https://www.scopus.com/pages/publications/85167663525
U2 - 10.1145/3597926.3598105
DO - 10.1145/3597926.3598105
M3 - 会议稿件
AN - SCOPUS:85167663525
T3 - ISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
SP - 904
EP - 916
BT - ISSTA 2023 - Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis
A2 - Just, Rene
A2 - Fraser, Gordon
PB - Association for Computing Machinery, Inc
Y2 - 17 July 2023 through 21 July 2023
ER -