TY - GEN
T1 - Smartunit
T2 - 40th ACM/IEEE International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2018
AU - Zhang, Chengyu
AU - Yan, Yichen
AU - Zhou, Hanru
AU - Yao, Yinbo
AU - Wu, Ke
AU - Su, Ting
AU - Miao, Weikai
AU - Pu, Geguang
N1 - Publisher Copyright:
© 2018 ACM.
PY - 2018/5/27
Y1 - 2018/5/27
N2 - In this paper, we aim at the automated unit coverage-based testing for embedded software. To achieve the goal, by analyzing the industrial requirements and our previous work on automated unit testing tool CAUT, we rebuild a new tool, SmartUnit, to solve the engineering requirements that take place in our partner companies. SmartUnit is a dynamic symbolic execution implementation, which supports statement, branch, boundary value and MC/DC coverage. SmartUnit has been used to test more than one million lines of code in real projects. For confidentiality motives, we select three in-house real projects for the empirical evaluations. We also carry out our evaluations on two open source database projects, SQLite and PostgreSQL, to test the scalability of our tool since the scale of the embedded software project is mostly not large, 5K-50K lines of code on average. From our experimental results, in general, more than 90% of functions in commercial embedded software achieve 100% statement, branch, MC/DC coverage, more than 80% of functions in SQLite achieve 100% MC/DC coverage, and more than 60% of functions in PostgreSQL achieve 100% MC/DC coverage. Moreover, SmartUnit is able to find the runtime exceptions at the unit testing level. We also have reported exceptions like array index out of bounds and divided-by-zero in SQLite. Furthermore, we analyze the reasons of low coverage in automated unit testing in our setting and give a survey on the situation of manual unit testing with respect to automated unit testing in industry.
AB - In this paper, we aim at the automated unit coverage-based testing for embedded software. To achieve the goal, by analyzing the industrial requirements and our previous work on automated unit testing tool CAUT, we rebuild a new tool, SmartUnit, to solve the engineering requirements that take place in our partner companies. SmartUnit is a dynamic symbolic execution implementation, which supports statement, branch, boundary value and MC/DC coverage. SmartUnit has been used to test more than one million lines of code in real projects. For confidentiality motives, we select three in-house real projects for the empirical evaluations. We also carry out our evaluations on two open source database projects, SQLite and PostgreSQL, to test the scalability of our tool since the scale of the embedded software project is mostly not large, 5K-50K lines of code on average. From our experimental results, in general, more than 90% of functions in commercial embedded software achieve 100% statement, branch, MC/DC coverage, more than 80% of functions in SQLite achieve 100% MC/DC coverage, and more than 60% of functions in PostgreSQL achieve 100% MC/DC coverage. Moreover, SmartUnit is able to find the runtime exceptions at the unit testing level. We also have reported exceptions like array index out of bounds and divided-by-zero in SQLite. Furthermore, we analyze the reasons of low coverage in automated unit testing in our setting and give a survey on the situation of manual unit testing with respect to automated unit testing in industry.
KW - Automated unit testing
KW - Dynamic symbolic execution
KW - Embedded system
UR - https://www.scopus.com/pages/publications/85049690332
U2 - 10.1145/3183519.3183554
DO - 10.1145/3183519.3183554
M3 - 会议稿件
AN - SCOPUS:85049690332
T3 - Proceedings - International Conference on Software Engineering
SP - 296
EP - 305
BT - Proceedings 2018 ACM/IEEE 40th International Conference on Software Engineering
PB - IEEE Computer Society
Y2 - 27 May 2018 through 1 June 2018
ER -