Precise Identification of Problems for Structural Test Generation

Xusheng Xiao, Tao Xie, Nikolai Tillmann, and Jonathan de Halleux
North Carolina State University, USA; Microsoft Research, USA

When tools built for automated test-generation approaches are applied on complex programs in practice, these tools face two main types of problems: (1) external-method-call problem (EMCP), where method calls from external libraries prevent tools achieving higher coverage; (2) object-creation problem (OCP), where tools fails to generate desirable object states. Since tools are not powerful enough, the developers can provide guidance to help tools. To reduce the efforts of developers in providing guidance to tools, we propose Covana to precisely identify and report problems by computing data dependency between problem candidates and not-covered branches or statements. To show the effectiveness of Covana, we conduct evaluations on two open source projects. Our results show that Covana effectively identifies 43 EMCP out of 1610 EMCP candidates with only 1 false positive and 2 false negative, and 155 OCP out of 451 OCP candidates with 20 false positives and 28 false negatives.