Publications

2026

1. TOSEM

   Complete, Sound, and Scalable Identification of Minimal Failure-Causing Schema

   Runshuo Xie, Xintao Niu*, Huayao Wu, and 1 more author

   ACM Transactions on Software Engineering and Methodology, Apr 2026

   * corresponding author

   Bib HTML

   @article{xie2026schema,
     author = {Xie, Runshuo and Niu*, Xintao and Wu, Huayao and Nie, Changhai},
     title = {Complete, Sound, and Scalable Identification of Minimal Failure-Causing Schema},
     journal = {ACM Transactions on Software Engineering and Methodology},
     year = {2026},
     month = apr,
     publisher = {Association for Computing Machinery},
     issn = {1557-7392},
     doi = {10.1145/3811916},
     note = {* corresponding author},
   }

2. OOPSLA

   Automated Debugging of Datalog Programs

   Jiashen Wei#, Baoyuan Luo#, Runshuo Xie, Yun Qi, Yiyu Zhang, Xizao Wang, Xintao Niu*, and Zhiqiang Zuo*

   In ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, Apr 2026

   Conditionally accepted by OOPSLA 2026. #equal contribution, * corresponding authors

   Bib

   @inproceedings{wei2026datalog,
     author = {Wei#, Jiashen and Luo#, Baoyuan and Xie, Runshuo and Qi, Yun and Zhang, Yiyu and Wang, Xizao and Niu*, Xintao and Zuo*, Zhiqiang},
     title = {Automated Debugging of Datalog Programs},
     booktitle = {ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications},
     year = {2026},
     note = {Conditionally accepted by OOPSLA 2026. #equal contribution, * corresponding authors},
   }

3. TSE

   How Composite Metamorphic Relations Enhance Test Effectiveness of DNN Testing: An Empirical Study

   Huayao Wu, Peng Wang, Shengyou Hu, Xintao Niu, Changhai Nie, and Tsong Yueh Chen

   IEEE Transactions on Software Engineering, Apr 2026

   Bib

   @article{wu2026composite,
     author = {Wu, Huayao and Wang, Peng and Hu, Shengyou and Niu, Xintao and Nie, Changhai and Chen, Tsong Yueh},
     title = {How Composite Metamorphic Relations Enhance Test Effectiveness of DNN Testing: An Empirical Study},
     journal = {IEEE Transactions on Software Engineering},
     year = {2026},
     volume = {52},
     number = {4},
     pages = {1617--1636},
   }

2025

1. TOSEM

   SETS: A Simple yet Effective DNN Test Selection Approach

   Jingling Wang, Huayao Wu, Peng Wang, Xintao Niu, and Changhai Nie

   ACM Trans. Softw. Eng. Methodol., Oct 2025

   Just Accepted

   Abs

   To reduce the substantial manual annotation costs in testing deep neural networks (DNNs), various test selection approaches have been proposed, with uncertainty serving as a key objective to trigger more failures. However, focusing solely on uncertainty may expose failures caused by the same underlying fault in the model. Accordingly, recent studies have introduced diversity as another important objective, proposing multi-objective search-based approaches to select test inputs that can reveal distinct faults. Despite their effectiveness, these approaches typically require long execution times, limiting their efficiency in large-scale datasets. In this paper, we propose SETS, a simple yet effective multi-objective DNN test selection approach. Like existing multi-objective approaches, SETS optimizes both uncertainty and diversity to maximize the number of unique faults detected by the selected test inputs. But, SETS prioritizes high-uncertainty test inputs to reduce the candidate set size, and employs an efficient greedy strategy to further reduce the number of fitness evaluations. Thus, SETS differentiates itself by considerably improving the efficiency of selection while preserving effectiveness. We evaluate SETS on eight subjects, demonstrating its significant superiority over baseline approaches. SETS can operate on average 42.98 times faster while still achieving better fault detection and model retraining with its simple implementation.

2. Internetware

   Cluster-Based Multi-Objective Metamorphic Test Case Pair Selection for Deep Neural Networks

   Jingling Wang, Shuwei Qiu, Peng Wang, Jiyuan Song, Huayao Wu, Xintao Niu, and Changhai Nie

   In Proceedings of the 16th International Conference on Internetware, Oct 2025

   Abs

   Due to the rapid development of deep neural networks (DNNs), ensuring their quality has become increasingly important. However, the test oracle problem poses an obstacle to DNN testing because of the massive unlabeled data. Metamorphic Testing (MT) has proven effective in alleviating the test oracle problem, and many efforts have been made to improve the cost-effectiveness of MT for DNNs. Some approaches focus on selecting good metamorphic relations (MRs), while others target the selection of suspicious source test cases. Since follow-up test cases are generated by combining source test cases with MRs, selecting effective pairs of source test cases and MRs is also quite essential and beneficial for MT. In this paper, we propose CMPS, a multi-objective black-box approach for metamorphic test case pair selection. Considering both uncertainty and diversity, CMPS aims to select pairs that can detect more unique faults in the model. It evaluates uncertainty based on model outputs and assesses diversity through clustering source test cases. Furthermore, CMPS can adaptively optimize the selection process based on feedback from the execution results of the selected pairs. We conduct extensive experiments on three datasets and five DNN models to evaluate CMPS’s performance. The experimental results demonstrate that CMPS significantly outperforms baseline approaches in both failure triggering and fault detection.

3. TSC

   A Systematic Literature Review on Fault Injection Testing of Microservice Systems

   Senyao Yu, Huayao Wu, Xintao Niu, and 1 more author

   IEEE Transactions on Services Computing, Oct 2025
4. ISSTA

   Effective REST APIs Testing with Error Message Analysis

   Lixin Xu, Huayao Wu, Zhenyu Pan, Tongtong Xu, Shaohua Wang, Xintao Niu, and  Changhai Nie

   In International Symposium on Software Testing and Analysis, Oct 2025

   Abs Bib

   REST APIs are essential in building modern enterprise systems, while effectively examining their behaviors remains challenging due to the difficulty in inferring constraints from the specifications. To generate valid test inputs for REST APIs, existing approaches are typically feedback-driven, leveraging HTTP status codes received to guide further test input generation. However, these approaches overlook the potentially valuable information described in error messages accompanying HTTP status codes, leading to inefficiencies in exploring the input space of REST APIs. In this paper, we propose EmRest, a black-box testing approach that leverages error message analysis to enhance both valid and exceptional test input generation for REST APIs. For each operation under test, EmRest first identifies all possible value assignment strategies for each of its input parameters. It then repeatedly applies combinatorial testing to sample test inputs based on these strategies, and statistically analyzes the error messages (of 400-range status code) received to infer and exclude invalid combinations of value assignment strategies (i.e., constraints of the input space). Additionally, EmRest seeks to mutate valid value assignment strategies that are finally identified to generate test inputs for exceptional testing. The error messages (of 500-range status code) received are categorized to identify bug-prone operations, for which more testing resources are allocated. Our experimental results on 16 real-world REST APIs demonstrates the effectiveness of EmRest. It achieves higher operation coverage than state-of-the-art approaches in 50% of APIs, and detects 226 unique bugs that cannot be found by the other approaches.

   @inproceedings{xu2025,
     title = {Effective REST APIs Testing with Error Message Analysis},
     booktitle = {International Symposium on Software Testing and Analysis},
     author = {Xu, Lixin and Wu, Huayao and Pan, Zhenyu and Xu, Tongtong and Wang, Shaohua and Niu, Xintao and and Changhai Nie},
     year = {2025},
     publisher = {ACM},
   }

2024

1. IST

   Top-down: A better strategy for incremental covering array generation

   Yan Wang, Xintao Niu*, Huayao Wu, and 4 more authors

   Information and Software Technology, Oct 2024

   Bib

   @article{Wangyan2024,
     author = {Wang, Yan and Niu*, Xintao and Wu, Huayao and Nie, Changhai and Yu, Lei and Wang, Xiaoyin and Xu, Jiaxi},
     title = {Top-down: A better strategy for incremental covering array generation},
     journal = {Information and Software Technology},
     year = {2024},
   }

2. ISSRE

   A Combinatorial Interaction Testing Method for Multi-Label Image Classifier

   Peng Wang, Shengyou Hu, Huayao Wu, Xintao Niu, Changhai Nie, and Lin Chen

   In International Symposium on Software Reliability Engineering, Oct 2024

   Bib

   @inproceedings{Wangpeng2024,
     author = {Wang, Peng and Hu, Shengyou and Wu, Huayao and Niu, Xintao and Nie, Changhai and Chen, Lin},
     title = {A Combinatorial Interaction Testing Method for Multi-Label Image Classifier},
     year = {2024},
     booktitle = {International Symposium on Software Reliability Engineering},
   }

2023

1. ASE

   ATOM: Automated Black-Box Testing of Multi-Label Image Classification Systems

   Shengyou Hu, Huayao Wu, Peng Wang, Jing Chang, Yongjun Tu, Xiu Jiang, Xintao Niu, and Changhai Nie

   In 2023 38th IEEE/ACM International Conference on Automated Software Engineering, Oct 2023

   Bib

   @inproceedings{Hushengyou2023,
     author = {Hu, Shengyou and Wu, Huayao and Wang, Peng and Chang, Jing and Tu, Yongjun and Jiang, Xiu and Niu, Xintao and Nie, Changhai},
     booktitle = {2023 38th IEEE/ACM International Conference on Automated Software Engineering},
     title = {ATOM: Automated Black-Box Testing of Multi-Label Image Classification Systems},
     year = {2023},
     pages = {230-242},
     keywords = {Microwave integrated circuits;Correlation;Systematics;Closed box;Artificial neural networks;Search engines;Tagging;Multi-label Image Classification Testing;Black-box Testing;Metamorphic Testing},
     doi = {10.1109/ASE56229.2023.00156},
   }

2. TSE

   Enhancing Fault Injection Testing of Service Systems via Fault-Tolerance Bottleneck

   Huayao Wu, Senyao Yu, Xintao Niu, and 4 more authors

   IEEE Transactions on Software Engineering, Oct 2023

   Abs Bib HTML

   Modern large-scale service systems are usually deployed with redundant components to ensure high dependability in distributed and volatile environments. Fault Injection Testing (FIT) is a popular technique for testing such systems, while the application of FIT to validating the correctness of redundant components remains a challenging task, especially when the system’s structural information is unavailable when testing starts. In this study, we refer to a minimum set of faults that, when injected, will cut off all execution paths in a service system as a fault-tolerance bottleneck, and we propose a novel Fault-tolerance Bottleneck driven Fault Injection (FBFI) approach to the exploration and validation of redundant components without prior knowledge of the system’s business structure. The core idea of FBFI is to iteratively infer and inject bottlenecks of the business structure constructed so far. In this way, FBFI is able to discover and test redundant components by repeatedly triggering new system behaviors. The effectiveness and efficiency of FBFI is evaluated using two microservice benchmark systems with different deployment scales. The results reveal that FBFI is more practical and cost-effective than random and lineage-driven FIT approaches in testing service systems of high redundancy levels.

   @article{wu2023enhance,
     title = {Enhancing Fault Injection Testing of Service Systems via Fault-Tolerance Bottleneck},
     author = {Wu, Huayao and Yu, Senyao and Niu, Xintao and Nie, Changhai and Pei, Yu and He, Qiang and Yang, Yun},
     journal = {IEEE Transactions on Software Engineering},
     year = {2023},
     publisher = {IEEE},
     volume = {1},
     number = {1},
     pages = {1--1},
     doi = {10.1109/TSE.2023.3285357},
   }

3. TOSEM

   Toward More Efficient Statistical Debugging with Abstraction Refinement

   Zhiqiang Zuo, Xintao Niu, Siyi Zhang, and 5 more authors

   ACM Trans. Softw. Eng. Methodol., Mar 2023

   Abs Bib HTML

   Debugging is known to be a notoriously painstaking and time-consuming task. As one major family of automated debugging, statistical debugging approaches have been well investigated over the past decade, which collect failing and passing executions and apply statistical techniques to identify discriminative elements as potential bug causes. Most of the existing approaches instrument the entire program to produce execution profiles for debugging, thus incurring hefty instrumentation and analysis cost. However, as in fact a major part of the program code is error-free, full-scale program instrumentation is wasteful and unnecessary. This article presents a systematic abstraction refinement-based pruning technique for statistical debugging. Our technique only needs to instrument and analyze the code partially. While guided by a mathematically rigorous analysis, our technique is guaranteed to produce the same debugging results as an exhaustive analysis in deterministic settings. With the help of the effective and safe pruning, our technique greatly saves the cost of failure diagnosis without sacrificing any debugging capability. We apply this technique to two different statistical debugging scenarios: in-house and production-run statistical debugging. The comprehensive evaluations validate that our technique can significantly improve the efficiency of statistical debugging in both scenarios, while without jeopardizing the debugging capability.

   @article{zzhiqiang2023,
     author = {Zuo, Zhiqiang and Niu, Xintao and Zhang, Siyi and Fang, Lu and Khoo, Siau Cheng and Lu, Shan and Sun, Chengnian and Xu, Guoqing Harry},
     title = {Toward More Efficient Statistical Debugging with Abstraction Refinement},
     year = {2023},
     issue_date = {March 2023},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     volume = {32},
     number = {2},
     issn = {1049-331X},
     doi = {10.1145/3544790},
     journal = {ACM Trans. Softw. Eng. Methodol.},
     month = mar,
     articleno = {36},
     numpages = {38},
   }

2022

1. ICSE

   Combinatorial Testing of RESTful APIs

   Huayao Wu, Lixin Xu, Xintao Niu, and 1 more author

   In 2022 IEEE/ACM 44rd International Conference on Software Engineering, Mar 2022

   Abs Bib HTML

   This paper presents RestCT, a systematic and fully automatic approach that adopts Combinatorial Testing (CT) to test RESTful APIs. RestCT is systematic in that it covers and tests not only the interactions of a certain number of operations in RESTful APIs, but also the interactions of particular input-parameters in every single operation. This is realised by a novel two-phase test case generation approach, which first generates a constrained sequence covering array to determine the execution orders of available operations, and then applies an adaptive strategy to generate and refine several constrained covering arrays to concretise input-parameters of each operation. RestCT is also automatic in that its application relies on only a given Swagger specification of RESTful APIs. The creation of CT test models (especially, the inferring of dependency relationships in both operations and input-parameters), and the generation and execution of test cases are performed without any human intervention. Experimental results on 11 real-world RESTful APIs demonstrate the effectiveness and efficiency of RestCT. In particular, RestCT can find eight new bugs, where only one of them can be triggered by the state-of-the-art testing tool of RESTful APIs.

   @inproceedings{wu2022restful,
     title = {Combinatorial Testing of RESTful APIs},
     author = {Wu, Huayao and Xu, Lixin and Niu, Xintao and Nie, Changhai},
     booktitle = {2022 IEEE/ACM 44rd International Conference on Software Engineering},
     pages = {426--437},
     year = {2022},
     organization = {IEEE},
     doi = {10.1145/3510003.3510151}
   }

2. TSE

   A theory of pending schemas in combinatorial testing

   Xintao Niu, Huayao Wu, Changhai Nie, and 2 more authors

   IEEE Transactions on Software Engineering, Mar 2022

   Abs Bib HTML

   Combinatorial Testing (CT) is an effective testing technique for detecting failures which are triggered by the interactions of various factors that influence the behaviour of a system. Although many studies in CT have designed elaborate test suites (called covering arrays) to systemically check each possible factor interaction, they provide weak support to locate the concrete failure-inducing interactions, i.e., the Minimal Failure-causing Schemas (MFS). To this end, a variety of MFS identification approaches have been proposed. However, as this study reveals, these approaches suffer from various issues such as cannot identify multiple overlapping MFSs, cannot handle MFSs with high degrees, cannot be applied to systems with large number of parameters, etc. These issues are essentially caused by the exponential computing complexity of checking every interaction in the test cases. Therefore, they can only focus on a subset of all the possible interactions, resulting in many interactions unnoticed. Ignoring these unnoticed interactions could potentially cause failures that have never been systematically checked. Hence, it is beneficial for MFS identification approaches to identify these interactions. In order to account for these unnoticed interactions in CT, this study introduces the notion of pending schema, based on which a theoretical framework of CT schemas is established. In particular, we formally define the determinability of a schema in CT with respect to given information; as such, the yet-to-be determined schemas are exactly the pending schemas. The relationships between the different schemas (faulty, healthy, and pending) and test cases are also theoretically analyzed. Based on which, we further propose three formulas, along with three corresponding algorithms, for the identification of the pending schemas in failing test cases, and formally prove their correctness. As a result, we reduce the complexity of obtaining pending schemas with respect to the number of factors that may have influences on the software.

   @article{niu2022theory,
     title = {A theory of pending schemas in combinatorial testing},
     author = {Niu, Xintao and Wu, Huayao and Nie, Changhai and Lei, Yu and Wang, Xiaoyin},
     journal = {IEEE Transactions on Software Engineering},
     year = {2022},
     publisher = {IEEE},
     volume = {48},
     number = {10},
     pages = {4119--4151},
     doi = {10.1109/TSE.2021.3113920}
   }

3. IST

   An Adaptive Penalty based Parallel Tabu Search for Constrained Covering Array Generation

   Yan Wang, Huayao Wu, Xintao Niu, and 2 more authors

   Information and Software Technology, Mar 2022

   Bib HTML

   @article{wang2022adaptive,
     title = {An Adaptive Penalty based Parallel Tabu Search for Constrained Covering Array Generation},
     author = {Wang, Yan and Wu, Huayao and Niu, Xintao and Nie, Changhai and Xu, Jiaxi},
     journal = {Information and Software Technology},
     volume = {143},
     pages = {106768},
     year = {2022},
     publisher = {Elsevier},
     doi = {10.1016/j.infsof.2021.106768}
   }

4. TSE

   Enhance Combinatorial Testing with Metamorphic Relations

   Xintao Niu, Yanjie Sun, Huayao Wu, and 4 more authors

   IEEE Transactions on Software Engineering, Mar 2022

   Abs Bib HTML

   Due to the effectiveness and efficiency in detecting defects caused by interactions of multiple factors, Combinatorial Testing (CT) has received considerable scholarly attention in the last decades. Despite numerous practical test case generation techniques being developed, there remains a paucity of studies addressing the automated oracle generation problem, which holds back the overall automation of CT. As a consequence, much human intervention is inevitable, which is time-consuming and error-prone. This costly manual task also restricts the application of higher testing strength, inhibiting the full exploitation of CT in industrial practice. To bridge the gap between test designs and fully automated test flows, and to extend the applicability of CT, this paper presents a novel CT methodology, named COMER, to enhance the traditional CT by accounting for Metamorphic Relations (MRs). COMER puts a high priority on generating pairs of test cases which match the input rules of MRs, i.e., the Metamorphic Group (MG), such that the correctness can be automatically determined by verifying whether the outputs of these test cases violate their MRs. As a result, COMER can not only satisfy the t-way coverage as what CT does, but also automatically check as many test oracle violations as possible. Several empirical studies conducted on 31 real-world software projects have shown that COMER increased the number of metamorphic groups by an average factor of 75.9 and also increased the failure detection rate by an average factor of 11.3, when compared with CT, while the overall number of test cases generated by COMER barely increased.

   @article{niu2021enhance,
     title = {Enhance Combinatorial Testing with Metamorphic Relations},
     author = {Niu, Xintao and Sun, Yanjie and Wu, Huayao and Li, Gang and Nie, Changhai and Lei, Yu and Wang, Xiaoyin},
     journal = {IEEE Transactions on Software Engineering},
     year = {2022},
     publisher = {IEEE},
     volume = {48},
     number = {12},
     pages = {5007--5029},
     doi = {10.1109/TSE.2021.3131548},
   }

2021

1. ICSE

   Identifying Key Features from App User Reviews

   Huayao Wu, Wenjun Deng, Xintao Niu, and 1 more author

   In 2021 IEEE/ACM 43rd International Conference on Software Engineering, Mar 2021

   Abs Bib HTML

   Due to the rapid growth and strong competition of mobile application (app) market, app developers should not only offer users with attractive new features, but also carefully maintain and improve existing features based on users’ feedbacks. User reviews indicate a rich source of information to plan such feature maintenance activities, and it could be of great benefit for developers to evaluate and magnify the contribution of specific features to the overall success of their apps. In this study, we refer to the features that are highly correlated to app ratings as key features, and we present KEFE, a novel approach that leverages app description and user reviews to identify key features of a given app. The application of KEFE especially relies on natural language processing, deep machine learning classifier, and regression analysis technique, which involves three main steps: 1) extracting feature-describing phrases from app description; 2) matching each app feature with its relevant user reviews; and 3) building a regression model to identify features that have significant relationships with app ratings. To train and evaluate KEFE, we collect 200 app descriptions and 1,108,148 user reviews from Chinese Apple App Store. Experimental results demonstrate the effectiveness of KEFE in feature extraction, where an average F-measure of 78.13% is achieved. The key features identified are also likely to provide hints for successful app releases, as for the releases that receive higher app ratings, 70% of features improvements are related to key features.

   @inproceedings{wu2021identifying,
     title = {Identifying Key Features from App User Reviews},
     author = {Wu, Huayao and Deng, Wenjun and Niu, Xintao and Nie, Changhai},
     booktitle = {2021 IEEE/ACM 43rd International Conference on Software Engineering},
     pages = {922--932},
     year = {2021},
     organization = {IEEE},
     doi = {10.1109/ICSE43902.2021.00088}
   }

2020

1. TSE

   Identifying Failure-Causing Schemas in the Presence of Multiple Faults

   Xintao Niu, Changhai Nie, Yu Lei, and 2 more authors

   IEEE Transactions on Software Engineering, Feb 2020

   Abs Bib HTML

   Combinatorial testing (CT) has been proven effective in revealing the failures caused by the interaction of factors that affect the behavior of a system. The theory of Minimal Failure-Causing Schema (MFS) has been proposed to isolate the cause of a failure after CT. Most algorithms that aim to identify MFS focus on handling a single fault in the System Under Test (SUT). However, we argue that multiple faults are more common in practice, under which masking effects may be triggered so that some failures cannot be observed. The traditional MFS theory lacks a mechanism to handle such effects; hence, they may incorrectly isolate the MFS. To address this problem, we propose a new MFS model that takes into account multiple faults. We first formally analyze the impact of the multiple faults on existing MFS identifying algorithms, especially in situations where masking effects are triggered by multiple faults. We then develop an approach that can assist traditional algorithms to better handle multiple faults. Empirical studies were conducted using several kinds of open-source software, which showed that multiple faults with masking effects do negatively affect traditional MFS identifying approaches and that our approach can help to alleviate these effects.

   @article{niu2020identifying,
     author = {Niu, Xintao and Nie, Changhai and Lei, Yu and Leung, Hareton and Wang, Xiaoyin},
     journal = {IEEE Transactions on Software Engineering},
     title = {Identifying Failure-Causing Schemas in the Presence of Multiple Faults},
     year = {2020},
     volume = {46},
     number = {2},
     pages = {141-162},
     keywords = {Testing;Bars;Fault diagnosis;Computer bugs;Software algorithms;Open source software;Software testing;combinatorial testing;failure-causing schemas;masking effects},
     doi = {10.1109/TSE.2018.2844259},
     issn = {2326-3881},
     month = feb,
   }

2. TSE

   An Interleaving Approach to Combinatorial Testing and Failure-Inducing Interaction Identification

   Xintao Niu, Changhai Nie, Hareton Leung, and 4 more authors

   IEEE Transactions on Software Engineering, Feb 2020

   Abs Bib HTML

   Combinatorial testing (CT) seeks to detect potential faults caused by various interactions of factors that can influence the software systems. When applying CT, it is a common practice to first generate a set of test cases to cover each possible interaction and then to identify the failure-inducing interaction after a failure is detected. Although this conventional procedure is simple and forthright, we conjecture that it is not the ideal choice in practice. This is because 1) testers desire to identify the root cause of failures before all the needed test cases are generated and executed 2) the early identified failure-inducing interactions can guide the remaining test case generation so that many unnecessary and invalid test cases can be avoided. For these reasons, we propose a novel CT framework that allows both generation and identification process to interact with each other. As a result, both generation and identification stages will be done more effectively and efficiently. We conducted a series of empirical studies on several open-source software, the results of which show that our framework can identify the failure-inducing interactions more quickly than traditional approaches while requiring fewer test cases.

   @article{niu2020interleaving,
     author = {Niu, Xintao and Nie, Changhai and Leung, Hareton and Lei, Yu and Wang, Xiaoyin and Xu, Jiaxi and Wang, Yan},
     journal = {IEEE Transactions on Software Engineering},
     title = {An Interleaving Approach to Combinatorial Testing and Failure-Inducing Interaction Identification},
     year = {2020},
     volume = {46},
     number = {6},
     pages = {584-615},
     doi = {10.1109/TSE.2018.2865772},
   }

2018

1. 软件学报

   覆盖表生成的禁忌搜索算法

   王燕,  聂长海,  钮鑫涛,  吴化尧, and  徐家喜

   Journal of Software, Feb 2018

   Bib

   @article{王燕2018覆盖表生成的禁忌搜索算法,
     title = {覆盖表生成的禁忌搜索算法},
     author = {王燕 and 聂长海 and 钮鑫涛 and 吴化尧 and 徐家喜},
     journal = {Journal of Software},
     volume = {29},
     number = {12},
     pages = {3665--3691},
     year = {2018},
   }

2015

1. IST

   Combinatorial testing, random testing, and adaptive random testing for detecting interaction triggered failures

   Changhai Nie, Huayao Wu, Xintao Niu, and 3 more authors

   Information and Software Technology, Feb 2015

   Bib HTML

   @article{NIE2015198,
     title = {Combinatorial testing, random testing, and adaptive random testing for detecting interaction triggered failures},
     journal = {Information and Software Technology},
     volume = {62},
     pages = {198 - 213},
     year = {2015},
     issn = {0950-5849},
     doi = {10.1016/j.infsof.2015.02.008},
     author = {Nie, Changhai and Wu, Huayao and Niu, Xintao and Kuo, Fei-Ching and Leung, Hareton and Colbourn, Charles J.},
   }

2014

1. 计算机学报

   组合测试故障定位的关系树模型

   钮鑫涛,  聂长海, and Chan Alvin

   计算机学报, Jul 2014

   Bib

   @article{钮鑫涛 2014 组合测试故障定位的关系树模型,
     author = {钮鑫涛 and 聂长海 and Alvin, Chan},
     title = {组合测试故障定位的关系树模型},
     journal = {计算机学报},
     year = {2014},
     volume = {37},
     number = {12},
     pages = {2505-2518},
     month = jul,
   }

2013

1. IWCT

   Identifying Failure-Inducing Combinations Using Tuple Relationship

   Xintao Niu, Changhai Nie, Yu Lei, and 1 more author

   In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, Mar 2013

   Abs Bib HTML

   Combinatorial testing (CT) aims at detecting interaction failures between parameters in a system. Identifying the failure-inducing combinations of a failing test configuration can help developers find the cause of this failure. However, most studies in CT focus on detecting the failures rather than identifying failure-inducing combinations. In this paper, we propose the notion of a tuple relationship tree (TRT) to describe the relationships among all the candidate parameter interactions. TRT reduces additional test configurations that need to be generated in the fault localization process, and it also provides a clear view of all possible candidate interactions. As a result, our approach will not omit any possible interaction that could be the cause of a failure. In particular, we can identify multiple failure-inducing combinations that overlap with each other. Moreover, we extend our approach to handle the case where additional failure-inducing combinations may be introduced by newly generated test configurations.

   @inproceedings{niu2013ttr,
     author = {Niu, Xintao and Nie, Changhai and Lei, Yu and Chan, Alvin},
     booktitle = {2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops},
     title = {Identifying Failure-Inducing Combinations Using Tuple Relationship},
     year = {2013},
     volume = {},
     number = {},
     pages = {271-280},
     keywords = {program testing;software fault tolerance;trees (mathematics);failure-inducing combination identification;combinatorial testing;CT;failing test configuration;TRT;tuple relationship tree;parameter interaction;fault localization process;test configuration;software system behavior;Testing;Pediatrics;Indexes;Fault diagnosis;Software;Arrays;Educational institutions;Combinatorial testing;Fault localization;Tuple relationship tree;overlapping combinations;New import combinations},
     doi = {10.1109/ICSTW.2013.38},
     issn = {null},
     month = mar,
   }