Computer Engineering / Bilgisayar Mühendisliği
Permanent URI for this collectionhttps://hdl.handle.net/11147/10
Browse
3 results
Search Results
Article Citation - WoS: 9Citation - Scopus: 11Test Input Generation From Cause-Effect Graphs(Springer, 2021) Kavzak Ufuktepe, Deniz; Ayav, Tolga; Belli, FevziCause-effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause-effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers' method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause-effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause-effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause-effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers' technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause-effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause-effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause-effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.Article Citation - WoS: 9Citation - Scopus: 10Fault Domain-Based Testing in Imperfect Situations: a Heuristic Approach and Case Studies(Springer Verlag, 2015) Belli, Fevzi; Beyazıt, Mutlu; Endo, Andre Takeshi; Mathur, Aditya; Simao, AdenilsoModel-based testing (MBT) involves creating an abstraction, called a model, to represent the system and automatically deriving test cases from this model. MBT can be performed using various approaches that generally employ certain assumptions or requirements affecting the test performance in practice. Here, we consider the harmonized state identifiers (HSI) method, which is based on finite state machine (FSM) models and generates test sets that cover all faults in a given domain under certain conditions. We are interested in the application of the HSI method in practical scenarios where some conditions do not hold or are not straightforward to satisfy. Thus, we propose a heuristic extension to the HSI method, called heuristic HSI (HHSI), to consider imperfect situations as they often occur in practice. To analyze the characteristics of HHSI, we empirically compare it to random testing and coverage-based testing using non-trivial case studies. The experiments include model-based mutation analyses over several FSM models.Article Citation - WoS: 53Citation - Scopus: 76Model-Based Mutation Testing-Approach and Case Studies(Elsevier Ltd., 2016) Belli, Fevzi; Budnik, Christof J.; Hollmann, Axel; Tuğlular, Tuğkan; Wong, W. EricThis paper rigorously introduces the concept of model-based mutation testing (MBMT) and positions it in the landscape of mutation testing. Two elementary mutation operators, insertion and omission, are exemplarily applied to a hierarchy of graph-based models of increasing expressive power including directed graphs, event sequence graphs, finite-state machines and statecharts. Test cases generated based on the mutated models (mutants) are used to determine not only whether each mutant can be killed but also whether there are any faults in the corresponding system under consideration (SUC) developed based on the original model. Novelties of our approach are: (1) evaluation of the fault detection capability (in terms of revealing faults in the SUC) of test sets generated based on the mutated models, and (2) superseding of the great variety of existing mutation operators by iterations and combinations of the two proposed elementary operators. Three case studies were conducted on industrial and commercial real-life systems to demonstrate the feasibility of using the proposed MBMT approach in detecting faults in SUC, and to analyze its characteristic features. Our experimental data suggest that test sets generated based on the mutated models created by insertion operators are more effective in revealing faults in SUC than those generated by omission operators. Worth noting is that test sets following the MBMT approach were able to detect faults in the systems that were tested by manufacturers and independent testing organizations before they were released. © 2016 Elsevier B.V.