Computer Engineering / Bilgisayar Mühendisliği
Permanent URI for this collectionhttps://hdl.handle.net/11147/10
Browse
14 results
Search Results
Article Spectral Test Generation for Boolean Expressions(World Scientific Publishing, 2023) Ayav, TolgaThis paper presents a novel method for testing Boolean expressions. It is based on spectral, aka Fourier analysis of Boolean functions which is exploited to generate test inputs. The approach has three important contributions: (i) It generates a relatively small test suite with a high capability of fault detection, (ii) The test suite is prioritized such that expected fault detection time is shorter, (iii) It is entirely mathematical relying on a simple and straightforward formula. The proposed method is formulated and evaluations are performed on both synthetic and real expressions. It is also compared with two common test generation criteria, MC/DC and Minimal MUMCUT. Evaluations show that the test suite generated by the spectral approach is relatively small while expressing the capability of a better and quicker fault detection. The approach presented in this paper provides a useful insight into how spectral/Fourier analysis of Boolean functions can be exploited in software testing.Article Citation - WoS: 2Citation - Scopus: 3Mutation-Based Minimal Test Suite Generation for Boolean Expressions(World Scientific Publishing, 2023) Ayav, Tolga; Belli, FevziBoolean expressions are highly involved in control flows of programs and software specifications. Coverage criteria for Boolean expressions aim at producing minimal test suites to detect software faults. There exist various testing criteria, efficiency of which is usually evaluated through mutation analysis. This paper proposes an integer programming-based minimal test suite generation technique relying on mutation analysis. The proposed technique also takes into account the cost of fault detection. The technique is optimal such that the resulting test suite guarantees to detect all the mutants under given fault assumptions, while maximizing the average percentage of fault detection of a test suite. Therefore, the approach presented can also be considered as a reference method to check the efficiency of any common technique. The method is evaluated using four well-known real benchmark sets of Boolean expressions and is also exemplary compared with MCDC criterion. The results show that the test suites generated by the proposed method provide better fault coverage values and faster fault detection.Article Citation - WoS: 3Citation - Scopus: 4Application of the Law of Minimum and Dissimilarity Analysis To Regression Test Case Prioritization(IEEE, 2023) Ufuktepe, Ekincan; Tuğlular, TuğkanRegression testing is one of the most expensive processes in testing. Prioritizing test cases in regression testing is critical for the goal of detecting the faults sooner within a large set of test cases. We propose a test case prioritization (TCP) technique for regression testing called LoM-Score inspired by the Law of Minimum (LoM) from biology. This technique calculates the impact probabilities of methods calculated by change impact analysis with forward slicing and orders test cases according to LoM. However, this ordering doesn't consider the possibility that consecutive test cases may be covering the same methods repeatedly. Thereby, such ordering can delay the time of revealing faults that exist in other methods. To solve this problem, we enhance the LoM-Score TCP technique with an adaptive approach, namely with a dissimilarity-based coordinate analysis approach. The dissimilarity-based coordinate analysis uses Jaccard Similarity for calculating the similarity coefficients between test cases in terms of covered methods and the enhanced technique called Dissimilarity-LoM-Score (Dis-LoM-Score) applies a penalty with respective on the ordered test cases. We performed our case study on 10 open-source Java projects from Defects4J, which is a dataset of real bugs and an infrastructure for controlled experiments provided for software engineering researchers. Then, we hand-seeded multiple mutants generated by Major, which is a mutation testing tool. Then we compared our TCP techniques LoM-Score and Dis-LoM-Score with the four traditional TCP techniques based on their Average Percentage of Faults Detected (APFD) results.Article Citation - WoS: 2Citation - Scopus: 6Incremental Testing in Software Product Lines-An Event Based Approach(IEEE, 2023) Beyazıt, Mutlu; Tuğlular, Tuğkan; Öztürk Kaya, DilekOne way of developing fast, effective, and high-quality software products is to reuse previously developed software components and products. In the case of a product family, the software product line (SPL) approach can make reuse more effective. The goal of SPLs is faster development of low-cost and high-quality software products. This paper proposes an incremental model-based approach to test products in SPLs. The proposed approach utilizes event-based behavioral models of the SPL features. It reuses existing event-based feature models and event-based product models along with their test cases to generate test cases for each new product developed by adding a new feature to an existing product. Newly introduced featured event sequence graphs (FESGs) are used for behavioral feature and product modeling; thus, generated test cases are event sequences. The paper presents evaluations with three software product lines to validate the approach and analyze its characteristics by comparing it to the state-of-the-art ESG-based testing approach. Results show that the proposed incremental testing approach highly reuses the existing test sets as intended. Also, it is superior to the state-of-the-art approach in terms of fault detection effectiveness and test generation effort but inferior in terms of test set size and test execution effort.Conference Object Sonlu Durum Makinelerinin Fourier Analizi Tabanlı Sınanması(CEUR Workshop Proceedings, 2018) Takan, Savaş; Ayav, TolgaSonlu durum makinesi (FSM), devre ve yazılım testlerinde yaygın kullanıma sahip bir modelleme tekniğidir. FSM'lerin testi için literatürde çeşitli yöntemler bulunmakla birlikte, modellerin büyümesi sonucu test kümesinin büyüklüğü, hata yakalama oranı ve test üretim süresi gibi konularda yüksek başarıma sahip alternatif test yöntemlerine ihtiyaç bulunmaktadır. Bu çalışma ikili fonksiyonların Fourier analizine dayanan yeni bir test oluşturma yöntemi önermektedir. İlk sonuçlar, fonksiyonun frekans bileşenlerinden yararlanarak oluşturulan test takımının daha yüksek bir performansa sahip olduğunu göstermektedir. Önerilen yöntem, karakteristiği, maliyeti ve hata yakalama oranı üzerinden literatürden seçilen iki yöntemle karşılaştırılmıştır.Conference Object Özellik Yönelimli Ürün Konfigürasyonlarının Olay Sıra Çizgeleri ile Doğrulanması(CEUR Workshop Proceedings, 2018) Tuğlular, Tuğkan; Belli, Fevzi; Öztürk, DilekThis study attempts to suggest an approach to systematically test potentially very large number of product variants in feature-oriented software. Feature-oriented software forms a popular concept to efficiently realize software reuse. Developing feature-oriented software is well accepted to accomplish software reuse in an efficient way. Developing product variants by exploiting software reuse requires verification of these variants by exploiting test reuse. However, the reuse of tests in the verification of variants is an underworked topic. In this study, we propose a model-based approach to top-down testing of feature-oriented software that does not have dependency or conflict between features. In the case study, event sequence graphs (ESGs) are used to model the software under consideration and then to generate test cases for positive and negative testing. The generated tests are executed via SahiPro web test automation tool, of which scripts are also automatically generated from ESGs.Article Citation - WoS: 7Citation - Scopus: 11Exploiting Model Morphology for Event-Based Testing(Institute of Electrical and Electronics Engineers Inc., 2015) Belli, Fevzi; Beyazıt, MutluModel-based testing employs models for testing. Model-based mutation testing (MBMT) additionally involves fault models, called mutants, by applying mutation operators to the original model. A problem encountered with MBMT is the elimination of equivalent mutants and multiple mutants modeling the same faults. Another problem is the need to compare a mutant to the original model for test generation. This paper proposes an event-based approach to MBMT that is not fixed on single events and a single model but rather operates on sequences of events of length k ≥ 1 and invokes a sequence of models that are derived from the original one by varying its morphology based on k. The approach employs formal grammars, related mutation operators, and algorithms to generate test cases, enabling the following: (1) the exclusion of equivalent mutants and multiple mutants; (2) the generation of a test case in linear time to kill a selected mutant without comparing it to the original model; (3) the analysis of morphologically different models enabling the systematic generation of mutants, thereby extending the set of fault models studied in related literature. Three case studies validate the approach and analyze its characteristics in comparison to random testing and another MBMT approach.Article Citation - WoS: 9Citation - Scopus: 10Fault Domain-Based Testing in Imperfect Situations: a Heuristic Approach and Case Studies(Springer Verlag, 2015) Belli, Fevzi; Beyazıt, Mutlu; Endo, Andre Takeshi; Mathur, Aditya; Simao, AdenilsoModel-based testing (MBT) involves creating an abstraction, called a model, to represent the system and automatically deriving test cases from this model. MBT can be performed using various approaches that generally employ certain assumptions or requirements affecting the test performance in practice. Here, we consider the harmonized state identifiers (HSI) method, which is based on finite state machine (FSM) models and generates test sets that cover all faults in a given domain under certain conditions. We are interested in the application of the HSI method in practical scenarios where some conditions do not hold or are not straightforward to satisfy. Thus, we propose a heuristic extension to the HSI method, called heuristic HSI (HHSI), to consider imperfect situations as they often occur in practice. To analyze the characteristics of HHSI, we empirically compare it to random testing and coverage-based testing using non-trivial case studies. The experiments include model-based mutation analyses over several FSM models.Article Citation - Scopus: 1Assuring Dependability of Software Reuse: an Industrial Standard(Springer Verlag, 2014) Belli, FevziWhereas a software component may be perfectly suited to one application, it may prove to cause severe faults in other applications. The prestandard IEC/PAS 62814 (Dependability of Software Products Containing Reusable Components – Guidance for Functionality and Tests), which has recently been released, addresses the functionality, testing, and dependability of software components to be reused and products that contain software to be used in more than one application; that is, reused by the same or by another development organization, regardless of whether it belongs to the same or another legal entity than the one that has developed this software. This paper introduces into this pre-standard and give hints how to use it. The author, who chaired its realization that started in 2006, briefly summarizes the difficult process to bring the industrial partners with controversial interests to a consensus.Article Citation - WoS: 53Citation - Scopus: 76Model-Based Mutation Testing-Approach and Case Studies(Elsevier Ltd., 2016) Belli, Fevzi; Budnik, Christof J.; Hollmann, Axel; Tuğlular, Tuğkan; Wong, W. EricThis paper rigorously introduces the concept of model-based mutation testing (MBMT) and positions it in the landscape of mutation testing. Two elementary mutation operators, insertion and omission, are exemplarily applied to a hierarchy of graph-based models of increasing expressive power including directed graphs, event sequence graphs, finite-state machines and statecharts. Test cases generated based on the mutated models (mutants) are used to determine not only whether each mutant can be killed but also whether there are any faults in the corresponding system under consideration (SUC) developed based on the original model. Novelties of our approach are: (1) evaluation of the fault detection capability (in terms of revealing faults in the SUC) of test sets generated based on the mutated models, and (2) superseding of the great variety of existing mutation operators by iterations and combinations of the two proposed elementary operators. Three case studies were conducted on industrial and commercial real-life systems to demonstrate the feasibility of using the proposed MBMT approach in detecting faults in SUC, and to analyze its characteristic features. Our experimental data suggest that test sets generated based on the mutated models created by insertion operators are more effective in revealing faults in SUC than those generated by omission operators. Worth noting is that test sets following the MBMT approach were able to detect faults in the systems that were tested by manufacturers and independent testing organizations before they were released. © 2016 Elsevier B.V.