Computer Engineering / Bilgisayar Mühendisliği

Permanent URI for this collectionhttps://hdl.handle.net/11147/10

Browse

Filters

2020 - 20216

Settings

Start date: 2020Institution Author: search.filters.institutionAuthor.Belli, Fevzi

Search Results

Now showing 1 - 6 of 6
  • Article
    Citation - Scopus: 1
    Model-Based Ideal Testing of Hardware Description Language (hdl) Programs
    (Springer, 2021) Kılınççeker, Onur; Türk, Ercüment; Belli, Fevzi; Challenger, Moharram
    An ideal test is supposed to show not only the presence of bugs but also their absence. Based on the Fundamental Test Theory of Goodenough and Gerhart (IEEE Trans Softw Eng SE-1(2):156–173, 1975), this paper proposes an approach to model-based ideal testing of hardware description language (HDL) programs based on their behavioral model. Test sequences are generated from both original (fault-free) and mutant (faulty) models in the sense of positive and negative testing, forming a holistic test view. These test sequences are then executed on original (fault-free) and mutant (faulty) HDL programs, in the sense of mutation testing. Using the techniques known from automata theory, test selection criteria are developed and formally show that they fulfill the major requirements of Fundamental Test Theory, that is, reliability and validity. The current paper comprises a preparation step (consisting of the sub-steps model construction, model mutation, model conversion, and test generation) and a composition step (consisting of the sub-steps pre-selection and construction of Ideal test suites). All the steps are supported by a toolchain that is already implemented and is available online. To critically validate the proposed approach, three case studies (a sequence detector, a traffic light controller, and a RISC-V processor) are used and the strengths and weaknesses of the approach are discussed. The proposed approach achieves the highest mutation score in positive and negative testing for all case studies in comparison with two existing methods (regular expression-based test generation and context-based random test generation), using four different techniques.
  • Article
    Citation - WoS: 3
    Citation - Scopus: 6
    Model-Based Ideal Testing of Gui Programs-Approach and Case Studies
    (IEEE-Inst Electrical Electronics Engineers inc, 2021) Kilincceker, Onur; Silistre, Alper; Belli, Fevzi; Challenger, Moharram
    Traditionally, software testing is aimed at showing the presence of faults. This paper proposes a novel approach to testing graphical user interfaces (GUI) for showing both the presence and absence of faults in the sense of ideal testing. The approach uses a positive testing concept to show that the GUI under consideration (GUC) does what the user expects; to the contrary, the negative testing concept shows that the GUC does not do anything that the user does not expect, building a holistic view. The first step of the approach models the GUC by a finite state machine (FSM) that enables the model-based generation of test cases. This is always possible as the GUIs are considered as strictly sequential processes. The next step converts the FSM to an equivalent regular expression (RE) that will be analyzed first to construct test selection criteria for excluding redundant test cases and construct test coverage criteria for terminating the positive test process. Both criteria enable us to assess the adequacy and efficiency of the positive tests performed. The negative tests will be realized by systematically mutating the FSM to model faults, the absence of which are to be shown. Those mutant FSMs will be handled and assessed in the same way as in positive testing. Two case studies illustrate and validate the approach; the experiments' results will be analyzed to discuss the pros and cons of the techniques introduced.
  • Article
    Citation - WoS: 9
    Citation - Scopus: 11
    Test Input Generation From Cause-Effect Graphs
    (Springer, 2021) Kavzak Ufuktepe, Deniz; Ayav, Tolga; Belli, Fevzi
    Cause-effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause-effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers' method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause-effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause-effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause-effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers' technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause-effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause-effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause-effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.
  • Conference Object
    Citation - Scopus: 5
    Community Detection in Model-Based Testing To Address Scalability: Study Design
    (Institute of Electrical and Electronics Engineers, 2020) Silistre, Alper; Kılınççeker, Onur; Belli, Fevzi; Challenger, Moharram; Kardaş, Geylani
    Model-based GUI testing has achieved widespread recognition in academy thanks to its advantages compared to code-based testing due to its potentials to automate testing and the ability to cover bigger parts more efficiently. In this study design paper, we address the scalability part of the model-based GUI testing by using community detection algorithms. A case study is presented as an example of possible improvements to make a model-based testing approach more efficient. We demonstrate layered ESG models as an example of our approach to consider the scalability problem. We present rough calculations with expected results, which show 9 times smaller time and space units for 100 events in the ESG model when a community detection algorithm is applied. © 2020 Polish Information Processing Society - as it is since 2011.
  • Conference Object
    Citation - WoS: 7
    Citation - Scopus: 8
    Models in Graphical User Interface Testing: Study Design
    (Institute of Electrical and Electronics Engineers, 2020) Silistre, Alper; Kılınççeker, Onur; Belli, Fevzi; Challenger, Moharram; Kardaş, Geylani
    Model-based GUI testing is an important concept in Software GUI testing. Manual testing is a time-consuming labor and heavily error-prone. It has several well-accepted models that Software Testing community has been working and contributing to them for many years. This paper reviews different models used in model-based GUI testing and presents a case study with a proposed approach for how to convert several well-accepted models to ESG (Event Sequence Graphs) to generate test cases and execute them with an aim to consolidate past and future works in a single model. © 2020 IEEE.
  • Conference Object
    Citation - WoS: 1
    Citation - Scopus: 1
    Mutation Operators for Decision Table-Based Contracts Used in Software Testing
    (Institute of Electrical and Electronics Engineers, 2020) Khalilov, Abbas; Tuğlular, Tuğkan; Belli, Fevzi
    The Design by Contract technique allows developers to improve source code with contracts, and testing using contracts helps to identify faults. However, the source code of the program under test is not always available. With black-box testing, it is possible to generate contracts from specifications of the software. In this paper, we apply mutation analysis on a model of a given specifications, where mutants are initially gained by applying proposed in this paper certain mutation operators on corresponding model, and then mutated specifications are examined. © 2020 IEEE.