Computer Engineering / Bilgisayar Mühendisliği

Permanent URI for this collectionhttps://hdl.handle.net/11147/10

Browse

Filters

2009 - 200942010 - 2019212020 - 202311

Settings

Author: search.filters.author.Belli, Fevzi

Search Results

Now showing 1 - 10 of 36
  • Article
    Citation - Scopus: 5
    Unifying Behavioral and Feature Modeling for Testing of Software Product Lines
    (World Scientific Publishing, 2023) Belli, Fevzi; Tuğlular, Tuğkan; Ufuktepe, Ekincan
    Existing software product line (SPL) engineering testing approaches generally provide positive testing that validates the SPL's functionality. Negative testing is commonly neglected. This research aims to unify behavioral and feature models of an SPL, enable testing before and after variability binding for domain-centric and product-centric testing, and combine positive and negative testing for a holistic testing view. This study suggests behavioral modeling with event sequence graphs (ESGs). This heterogeneous modeling strategy supports bottom-up domain testing and top-down product testing with the feature model. This new feature-oriented ESG test creation method generates shorter test sequences than the original ESG optimum test sequences. Statechart and original ESG test-generating methods are compared. Positive testing findings are similar. The Statechart technique generated 12 test cases with 59 events, whereas the ESG technique created six test cases with 60 events. The ESG technique generated 205 negative test cases with 858 events with the Test Suite Designer tool. However, the Conformiq Designer tool for the Statechart technique does not have a negative test case generation capability. It is shown that the proposed ESG-based holistic approach confirms not only the desirable (positive) properties but also the undesirable (negative) ones. As an additional research, the traditional ESG test-generating approach is compared to the new feature-oriented method on six SPLs of different sizes and features. Our case study results show that the traditional ESG test generation approach demonstrated higher positive test generation scores compare to the proposed feature-oriented test generation approach. However, our proposed feature-oriented test generation approach is capable of generating shorter test sequences, which could be beneficial for reducing the execution time of test cases compared to traditional ESG approach. Finally, our case study has also shown that regardless of the test generation approach, there has been found no significant difference between the Bottom-up and Top-down test strategies with respect to their positive test generation scores. © World Scientific Publishing Company.
  • Article
    Link Prediction for Completing Graphical Software Models Using Neural Networks
    (IEEE, 2023) Leblebici, Onur; Tuğlular, Tuğkan; Belli, Fevzi
    Deficiencies and inconsistencies introduced during the modeling of software systems may result in high costs and negatively impact the quality of all developments performed using these models. Therefore, developing more accurate models will aid software architects in developing software systems that match and exceed expectations. This paper proposes a graph neural network (GNN) method for predicting missing connections, or links, in graphical models, which are widely employed in modeling software systems. The proposed method utilizes graphs as allegedly incomplete, primitive graphical models of the system under consideration (SUC) as input and proposes links between its elements through the following steps: (i) transform the models into graph-structured data and extract features from the nodes, (ii) train the GNN model, and (iii) evaluate the performance of the trained model. Two GNN models based on SEAL and DeepLinker are evaluated using three performance metrics, namely cross-entropy loss, area under curve, and accuracy. Event sequence graphs (ESGs) are used as an example of applying the approach to an event-based behavioral modeling technique. Examining the results of experiments conducted on various datasets and variations of GNN reveals that missing connections between events in an ESG can be predicted even with relatively small datasets generated from ESG models. Author
  • Article
    Citation - WoS: 2
    Citation - Scopus: 3
    Mutation-Based Minimal Test Suite Generation for Boolean Expressions
    (World Scientific Publishing, 2023) Ayav, Tolga; Belli, Fevzi
    Boolean expressions are highly involved in control flows of programs and software specifications. Coverage criteria for Boolean expressions aim at producing minimal test suites to detect software faults. There exist various testing criteria, efficiency of which is usually evaluated through mutation analysis. This paper proposes an integer programming-based minimal test suite generation technique relying on mutation analysis. The proposed technique also takes into account the cost of fault detection. The technique is optimal such that the resulting test suite guarantees to detect all the mutants under given fault assumptions, while maximizing the average percentage of fault detection of a test suite. Therefore, the approach presented can also be considered as a reference method to check the efficiency of any common technique. The method is evaluated using four well-known real benchmark sets of Boolean expressions and is also exemplary compared with MCDC criterion. The results show that the test suites generated by the proposed method provide better fault coverage values and faster fault detection.
  • Conference Object
    Citation - WoS: 3
    Citation - Scopus: 2
    Heterogeneous Modeling and Testing of Software Product Lines
    (IEEE, 2021) Belli, Fevzi; Tuğlular, Tuğkan; Ufuktepe, Ekincan
    Software product line (SPL) engineering is a widely accepted approach to systematically realizing software reuse in an industrial environment. Feature models, a centerpiece of most SPL engineering techniques, are appropriate to model the variability and the structure of SPLs, but not their behavior. This paper uses the idea to link feature modeling to model-based behavior modeling and to determine the test direction (top-down or bottom-up) based on the variability binding. This heterogeneous modeling enables a holistic system testing for validating both desirable (positive) and undesirable (negative) properties of the SPL and variants. The proposed approach is validated by a non-trivial example and evaluated by comparison.
  • Conference Object
    Citation - Scopus: 1
    Mutation Analysis of Specification-Based Contracts in Software Testing [conference Object]
    (IEEE, 2021) Khalilov, Abbas; Tuğlular, Tuğkan; Belli, Fevzi
    This work focuses on checking the adequacy of the test cases generated using Decision-Table-augmented Event Sequence Graphs (ESG-DTs), which represents the specification of a system under test, by using mutation analysis. Test cases are represented in the Complete Event Sequence (CES) and Faulty CES (FCES) forms. We present a new set of mutation operators for mutation of contracts represented in Multi-Terminal Binary Decision Diagram (MTBDD) and introduce a new approach to mutation of the ESG-DT model by using the proposed mutation operators. The approach is evaluated on three cases. The results show the drawback of specific FCES test sequences and the relationship between the mutant detection by CES/FCES sequences and proposed mutation operators.
  • Article
    Citation - Scopus: 1
    Model-Based Ideal Testing of Hardware Description Language (hdl) Programs
    (Springer, 2021) Kılınççeker, Onur; Türk, Ercüment; Belli, Fevzi; Challenger, Moharram
    An ideal test is supposed to show not only the presence of bugs but also their absence. Based on the Fundamental Test Theory of Goodenough and Gerhart (IEEE Trans Softw Eng SE-1(2):156–173, 1975), this paper proposes an approach to model-based ideal testing of hardware description language (HDL) programs based on their behavioral model. Test sequences are generated from both original (fault-free) and mutant (faulty) models in the sense of positive and negative testing, forming a holistic test view. These test sequences are then executed on original (fault-free) and mutant (faulty) HDL programs, in the sense of mutation testing. Using the techniques known from automata theory, test selection criteria are developed and formally show that they fulfill the major requirements of Fundamental Test Theory, that is, reliability and validity. The current paper comprises a preparation step (consisting of the sub-steps model construction, model mutation, model conversion, and test generation) and a composition step (consisting of the sub-steps pre-selection and construction of Ideal test suites). All the steps are supported by a toolchain that is already implemented and is available online. To critically validate the proposed approach, three case studies (a sequence detector, a traffic light controller, and a RISC-V processor) are used and the strengths and weaknesses of the approach are discussed. The proposed approach achieves the highest mutation score in positive and negative testing for all case studies in comparison with two existing methods (regular expression-based test generation and context-based random test generation), using four different techniques.
  • Article
    Citation - WoS: 3
    Citation - Scopus: 6
    Model-Based Ideal Testing of Gui Programs-Approach and Case Studies
    (IEEE-Inst Electrical Electronics Engineers inc, 2021) Kilincceker, Onur; Silistre, Alper; Belli, Fevzi; Challenger, Moharram
    Traditionally, software testing is aimed at showing the presence of faults. This paper proposes a novel approach to testing graphical user interfaces (GUI) for showing both the presence and absence of faults in the sense of ideal testing. The approach uses a positive testing concept to show that the GUI under consideration (GUC) does what the user expects; to the contrary, the negative testing concept shows that the GUC does not do anything that the user does not expect, building a holistic view. The first step of the approach models the GUC by a finite state machine (FSM) that enables the model-based generation of test cases. This is always possible as the GUIs are considered as strictly sequential processes. The next step converts the FSM to an equivalent regular expression (RE) that will be analyzed first to construct test selection criteria for excluding redundant test cases and construct test coverage criteria for terminating the positive test process. Both criteria enable us to assess the adequacy and efficiency of the positive tests performed. The negative tests will be realized by systematically mutating the FSM to model faults, the absence of which are to be shown. Those mutant FSMs will be handled and assessed in the same way as in positive testing. Two case studies illustrate and validate the approach; the experiments' results will be analyzed to discuss the pros and cons of the techniques introduced.
  • Article
    Citation - WoS: 9
    Citation - Scopus: 11
    Test Input Generation From Cause-Effect Graphs
    (Springer, 2021) Kavzak Ufuktepe, Deniz; Ayav, Tolga; Belli, Fevzi
    Cause-effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause-effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers' method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause-effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause-effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause-effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers' technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause-effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause-effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause-effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.
  • Conference Object
    Citation - WoS: 10
    Citation - Scopus: 12
    Random Test Generation From Regular Expressions for Graphical User Interface (gui) Testing
    (Institute of Electrical and Electronics Engineers, 2019) Kılınççeker, Onur; Silistre, Alper; Challenger, Moharram; Belli, Fevzi
    Generation of test sequences, that is, (user) inputs - expected (system) outputs, is an important task of testing of graphical user interfaces (GUI). This work proposes an approach to randomly generate test sequences that might he used for comparison with existing GUI testing techniques to evaluate their efficiency. The proposed approach first models CUI under test by a finite state machine (FSM) and then converts it to a regular expression (RE). A tool based on a special technique we developed analyzes the RE to fulfill missing context information such as the position of a symbol in the RE. The result is a context table representing the RE. The proposed approach traverses the context table to generate the test sequences. To do this, the approach repeatedly selects a symbol in the table, starting from the initial symbol, in a random manner until reaching a special, finalizing symbol for constructing a test sequence. Thus, the approach uses a symbol coverage criterion to assess the adequacy of the test generation. To evaluate the approach, mutation testing is used. The proposed technique is to a great extent implemented and is available as a tool called PQ-Ran Test (PQ-analysis based Random Test Generation). A case study demonstrates the proposed approach and analyzes its effectiveness by mutation testing.
  • Conference Object
    Citation - Scopus: 5
    Community Detection in Model-Based Testing To Address Scalability: Study Design
    (Institute of Electrical and Electronics Engineers, 2020) Silistre, Alper; Kılınççeker, Onur; Belli, Fevzi; Challenger, Moharram; Kardaş, Geylani
    Model-based GUI testing has achieved widespread recognition in academy thanks to its advantages compared to code-based testing due to its potentials to automate testing and the ability to cover bigger parts more efficiently. In this study design paper, we address the scalability part of the model-based GUI testing by using community detection algorithms. A case study is presented as an example of possible improvements to make a model-based testing approach more efficient. We demonstrate layered ESG models as an example of our approach to consider the scalability problem. We present rough calculations with expected results, which show 9 times smaller time and space units for 100 events in the ESG model when a community detection algorithm is applied. © 2020 Polish Information Processing Society - as it is since 2011.