Browsing by Author "Belli, Fevzi"

Now showing 1 - 16 of 16

Citation Count: 4
Advances in Model-Based Testing of Graphical User Interfaces
(Elsevier Academic Press inc, 2017) Belli, Fevzi; Beyazit, Mutlu; Budnik, Christof J.; Tuglular, Tugkan; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
Graphical user interfaces (GUIs) enable comfortable interactions of the computer-based systems with their environment. Large systems usually require complex GUIs, which are commonly fault prone and thus are to be carefully designed, implemented, and tested. As a thorough testing is not feasible, techniques are favored to test relevant features of the system under test that will be specifically modeled. This chapter summarizes, reviews, and exemplifies conventional and novel techniques for model-based GUI testing.
Application of graph neural networks on software modeling
(01. Izmir Institute of Technology, 2020-12) Leblebici, Onur Yusuf; Tuğlular, Tuğkan; Tuğlular, Tuğkan; Belli, Fevzi; 01. Izmir Institute of Technology
Deficiencies and inconsistencies introduced during the modeling of software systems can cause undesirable consequences that may result in high costs and negatively affect the quality of all developments made using these models. Therefore, creating better models will help the software engineers to build better software systems that meet expectations. One of the software modelling methods used for analysis of graphical user interfaces is Event Sequence Graphs (ESG). The goal of this thesis is to propose a method that predicts missing or forgotten links between events defined in an ESG via Graph Neural Networks (GNN). A five-step process consisting of the following steps is proposed: (i) data collection from ESG model, (ii) dataset transformation, (iii) GNN model training, (iv) validation of trained model and (v) testing the model on unseen data. Three performance metrics, namely cross entropy loss, area under curve and accuracy, were used to measure the performance of the GNN models. Examining the results of the experiments performed on different datasets and different variations of GNN, shows that even with relatively small datasets prepared from ESG models, predicts missing or forgotten links between events defined in an ESG can be achieved.
Citation Count: 3
Boolean Differentiation for Formalizing Myers' Cause-Effect Graph Testing Technique
(Ieee, 2015) Ayav, Tolga; Belli, Fevzi; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü
Cause-Effect Graph Testing is a popular technique used for almost four decades. Based on Boolean algebra, this technique assists deriving test cases from a given specification informally written in a natural language. The present paper suggests Boolean differentiation for formalizing this technique. The new approach is applied to an example, borrowed from G. Myers, for demonstrating and analyzing its features. Evaluations show that the new approach outperforms Myers' approach in terms of the detected faults per test cases.
Citation Count: 1
Heterogeneous Modeling and Testing of Software Product Lines
(Ieee Computer Soc, 2021) Belli, Fevzi; Tuglular, Tugkan; Ufuktepe, Ekincan; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
Software product line (SPL) engineering is a widely accepted approach to systematically realizing software reuse in an industrial environment. Feature models, a centerpiece of most SPL engineering techniques, are appropriate to model the variability and the structure of SPLs, but not their behavior. This paper uses the idea to link feature modeling to model-based behavior modeling and to determine the test direction (top-down or bottom-up) based on the variability binding. This heterogeneous modeling enables a holistic system testing for validating both desirable (positive) and undesirable (negative) properties of the SPL and variants. The proposed approach is validated by a non-trivial example and evaluated by comparison.
Citation Count: 6
Input Contract Testing of Graphical User Interfaces
(World Scientific Publ Co Pte Ltd, 2016) Tuglular, Tugkan; Belli, Fevzi; Linschulte, Michael; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
User inputs are critical for the security, safety, and reliability of software systems. This paper proposes a new concept called user input contracts, which is an integral part of a design-by-contract supplemented development process, and a model-based testing approach to detect violations of user input contracts. The approach generates test cases from an input contract integrated with graph-based model of user interface specification and applies them to the system under consideration. The paper presents a proof-of-concept tool that has been developed and used to validate the approach by experiments. The experiments are conducted on a web-based system for marketing tourist services to analyze input robustness of system under consideration with respect to user input contracts.
Citation Count: 0
Model Based Testing of VHDL Programs
(Ieee, 2015) Ayav, Tolga; Tuglular, Tugkan; Belli, Fevzi; Tuğlular, Tuğkan; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü; Bilgisayar Mühendisliği Bölümü
VHDL programs are often validated by means of test benches constructed from formal system specification. To include real-time properties of VHDL programs, the proposed approach first transforms them to concurrently running network of timed automata and then performs model checking on properties taken from the specification. Counterexamples generated by the model checker are used to form a test bench. The approach is validated by a case study composed of a nontrivial application running on a microprocessor. As presented, the approach enables testing both hardware and software at once.
Citation Count: 0
Model Based Testing of VHDL Programs
(Ieee, 2015) Ayav, Tolga; Tuglular, Tugkan; Belli, Fevzi; Tuğlular, Tuğkan; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü
VHDL programs are often validated by means of test benches constructed from formal system specification. To include real-time properties of VHDL programs, the proposed approach first transforms them to concurrently running network of timed automata and then performs model checking on properties taken from the specification. Counterexamples generated by the model checker are used to form a test bench. The approach is validated by a case study composed of a nontrivial application running on a microprocessor. As presented, the approach enables testing both hardware and software at once.
Citation Count: 3
Model-Based Contract Testing of Graphical User Interfaces
(Ieice-inst Electronics information Communication Engineers, 2015) Tuglular, Tugkan; Muftuoglu, Arda; Belli, Fevzi; Linschulte, Michael; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
Graphical User Interfaces (GUIs) are critical for the security, safety and reliability of software systems. Injection attacks, for instance via SQL, succeed due to insufficient input validation and can be avoided if contract-based approaches, such as Design by Contract, are followed in the software development lifecycle of GUIs. This paper proposes a model-based testing approach for detecting GUI data contract violations, which may result in serious failures such as system crash. A contract-based model of GUI data specifications is used to develop test scenarios and to serve as test oracle. The technique introduced uses multi terminal binary decision diagrams, which are designed as an integral part of decision table-augmented event sequence graphs, to implement a GUI testing process. A case study, which validates the presented approach on a port scanner written in Java programming language, is presented.
Citation Count: 45
Model-based mutation testing-Approach and case studies
(Elsevier, 2016) Belli, Fevzi; Budnik, Christof J.; Hollmann, Axel; Tuglular, Tugkan; Wong, W. Eric; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
This paper rigorously introduces the concept of model-based mutation testing (MBMT) and positions it in the landscape of mutation testing. Two elementary mutation operators, insertion and omission, are exemplarily applied to a hierarchy of graph-based models of increasing expressive power including directed graphs, event sequence graphs, finite-state machines and statecharts. Test cases generated based on the mutated models (mutants) are used to determine not only whether each mutant can be killed but also whether there are any faults in the corresponding system under consideration (SUC) developed based on the original model. Novelties of our approach are: (1) evaluation of the fault detection capability (in terms of revealing faults in the SUC) of test sets generated based on the mutated models, and (2) superseding of the great variety of existing mutation operators by iterations and combinations of the two proposed elementary operators. Three case studies were conducted on industrial and commercial real-life systems to demonstrate the feasibility of using the proposed MBMT approach in detecting faults in SUC, and to analyze its characteristic features. Our experimental data suggest that test sets generated based on the mutated models created by insertion operators are more effective in revealing faults in SUC than those generated by omission operators. Worth noting is that test sets following the MBMT approach were able to detect faults in the systems that were tested by manufacturers and independent testing organizations before they were released. (C) 2016 Elsevier B.V. All rights reserved.
Citation Count: 0
Mutation Analysis of Specification-Based Contracts in Software Testing
(Ieee, 2021) Khalilov, Abbas; Tuglular, Tugkan; Belli, Fevzi; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
This work focuses on checking the adequacy of the test cases generated using Decision-Table-augmented Event Sequence Graphs (ESG-DTs), which represents the specification of a system under test, by using mutation analysis. Test cases are represented in the Complete Event Sequence (CES) and Faulty CES (FCES) forms. We present a new set of mutation operators for mutation of contracts represented in Multi-Terminal Binary Decision Diagram (MTBDD) and introduce a new approach to mutation of the ESG-DT model by using the proposed mutation operators. The approach is evaluated on three cases. The results show the drawback of specific FCES test sequences and the relationship between the mutant detection by CES/FCES sequences and proposed mutation operators.
Mutation analysis of specification-based contracts in software testing [Master Thesis]
(01. Izmir Institute of Technology, 2021-07) Khalilov, Abbas; Tuğlular, Tuğkan; Tuğlular, Tuğkan; Belli, Fevzi; 01. Izmir Institute of Technology
Software used in fields such as medicine, finance, aviation and aerospace, nuclear power etc. is required to be reliable. Any software failures in these fields may have catastrophic consequences such as human and financial losses, which may cause a great damage to the economy and to social well-being. Hence, before launching, software should be rigorously tested. Testing can uncover the conditions, which software cannot handle. Those conditions might be overlooked during development. So, software testing points to the faults in the software under development to be patched. The important element of software testing is the use of the adequate test cases. If the outcome of the test case is positive, that means testing did not reveal any fault, then this test case might be considered as inefficient and useless for the tested version of software. Therefore, it is important to check test cases on adequacy, which can be achieved by mutation analysis. This thesis focuses on checking the adequacy of the test cases for Decision-Table-augmented Event Sequence Graphs (ESG-DTs) representation of a system under test by using mutation analysis. Test cases are represented in the Complete Event Sequence (CES) and Faulty CES (FCES) forms. This thesis presents a new set of mutation operators for mutation of contracts represented in Multi-Terminal Binary Decision Diagram (MTBDD). This thesis introduces a new approach for mutation of the ESG-DT model by using the proposed MTBDD mutation operators. The proposed approach is evaluated on three cases. The results for all cases show the drawback of specific FCES test sequences and the relationship between the mutant detection by CES/FCES sequences and proposed mutation operators.
Citation Count: 1
Mutation Operators for Decision Table-Based Contracts Used in Software Testing
(Ieee, 2020) Khalilov, Abbas; Tuglular, Tugkan; Belli, Fevzi; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
The Design by Contract technique allows developers to improve source code with contracts, and testing using contracts helps to identify faults. However, the source code of the program under test is not always available. With black-box testing, it is possible to generate contracts from specifications of the software. In this paper, we apply mutation analysis on a model of a given specifications, where mutants are initially gained by applying proposed in this paper certain mutation operators on corresponding model, and then mutated specifications are examined.
Citation Count: 0
Mutation-Based Minimal Test Suite Generation for Boolean Expressions
(World Scientific Publ Co Pte Ltd, 2023) Ayav, Tolga; Belli, Fevzi; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü
Boolean expressions are highly involved in control flows of programs and software specifications. Coverage criteria for Boolean expressions aim at producing minimal test suites to detect software faults. There exist various testing criteria, efficiency of which is usually evaluated through mutation analysis. This paper proposes an integer programming-based minimal test suite generation technique relying on mutation analysis. The proposed technique also takes into account the cost of fault detection. The technique is optimal such that the resulting test suite guarantees to detect all the mutants under given fault assumptions, while maximizing the average percentage of fault detection of a test suite. Therefore, the approach presented can also be considered as a reference method to check the efficiency of any common technique. The method is evaluated using four well-known real benchmark sets of Boolean expressions and is also exemplary compared with MCDC criterion. The results show that the test suites generated by the proposed method provide better fault coverage values and faster fault detection.
Citation Count: 5
Test input generation from cause-effect graphs
(Springer, 2021) Ufuktepe, Deniz Kavzak; Ayav, Tolga; Belli, Fevzi; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü
Cause-effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause-effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers' method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause-effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause-effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause-effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers' technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause-effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause-effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause-effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.
Citation Count: 0
Unifying Behavioral and Feature Modeling for Testing of Software Product Lines
(World Scientific Publ Co Pte Ltd, 2023) Belli, Fevzi; Tuglular, Tugkan; Ufuktepe, Ekincan; Tuğlular, Tuğkan; Bilgisayar Mühendisliği Bölümü
Existing software product line (SPL) engineering testing approaches generally provide positive testing that validates the SPL's functionality. Negative testing is commonly neglected. This research aims to unify behavioral and feature models of an SPL, enable testing before and after variability binding for domain-centric and product-centric testing, and combine positive and negative testing for a holistic testing view. This study suggests behavioral modeling with event sequence graphs (ESGs). This heterogeneous modeling strategy supports bottom-up domain testing and top-down product testing with the feature model. This new feature-oriented ESG test creation method generates shorter test sequences than the original ESG optimum test sequences. Statechart and original ESG test-generating methods are compared. Positive testing findings are similar. The Statechart technique generated 12 test cases with 59 events, whereas the ESG technique created six test cases with 60 events. The ESG technique generated 205 negative test cases with 858 events with the Test Suite Designer tool. However, the Conformiq Designer tool for the Statechart technique does not have a negative test case generation capability. It is shown that the proposed ESG-based holistic approach confirms not only the desirable (positive) properties but also the undesirable (negative) ones. As an additional research, the traditional ESG test-generating approach is compared to the new feature-oriented method on six SPLs of different sizes and features. Our case study results show that the traditional ESG test generation approach demonstrated higher positive test generation scores compare to the proposed feature-oriented test generation approach. However, our proposed feature-oriented test generation approach is capable of generating shorter test sequences, which could be beneficial for reducing the execution time of test cases compared to traditional ESG approach. Finally, our case study has also shown that regardless of the test generation approach, there has been found no significant difference between the Bottom-up and Top-down test strategies with respect to their positive test generation scores.
Wirelless mesh network throughput analysis using petri nets
(Izmir Institute of Technology, 2022-07) Oğuzer, Lütfü Melih Buğra; Tuğlular, Tuğkan; Tuğlular, Tuğkan; Belli, Fevzi
Evolving technology has made the understanding of quality perception in software processes more difficult. Unlike other sectors, rapid adaptation and software development processes have become a critical issue. This issue can especially be observed in the service, telecommunication, and high technology sectors. User demands and competition are quite high and with this competition, the need to subject the customized or developed software to rapid testing processes has formed. Undoubtedly, this process implies a great responsibility for the "quality assurance" teams. This responsibility has reached a level that can only be handled by the quality assurance departments that automate the testing cycles. However, it is also important that these cycles are very efficient. Our research is concerned with modeling test processes with Petri nets and creating test scenarios based on this modeling to make automation processes in the telecommunications industry more efficient. In this research, the performance analysis of wireless mesh networks is executed through place/transition petri-net modeling. Through this modeling, reusable test scenarios which were compared and analyzed with traditional automation processes were created for performance tests. The research also addresses another topic which is the shortening of the modeling processes created with Petri nets and how to make them more efficient. In this context, a tool has been developed in order to shorten the modeling process and analyze the reusable test scenarios. Finally, ten test engineers were interviewed about reusable test processes. In these interviews, feedback was provided on reusable test scenarios in test automation processes.