Browsing by Author "Ufuktepe, Deniz Kavzak"

Now showing 1 - 2 of 2

Citation - WoS: 1
Citation - Scopus: 1
A Metric for Measuring Test Input Generation Effectiveness of Test Generation Methods for Boolean Expressions
(Ieee, 2021) Ufuktepe, Deniz Kavzak; Ufuktepe, Ekincan; Ayav, Tolga; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü
The literature includes several methods to generate test inputs for Boolean expressions. The effectiveness of those methods needs to be analyzed by extensive comparisons. To this end, mutation analysis is often benefited by applying a distinctively selected set of mutants on each test generation method. Mutation analysis provides substantive information about the effectiveness of a test suite by indicating the percentage of killed mutants, which is a common metric. However, as we claim and show in this paper, this metric alone is not sufficient to demonstrate the effectiveness of the methods. For a test generation method, the amount of generated test inputs is also an important attribute to evaluate effectiveness. To the best of our knowledge, there is no metric that measures the effectiveness within a scale taking into account several attributes. In this study, we propose a new metric to measure the effectiveness of test input generation methods, which takes into account both the number of killed mutants and the number of test inputs. We demonstrate our new metric on three well-known test input generation methods for Boolean expressions.
Citation - WoS: 9
Test input generation from cause-effect graphs
(Springer, 2021) Ufuktepe, Deniz Kavzak; Ayav, Tolga; Belli, Fevzi; Ayav, Tolga; Bilgisayar Mühendisliği Bölümü
Cause-effect graphing is a well-known requirement-based and systematic testing method with a heuristic approach. Since it was introduced by Myers in 1979, there have not been any sufficiently comprehensive studies to generate test inputs from these graphs. However, there exist several methods for test input generation from Boolean expressions. Cause-effect graphs can be more convenient for a wide variety of users compared to Boolean expressions. Moreover, they can be used to enforce common constraints and rules on the system variables of different expressions of the system. This study proposes a new mutant-based test input generation method, Spectral Testing for Boolean specification models based on spectral analysis of Boolean expressions using mutations of the original expression. Unlike Myers' method, Spectral Testing is an algorithmic and deterministic method, in which we model the possible faults systematically. Furthermore, the conversion of cause-effect graphs between Boolean expressions is explored so that the existing test input generation methods for Boolean expressions can be exploited for cause-effect graphing. A software is developed as an open-source extendable tool for generating test inputs from cause-effect graphs by using different methods and performing mutation analysis for quantitative evaluation on these methods for further analysis and comparison. Selected methods, MI, MAX-A, MUTP, MNFP, CUTPNFP, MUMCUT, Unique MC/DC, and Masking MC/DC are implemented together with Myers' technique and the proposed Spectral Testing in the developed tool. For mutation testing, 9 common fault types of Boolean expressions are modeled, implemented, and generated in the tool. An XML-based standard on top of GraphML representing a cause-effect graph is proposed and is used as the input type to the approach. An empirical study is performed by a case study on 5 different systems with various requirements, including the benchmark set from the TCAS-II system. Our results show that the proposed XML-based cause-effect graph model can be used to represent system requirements. The developed tool can be used for test input generation from proposed cause-effect graph models and can perform mutation analysis to distinguish between the methods with respect to the effectiveness of test inputs and their mutant kill scores. The proposed Spectral Testing method outperforms the state-of-the-art methods in the context of critical systems, regarding both the effectiveness and mutant kill scores of the generated test inputs, and increasing the chances of revealing faults in the system and reducing the cost of testing. Moreover, the proposed method can be used as a separate or complementary method to other well-performing test input generation methods for covering specific fault types.