Edge devices,due to their limited computational and storage resources,often require the use of compilers for program optimization.Therefore,ensuring the security and reliability of these compilers is of paramount impo...Edge devices,due to their limited computational and storage resources,often require the use of compilers for program optimization.Therefore,ensuring the security and reliability of these compilers is of paramount importance in the emerging field of edge AI.One widely used testing method for this purpose is fuzz testing,which detects bugs by inputting random test cases into the target program.However,this process consumes significant time and resources.To improve the efficiency of compiler fuzz testing,it is common practice to utilize test case prioritization techniques.Some researchers use machine learning to predict the code coverage of test cases,aiming to maximize the test capability for the target compiler by increasing the overall predicted coverage of the test cases.Nevertheless,these methods can only forecast the code coverage of the compiler at a specific optimization level,potentially missing many optimization-related bugs.In this paper,we introduce C-CORE(short for Clustering by Code Representation),the first framework to prioritize test cases according to their code representations,which are derived directly from the source codes.This approach avoids being limited to specific compiler states and extends to a broader range of compiler bugs.Specifically,we first train a scaled pre-trained programming language model to capture as many common features as possible from the test cases generated by a fuzzer.Using this pre-trained model,we then train two downstream models:one for predicting the likelihood of triggering a bug and another for identifying code representations associated with bugs.Subsequently,we cluster the test cases according to their code representations and select the highest-scoring test case from each cluster as the high-quality test case.This reduction in redundant testing cases leads to time savings.Comprehensive evaluation results reveal that code representations are better at distinguishing test capabilities,and C-CORE significantly enhances testing efficiency.Across four datasets,C-CORE increases the average of the percentage of faults detected(APFD)value by 0.16 to 0.31 and reduces test time by over 50% in 46% of cases.When compared to the best results from approaches using predicted code coverage,C-CORE improves the APFD value by 1.1% to 12.3% and achieves an overall time-saving of 159.1%.展开更多
In software testing,the quality of test cases is crucial,but manual generation is time-consuming.Various automatic test case generation methods exist,requiring careful selection based on program features.Current evalu...In software testing,the quality of test cases is crucial,but manual generation is time-consuming.Various automatic test case generation methods exist,requiring careful selection based on program features.Current evaluation methods compare a limited set of metrics,which does not support a larger number of metrics or consider the relative importance of each metric to the final assessment.To address this,we propose an evaluation tool,the Test Case Generation Evaluator(TCGE),based on the learning to rank(L2R)algorithm.Unlike previous approaches,our method comprehensively evaluates algorithms by considering multiple metrics,resulting in a more reasoned assessment.The main principle of the TCGE is the formation of feature vectors that are of concern by the tester.Through training,the feature vectors are sorted to generate a list,with the order of the methods on the list determined according to their effectiveness on the tested assembly.We implement TCGE using three L2R algorithms:Listnet,LambdaMART,and RFLambdaMART.Evaluation employs a dataset with features of classical test case generation algorithms and three metrics—Normalized Discounted Cumulative Gain(NDCG),Mean Average Precision(MAP),and Mean Reciprocal Rank(MRR).Results demonstrate the TCGE’s superior effectiveness in evaluating test case generation algorithms compared to other methods.Among the three L2R algorithms,RFLambdaMART proves the most effective,achieving an accuracy above 96.5%,surpassing LambdaMART by 2%and Listnet by 1.5%.Consequently,the TCGE framework exhibits significant application value in the evaluation of test case generation algorithms.展开更多
Software needs modifications and requires revisions regularly.Owing to these revisions,retesting software becomes essential to ensure that the enhancements made,have not affected its bug-free functioning.The time and ...Software needs modifications and requires revisions regularly.Owing to these revisions,retesting software becomes essential to ensure that the enhancements made,have not affected its bug-free functioning.The time and cost incurred in this process,need to be reduced by the method of test case selection and prioritization.It is observed that many nature-inspired techniques are applied in this area.African Buffalo Optimization is one such approach,applied to regression test selection and prioritization.In this paper,the proposed work explains and proves the applicability of the African Buffalo Optimization approach to test case selection and prioritization.The proposed algorithm converges in polynomial time(O(n^(2))).In this paper,the empirical evaluation of applying African Buffalo Optimization for test case prioritization is done on sample data set with multiple iterations.An astounding 62.5%drop in size and a 48.57%drop in the runtime of the original test suite were recorded.The obtained results are compared with Ant Colony Optimization.The comparative analysis indicates that African Buffalo Optimization and Ant Colony Optimization exhibit similar fault detection capabilities(80%),and a reduction in the overall execution time and size of the resultant test suite.The results and analysis,hence,advocate and encourages the use of African Buffalo Optimization in the area of test case selection and prioritization.展开更多
Test Case Prioritization(TCP)techniques perform better than other regression test optimization techniques including Test Suite Reduction(TSR)and Test Case Selection(TCS).Many TCP techniques are available,and their per...Test Case Prioritization(TCP)techniques perform better than other regression test optimization techniques including Test Suite Reduction(TSR)and Test Case Selection(TCS).Many TCP techniques are available,and their performance is usually measured through a metric Average Percentage of Fault Detection(APFD).This metric is value-neutral because it only works well when all test cases have the same cost,and all faults have the same severity.Using APFD for performance evaluation of test case orders where test cases cost or faults severity varies is prone to produce false results.Therefore,using the right metric for performance evaluation of TCP techniques is very important to get reliable and correct results.In this paper,two value-based TCP techniques have been introduced using Genetic Algorithm(GA)including Value-Cognizant Fault Detection-Based TCP(VCFDB-TCP)and Value-Cognizant Requirements Coverage-Based TCP(VCRCB-TCP).Two novel value-based performance evaluation metrics are also introduced for value-based TCP including Average Percentage of Fault Detection per value(APFDv)and Average Percentage of Requirements Coverage per value(APRCv).Two case studies are performed to validate proposed techniques and performance evaluation metrics.The proposed GA-based techniques outperformed the existing state-of-the-art TCP techniques including Original Order(OO),Reverse Order(REV-O),Random Order(RO),and Greedy algorithm.展开更多
Regression testing is a widely used approach to confirm the correct functionality of the software in incremental development.The use of test cases makes it easier to test the ripple effect of changed requirements.Rigo...Regression testing is a widely used approach to confirm the correct functionality of the software in incremental development.The use of test cases makes it easier to test the ripple effect of changed requirements.Rigorous testingmay help in meeting the quality criteria that is based on the conformance to the requirements as given by the intended stakeholders.However,a minimized and prioritized set of test cases may reduce the efforts and time required for testingwhile focusing on the timely delivery of the software application.In this research,a technique named Test Reduce has been presented to get a minimal set of test cases based on high priority to ensure that the web applicationmeets the required quality criteria.A new technique TestReduce is proposed with a blend of genetic algorithm to find an optimized and minimal set of test cases.The ultimate objective associated with this study is to provide a technique that may solve the minimization problem of regression test cases in the case of linked requirements.In this research,the 100-Dollar prioritization approach is used to define the priority of the new requirements.展开更多
Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can ...Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can bring unrestricted and misimplemented behaviors of a system and can lead to fatal vulnerabilities such as remote code execution or sensitive data leakage.Therefore,it is an essential task to discover unrestricted and misimplemented behaviors of a system.However,it is a daunting task for security experts to discover such vulnerabilities in advance because it is timeconsuming and error-prone to analyze the whole code in detail.Also,most of the existing vulnerability detection approaches still focus on detecting memory corruption bugs because these bugs are the dominant root cause of software vulnerabilities.This paper proposes SMINER,a novel approach that discovers vulnerabilities caused by unrestricted and misimplemented behaviors.SMINER first collects unit test cases for the target system from the official repository.Next,preprocess the collected code fragments.SMINER uses pre-processed data to show the security policies that can occur on the target system and creates a test case for security policy testing.To demonstrate the effectiveness of SMINER,this paper evaluates SMINER against Robot Operating System(ROS),a real-world system used for intelligent robots in Amazon and controlling satellites in National Aeronautics and Space Administration(NASA).From the evaluation,we discovered two real-world vulnerabilities in ROS.展开更多
Both unit and integration testing are incredibly crucial for almost any software application because each of them operates a distinct process to examine the product.Due to resource constraints,when software is subject...Both unit and integration testing are incredibly crucial for almost any software application because each of them operates a distinct process to examine the product.Due to resource constraints,when software is subjected to modifications,the drastic increase in the count of test cases forces the testers to opt for a test optimization strategy.One such strategy is test case prioritization(TCP).Existing works have propounded various methodologies that re-order the system-level test cases intending to boost either the fault detection capabilities or the coverage efficacy at the earliest.Nonetheless,singularity in objective functions and the lack of dissimilitude among the re-ordered test sequences have degraded the cogency of their approaches.Considering such gaps and scenarios when the meteoric and continuous updations in the software make the intensive unit and integration testing process more fragile,this study has introduced a memetics-inspired methodology for TCP.The proposed structure is first embedded with diverse parameters,and then traditional steps of the shuffled-frog-leaping approach(SFLA)are followed to prioritize the test cases at unit and integration levels.On 5 standard test functions,a comparative analysis is conducted between the established algorithms and the proposed approach,where the latter enhances the coverage rate and fault detection of re-ordered test sets.Investigation results related to the mean average percentage of fault detection(APFD)confirmed that the proposed approach exceeds the memetic,basic multi-walk,PSO,and optimized multi-walk by 21.7%,13.99%,12.24%,and 11.51%,respectively.展开更多
Software testing is an important and cost intensive activity in software development.The major contribution in cost is due to test case generations.Requirement-based testing is an approach in which test cases are deri...Software testing is an important and cost intensive activity in software development.The major contribution in cost is due to test case generations.Requirement-based testing is an approach in which test cases are derivative from requirements without considering the implementation’s internal structure.Requirement-based testing includes functional and nonfunctional requirements.The objective of this study is to explore the approaches that generate test cases from requirements.A systematic literature review based on two research questions and extensive quality assessment criteria includes studies.The study identies 30 primary studies from 410 studies spanned from 2000 to 2018.The review’s nding shows that 53%of journal papers,42%of conference papers,and 5%of book chapters’address requirementsbased testing.Most of the studies use UML,activity,and use case diagrams for test case generation from requirements.One of the signicant lessons learned is that most software testing errors are traced back to errors in natural language requirements.A substantial amount of work focuses on UML diagrams for test case generations,which cannot capture all the system’s developed attributes.Furthermore,there is a lack of UML-based models that can generate test cases from natural language requirements by rening them in context.Coverage criteria indicate how efciently the testing has been performed 12.37%of studies use requirements coverage,20%of studies cover path coverage,and 17%study basic coverage.展开更多
In order to improve the efficiency of regression testing in web application,the control flow graph and the greedy algorithm are adopted.This paper considers a web page as a basic unit and introduces a test case select...In order to improve the efficiency of regression testing in web application,the control flow graph and the greedy algorithm are adopted.This paper considers a web page as a basic unit and introduces a test case selection method for web application regression testing based on the control flow graph.This method is safe enough to the test case selection.On the base of features of request sequence in web application,the minimization technique and the priority of test cases are taken into consideration in the process of execution of test cases in regression testing for web application.The improved greedy algorithm is also raised resulting in optimization of execution of test cases.The experiments indicate that the number of test cases which need to be retested is reduced,and the efficiency of execution of test cases is also improved.展开更多
Reusing test cases from existing test case library is quite common in the software testing field. Testing practice tells us that there is a strong relationship between the granularity of a function unit under testing ...Reusing test cases from existing test case library is quite common in the software testing field. Testing practice tells us that there is a strong relationship between the granularity of a function unit under testing and that of the test case. A function unit with small granularity usually results in the test cases with the same small granularity. Therefore a test case defined as the function point,i. e.,the smallest size function unit,was provided for the first time.Though test cases with smaller granularity usually have better reusability,the cost of accurately reusing and integrating such test cases is also higher. In order to balance the test case reusability and the cost of test case reuse,a novel test case reuse model based on the function point was proposed in this paper. In this model,a reusable test case for specification-based testing was defined and some reuse strategies and three formal reuse methods were given. Finally,the complete automatic software process was realized by a reusing generation tool. The new method has improved reuse accuracy,while greatly enhances the software productivity.展开更多
Selection of test cases plays a key role in improving testing efficiency. Black-box testing is an important way of testing, and its validity lies on the selection of test cases in some sense. A reasonable and effectiv...Selection of test cases plays a key role in improving testing efficiency. Black-box testing is an important way of testing, and its validity lies on the selection of test cases in some sense. A reasonable and effective method about the selection and generation of test cases is urgently needed. This letter first introduces some usualmethods on black-box test case generation,then proposes a new algorithm based on interface parameters and discusses its properties, finally shows the effectiveness of the algorithm.展开更多
This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. A...This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. According to test adequacy criteria in scenario testing, the model is designed to correspond test cases in the light of logic block(LB). The final test cases that conform to the test adequacy criteria can be obtained through test case combination and test case reduction. In the last part of the paper, example of actual workflow is to design the efficient test case. Therefore the method is proved to be effective.展开更多
Software operational profile （SOP） is used in software reliability prediction, software quality assessment, performance analysis of software, test case allocation, determination of ＂when to stop testing,＂ etc. Due...Software operational profile （SOP） is used in software reliability prediction, software quality assessment, performance analysis of software, test case allocation, determination of ＂when to stop testing,＂ etc. Due to the limited data resources and large efforts required to collect and convert the gathered data into point estimates, reluctance is observed by the software professionals to develop the SOP. A framework is proposed to develop SOP using fuzzy logic, which requires usage data in the form of linguistics. The resulting profile is named fuzzy software operational profile （FSOP）. Based on this work, this paper proposes a generalized approach for the allocation of test cases, in which occurrence probability of operations obtained from FSOP are combined with the criticality of the operations using fuzzy inference system （FIS）. Traditional methods for the allocation of test cases do not consider the application in which software operates. This is intuitively incorrect. To solve this problem, allocation of test cases with respect to software application using the FIS model is also proposed in this paper.展开更多
To solve the problem of time-awarc test case prioritization,a hybrid algorithm composed of integer linear programming and the genetic algorithm(ILP-GA)is proposed.First,the test case suite which cm maximize the number...To solve the problem of time-awarc test case prioritization,a hybrid algorithm composed of integer linear programming and the genetic algorithm(ILP-GA)is proposed.First,the test case suite which cm maximize the number of covered program entities a d satisfy time constraints is selected by integer linea progamming.Secondly,the individual is encoded according to the cover matrices of entities,and the coverage rate of program entities is used as the fitness function and the genetic algorithm is used to prioritize the selected test cases.Five typical open source projects are selected as benchmark programs.Branch and method are selected as program entities,and time constraint percentages a e 25%and 75%.The experimental results show that the ILP-GA convergence has faster speed and better stability than ILP-additional and IP-total in most cases,which contributes to the detection of software defects as early as possible and reduces the software testing costs.展开更多
Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capabi...Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capability of software testing activity.Test Case Prioritization(TCP)remains a challenging issue since prioritizing test cases is unsatisfactory in terms of Average Percentage of Faults Detected(APFD)and time spent upon execution results.TCP ismainly intended to design a collection of test cases that can accomplish early optimization using preferred characteristics.The studies conducted earlier focused on prioritizing the available test cases in accelerating fault detection rate during software testing.In this aspect,the current study designs aModified Harris Hawks Optimization based TCP(MHHO-TCP)technique for software testing.The aim of the proposed MHHO-TCP technique is to maximize APFD and minimize the overall execution time.In addition,MHHO algorithm is designed to boost the exploration and exploitation abilities of conventional HHO algorithm.In order to validate the enhanced efficiency of MHHO-TCP technique,a wide range of simulations was conducted on different benchmark programs and the results were examined under several aspects.The experimental outcomes highlight the improved efficiency of MHHO-TCP technique over recent approaches under different measures.展开更多
A new model of event and message driven Petri network(EMDPN) based on the characteristic of class interaction for messages passing between two objects was extended. Using EMDPN interaction graph, a class hierarchical ...A new model of event and message driven Petri network(EMDPN) based on the characteristic of class interaction for messages passing between two objects was extended. Using EMDPN interaction graph, a class hierarchical test-case generation algorithm with cooperated paths (copaths) was proposed, which can be used to solve the problems resulting from the class inheritance mechanism encountered in object-oriented software testing such as oracle, message transfer errors, and unreachable statement. Finally, the testing sufficiency was analyzed with the ordered sequence testing criterion(OSC). The results indicate that the test cases stemmed from newly proposed automatic algorithm of copaths generation satisfies synchronization message sequences testing criteria, therefore the proposed new algorithm of copaths generation has a good coverage rate.展开更多
Model checking techniques have been widely used in verifying web service compositions to ensure the trustworthi- ness. However, little research has focused on testing web services. Based on the research of model check...Model checking techniques have been widely used in verifying web service compositions to ensure the trustworthi- ness. However, little research has focused on testing web services. Based on the research of model checking techniques~ we propose a model checking based approach for testing web service composition which is described by using the web services choreography description language （WS-CDL）. According to worldwide web consortium （W3C） candidate recommendation, the WS-CDL specification provides a language for characterizing interactions between distinct web services using XML. Since the behaviors of web service composition are asynchronous, distributed, low-coupled and platform independent, we employ the guarded automata （GA） model for specifying the composition described in WS-CDL and using the simple promela interpreter （SPIN） model checker for detecting the collaborations of web services. Test cases can be transformed from counterexamples generated by SPIN using adequacy criteria. In this paper we apply the transition coverage criterion for generating counterex- amples. To illustrate our approach, we set ＂E-commerce service system＂ as an example for demonstrating how test cases can be generated using SPIN for compositions specified in WS-CDL.展开更多
Regression testing(RT)is an essential but an expensive activity in software development.RT confirms that new faults/errors will not have occurred in the modified program.RT efficiency can be improved through an effect...Regression testing(RT)is an essential but an expensive activity in software development.RT confirms that new faults/errors will not have occurred in the modified program.RT efficiency can be improved through an effective technique of selected only modified test cases that appropriate to the modifications within the given time frame.Earlier,several test case selection approaches have been introduced,but either these techniques were not sufficient according to the requirements of software tester experts or they are ineffective and cannot be used for available test suite specifications and architecture.To address these limitations,we recommend an improved and efficient test case selection(TCS)algorithm for RT.Our proposed technique decreases the execution time and redundancy of the duplicate test cases(TC)and detects onlymodified changes that appropriate to themodifications in test cases.To reduce execution time for TCS,evaluation results of our proposed approach are established on fault detection,redundancy and already executed test case.Results indicate that proposed technique decreases the inclusive testing time of TCS to execute modified test cases by,on average related to a method of Hybrid Whale Algorithm(HWOA),which is a progressive TCS approach in regression testing for a single product.展开更多
Aiming to improve the Structured Query Language( SQL) injection penetration test accuracy through the formalismguided test case generation,an attack purpose based attack tree model of SQL injection is proposed,and the...Aiming to improve the Structured Query Language( SQL) injection penetration test accuracy through the formalismguided test case generation,an attack purpose based attack tree model of SQL injection is proposed,and then under the guidance of this model, the formal descriptions for the SQL injection vulnerability feature and SQL injection attack inputs are established. Moreover,according to new coverage criteria,these models are instantiated and the executable test cases are generated.Experiments show that compared with the random enumerated test case used in other works,the test case generated by our method can detect the SQL injection vulnerability more effectively. Therefore,the false negative is reduced and the test accuracy is improved.展开更多
Software testing is an important means to assure the software quality. This paper presents a practicable method to generate test cases of software testing, which is operational and high efficient. We discuss the ident...Software testing is an important means to assure the software quality. This paper presents a practicable method to generate test cases of software testing, which is operational and high efficient. We discuss the identification of software specification categories and choices and make a classification tree. Based on the orthogonal array, it is easy to generate test cases. The number of this method is less than that of all combination of the choices.展开更多
文摘Edge devices,due to their limited computational and storage resources,often require the use of compilers for program optimization.Therefore,ensuring the security and reliability of these compilers is of paramount importance in the emerging field of edge AI.One widely used testing method for this purpose is fuzz testing,which detects bugs by inputting random test cases into the target program.However,this process consumes significant time and resources.To improve the efficiency of compiler fuzz testing,it is common practice to utilize test case prioritization techniques.Some researchers use machine learning to predict the code coverage of test cases,aiming to maximize the test capability for the target compiler by increasing the overall predicted coverage of the test cases.Nevertheless,these methods can only forecast the code coverage of the compiler at a specific optimization level,potentially missing many optimization-related bugs.In this paper,we introduce C-CORE(short for Clustering by Code Representation),the first framework to prioritize test cases according to their code representations,which are derived directly from the source codes.This approach avoids being limited to specific compiler states and extends to a broader range of compiler bugs.Specifically,we first train a scaled pre-trained programming language model to capture as many common features as possible from the test cases generated by a fuzzer.Using this pre-trained model,we then train two downstream models:one for predicting the likelihood of triggering a bug and another for identifying code representations associated with bugs.Subsequently,we cluster the test cases according to their code representations and select the highest-scoring test case from each cluster as the high-quality test case.This reduction in redundant testing cases leads to time savings.Comprehensive evaluation results reveal that code representations are better at distinguishing test capabilities,and C-CORE significantly enhances testing efficiency.Across four datasets,C-CORE increases the average of the percentage of faults detected(APFD)value by 0.16 to 0.31 and reduces test time by over 50% in 46% of cases.When compared to the best results from approaches using predicted code coverage,C-CORE improves the APFD value by 1.1% to 12.3% and achieves an overall time-saving of 159.1%.
文摘In software testing,the quality of test cases is crucial,but manual generation is time-consuming.Various automatic test case generation methods exist,requiring careful selection based on program features.Current evaluation methods compare a limited set of metrics,which does not support a larger number of metrics or consider the relative importance of each metric to the final assessment.To address this,we propose an evaluation tool,the Test Case Generation Evaluator(TCGE),based on the learning to rank(L2R)algorithm.Unlike previous approaches,our method comprehensively evaluates algorithms by considering multiple metrics,resulting in a more reasoned assessment.The main principle of the TCGE is the formation of feature vectors that are of concern by the tester.Through training,the feature vectors are sorted to generate a list,with the order of the methods on the list determined according to their effectiveness on the tested assembly.We implement TCGE using three L2R algorithms:Listnet,LambdaMART,and RFLambdaMART.Evaluation employs a dataset with features of classical test case generation algorithms and three metrics—Normalized Discounted Cumulative Gain(NDCG),Mean Average Precision(MAP),and Mean Reciprocal Rank(MRR).Results demonstrate the TCGE’s superior effectiveness in evaluating test case generation algorithms compared to other methods.Among the three L2R algorithms,RFLambdaMART proves the most effective,achieving an accuracy above 96.5%,surpassing LambdaMART by 2%and Listnet by 1.5%.Consequently,the TCGE framework exhibits significant application value in the evaluation of test case generation algorithms.
基金This research is funded by the Deanship of Scientific Research at Umm Al-Qura University,Grant Code:22UQU4281755DSR02.
文摘Software needs modifications and requires revisions regularly.Owing to these revisions,retesting software becomes essential to ensure that the enhancements made,have not affected its bug-free functioning.The time and cost incurred in this process,need to be reduced by the method of test case selection and prioritization.It is observed that many nature-inspired techniques are applied in this area.African Buffalo Optimization is one such approach,applied to regression test selection and prioritization.In this paper,the proposed work explains and proves the applicability of the African Buffalo Optimization approach to test case selection and prioritization.The proposed algorithm converges in polynomial time(O(n^(2))).In this paper,the empirical evaluation of applying African Buffalo Optimization for test case prioritization is done on sample data set with multiple iterations.An astounding 62.5%drop in size and a 48.57%drop in the runtime of the original test suite were recorded.The obtained results are compared with Ant Colony Optimization.The comparative analysis indicates that African Buffalo Optimization and Ant Colony Optimization exhibit similar fault detection capabilities(80%),and a reduction in the overall execution time and size of the resultant test suite.The results and analysis,hence,advocate and encourages the use of African Buffalo Optimization in the area of test case selection and prioritization.
文摘Test Case Prioritization(TCP)techniques perform better than other regression test optimization techniques including Test Suite Reduction(TSR)and Test Case Selection(TCS).Many TCP techniques are available,and their performance is usually measured through a metric Average Percentage of Fault Detection(APFD).This metric is value-neutral because it only works well when all test cases have the same cost,and all faults have the same severity.Using APFD for performance evaluation of test case orders where test cases cost or faults severity varies is prone to produce false results.Therefore,using the right metric for performance evaluation of TCP techniques is very important to get reliable and correct results.In this paper,two value-based TCP techniques have been introduced using Genetic Algorithm(GA)including Value-Cognizant Fault Detection-Based TCP(VCFDB-TCP)and Value-Cognizant Requirements Coverage-Based TCP(VCRCB-TCP).Two novel value-based performance evaluation metrics are also introduced for value-based TCP including Average Percentage of Fault Detection per value(APFDv)and Average Percentage of Requirements Coverage per value(APRCv).Two case studies are performed to validate proposed techniques and performance evaluation metrics.The proposed GA-based techniques outperformed the existing state-of-the-art TCP techniques including Original Order(OO),Reverse Order(REV-O),Random Order(RO),and Greedy algorithm.
基金The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through Large Groups,Project under grant number RGP.2/49/43.
文摘Regression testing is a widely used approach to confirm the correct functionality of the software in incremental development.The use of test cases makes it easier to test the ripple effect of changed requirements.Rigorous testingmay help in meeting the quality criteria that is based on the conformance to the requirements as given by the intended stakeholders.However,a minimized and prioritized set of test cases may reduce the efforts and time required for testingwhile focusing on the timely delivery of the software application.In this research,a technique named Test Reduce has been presented to get a minimal set of test cases based on high priority to ensure that the web applicationmeets the required quality criteria.A new technique TestReduce is proposed with a blend of genetic algorithm to find an optimized and minimal set of test cases.The ultimate objective associated with this study is to provide a technique that may solve the minimization problem of regression test cases in the case of linked requirements.In this research,the 100-Dollar prioritization approach is used to define the priority of the new requirements.
基金This work was supported in part by the National Research Foundation of Korea(NRF)funded by the Ministry of Science and ICT(MSIT)Future Planning under Grant NRF-2020R1A2C2014336 and Grant NRF-2021R1A4A1029650.
文摘Despite the advances in automated vulnerability detection approaches,security vulnerabilities caused by design flaws in software systems are continuously appearing in real-world systems.Such security design flaws can bring unrestricted and misimplemented behaviors of a system and can lead to fatal vulnerabilities such as remote code execution or sensitive data leakage.Therefore,it is an essential task to discover unrestricted and misimplemented behaviors of a system.However,it is a daunting task for security experts to discover such vulnerabilities in advance because it is timeconsuming and error-prone to analyze the whole code in detail.Also,most of the existing vulnerability detection approaches still focus on detecting memory corruption bugs because these bugs are the dominant root cause of software vulnerabilities.This paper proposes SMINER,a novel approach that discovers vulnerabilities caused by unrestricted and misimplemented behaviors.SMINER first collects unit test cases for the target system from the official repository.Next,preprocess the collected code fragments.SMINER uses pre-processed data to show the security policies that can occur on the target system and creates a test case for security policy testing.To demonstrate the effectiveness of SMINER,this paper evaluates SMINER against Robot Operating System(ROS),a real-world system used for intelligent robots in Amazon and controlling satellites in National Aeronautics and Space Administration(NASA).From the evaluation,we discovered two real-world vulnerabilities in ROS.
文摘Both unit and integration testing are incredibly crucial for almost any software application because each of them operates a distinct process to examine the product.Due to resource constraints,when software is subjected to modifications,the drastic increase in the count of test cases forces the testers to opt for a test optimization strategy.One such strategy is test case prioritization(TCP).Existing works have propounded various methodologies that re-order the system-level test cases intending to boost either the fault detection capabilities or the coverage efficacy at the earliest.Nonetheless,singularity in objective functions and the lack of dissimilitude among the re-ordered test sequences have degraded the cogency of their approaches.Considering such gaps and scenarios when the meteoric and continuous updations in the software make the intensive unit and integration testing process more fragile,this study has introduced a memetics-inspired methodology for TCP.The proposed structure is first embedded with diverse parameters,and then traditional steps of the shuffled-frog-leaping approach(SFLA)are followed to prioritize the test cases at unit and integration levels.On 5 standard test functions,a comparative analysis is conducted between the established algorithms and the proposed approach,where the latter enhances the coverage rate and fault detection of re-ordered test sets.Investigation results related to the mean average percentage of fault detection(APFD)confirmed that the proposed approach exceeds the memetic,basic multi-walk,PSO,and optimized multi-walk by 21.7%,13.99%,12.24%,and 11.51%,respectively.
文摘Software testing is an important and cost intensive activity in software development.The major contribution in cost is due to test case generations.Requirement-based testing is an approach in which test cases are derivative from requirements without considering the implementation’s internal structure.Requirement-based testing includes functional and nonfunctional requirements.The objective of this study is to explore the approaches that generate test cases from requirements.A systematic literature review based on two research questions and extensive quality assessment criteria includes studies.The study identies 30 primary studies from 410 studies spanned from 2000 to 2018.The review’s nding shows that 53%of journal papers,42%of conference papers,and 5%of book chapters’address requirementsbased testing.Most of the studies use UML,activity,and use case diagrams for test case generation from requirements.One of the signicant lessons learned is that most software testing errors are traced back to errors in natural language requirements.A substantial amount of work focuses on UML diagrams for test case generations,which cannot capture all the system’s developed attributes.Furthermore,there is a lack of UML-based models that can generate test cases from natural language requirements by rening them in context.Coverage criteria indicate how efciently the testing has been performed 12.37%of studies use requirements coverage,20%of studies cover path coverage,and 17%study basic coverage.
基金The National Natural Science Foundation of China(No.60503020,60503033,60703086)Opening Foundation of Jiangsu Key Laboratory of Computer Information Processing Technology in Soochow University(No.KJS0714)
文摘In order to improve the efficiency of regression testing in web application,the control flow graph and the greedy algorithm are adopted.This paper considers a web page as a basic unit and introduces a test case selection method for web application regression testing based on the control flow graph.This method is safe enough to the test case selection.On the base of features of request sequence in web application,the minimization technique and the priority of test cases are taken into consideration in the process of execution of test cases in regression testing for web application.The improved greedy algorithm is also raised resulting in optimization of execution of test cases.The experiments indicate that the number of test cases which need to be retested is reduced,and the efficiency of execution of test cases is also improved.
基金National Natural Science Foundation of China(No.61262010)
文摘Reusing test cases from existing test case library is quite common in the software testing field. Testing practice tells us that there is a strong relationship between the granularity of a function unit under testing and that of the test case. A function unit with small granularity usually results in the test cases with the same small granularity. Therefore a test case defined as the function point,i. e.,the smallest size function unit,was provided for the first time.Though test cases with smaller granularity usually have better reusability,the cost of accurately reusing and integrating such test cases is also higher. In order to balance the test case reusability and the cost of test case reuse,a novel test case reuse model based on the function point was proposed in this paper. In this model,a reusable test case for specification-based testing was defined and some reuse strategies and three formal reuse methods were given. Finally,the complete automatic software process was realized by a reusing generation tool. The new method has improved reuse accuracy,while greatly enhances the software productivity.
基金Supported in part by the National Natural Science Foundation of China (NSFC)(60073012),Natural Science Foundation of Jiangsu(BK2001004)
文摘Selection of test cases plays a key role in improving testing efficiency. Black-box testing is an important way of testing, and its validity lies on the selection of test cases in some sense. A reasonable and effective method about the selection and generation of test cases is urgently needed. This letter first introduces some usualmethods on black-box test case generation,then proposes a new algorithm based on interface parameters and discusses its properties, finally shows the effectiveness of the algorithm.
基金National Torch Project, China ( No. 2009GH510068 )National High-Tech R & D Program of China ( 863 ) ( No.2007AA010401)
文摘This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. According to test adequacy criteria in scenario testing, the model is designed to correspond test cases in the light of logic block(LB). The final test cases that conform to the test adequacy criteria can be obtained through test case combination and test case reduction. In the last part of the paper, example of actual workflow is to design the efficient test case. Therefore the method is proved to be effective.
文摘Software operational profile （SOP） is used in software reliability prediction, software quality assessment, performance analysis of software, test case allocation, determination of ＂when to stop testing,＂ etc. Due to the limited data resources and large efforts required to collect and convert the gathered data into point estimates, reluctance is observed by the software professionals to develop the SOP. A framework is proposed to develop SOP using fuzzy logic, which requires usage data in the form of linguistics. The resulting profile is named fuzzy software operational profile （FSOP）. Based on this work, this paper proposes a generalized approach for the allocation of test cases, in which occurrence probability of operations obtained from FSOP are combined with the criticality of the operations using fuzzy inference system （FIS）. Traditional methods for the allocation of test cases do not consider the application in which software operates. This is intuitively incorrect. To solve this problem, allocation of test cases with respect to software application using the FIS model is also proposed in this paper.
基金The Natural Science Foundation of Education Ministry of Shaanxi Province(No.15JK1672)the Industrial Research Project of Shaanxi Province(No.2017GY-092)Special Fund for Key Discipline Construction of General Institutions of Higher Education in Shaanxi Province
文摘To solve the problem of time-awarc test case prioritization,a hybrid algorithm composed of integer linear programming and the genetic algorithm(ILP-GA)is proposed.First,the test case suite which cm maximize the number of covered program entities a d satisfy time constraints is selected by integer linea progamming.Secondly,the individual is encoded according to the cover matrices of entities,and the coverage rate of program entities is used as the fitness function and the genetic algorithm is used to prioritize the selected test cases.Five typical open source projects are selected as benchmark programs.Branch and method are selected as program entities,and time constraint percentages a e 25%and 75%.The experimental results show that the ILP-GA convergence has faster speed and better stability than ILP-additional and IP-total in most cases,which contributes to the detection of software defects as early as possible and reduces the software testing costs.
基金The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work under Grant Number(RGP.1/127/42)Princess Nourah bint Abdulrahman University Researchers Supporting Project Number(PNURSP2022R237),Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capability of software testing activity.Test Case Prioritization(TCP)remains a challenging issue since prioritizing test cases is unsatisfactory in terms of Average Percentage of Faults Detected(APFD)and time spent upon execution results.TCP ismainly intended to design a collection of test cases that can accomplish early optimization using preferred characteristics.The studies conducted earlier focused on prioritizing the available test cases in accelerating fault detection rate during software testing.In this aspect,the current study designs aModified Harris Hawks Optimization based TCP(MHHO-TCP)technique for software testing.The aim of the proposed MHHO-TCP technique is to maximize APFD and minimize the overall execution time.In addition,MHHO algorithm is designed to boost the exploration and exploitation abilities of conventional HHO algorithm.In order to validate the enhanced efficiency of MHHO-TCP technique,a wide range of simulations was conducted on different benchmark programs and the results were examined under several aspects.The experimental outcomes highlight the improved efficiency of MHHO-TCP technique over recent approaches under different measures.
基金Project(05JT1035) supported by the Science and Technology Plan of Hunan Province
文摘A new model of event and message driven Petri network(EMDPN) based on the characteristic of class interaction for messages passing between two objects was extended. Using EMDPN interaction graph, a class hierarchical test-case generation algorithm with cooperated paths (copaths) was proposed, which can be used to solve the problems resulting from the class inheritance mechanism encountered in object-oriented software testing such as oracle, message transfer errors, and unreachable statement. Finally, the testing sufficiency was analyzed with the ordered sequence testing criterion(OSC). The results indicate that the test cases stemmed from newly proposed automatic algorithm of copaths generation satisfies synchronization message sequences testing criteria, therefore the proposed new algorithm of copaths generation has a good coverage rate.
基金Project supported by the Open Foundation of State Key Laboratory of Software Engineering(Grant No.SKLSE20080712)the National Natural Science Foundation of China(Grant No.60970007)+2 种基金the National Basic Research Program of China(Grant No.2007CB310800)the Shanghai Leading Academic Discipline Project(Grant No.J50103)the Science and Technology Commission of Shanghai Municipality(Grant No.09DZ2272600)
文摘Model checking techniques have been widely used in verifying web service compositions to ensure the trustworthi- ness. However, little research has focused on testing web services. Based on the research of model checking techniques~ we propose a model checking based approach for testing web service composition which is described by using the web services choreography description language （WS-CDL）. According to worldwide web consortium （W3C） candidate recommendation, the WS-CDL specification provides a language for characterizing interactions between distinct web services using XML. Since the behaviors of web service composition are asynchronous, distributed, low-coupled and platform independent, we employ the guarded automata （GA） model for specifying the composition described in WS-CDL and using the simple promela interpreter （SPIN） model checker for detecting the collaborations of web services. Test cases can be transformed from counterexamples generated by SPIN using adequacy criteria. In this paper we apply the transition coverage criterion for generating counterex- amples. To illustrate our approach, we set ＂E-commerce service system＂ as an example for demonstrating how test cases can be generated using SPIN for compositions specified in WS-CDL.
基金This work was supported in part by the Research Management Center(RMC),Universiti Teknologi Malaysia(UTM)and Ministry of Higher Education Malaysia(MOHE)through the UTM High Impact Research(UTMHR)grant scheme under(Vot Number Q.J130000.2451.08G55).
文摘Regression testing(RT)is an essential but an expensive activity in software development.RT confirms that new faults/errors will not have occurred in the modified program.RT efficiency can be improved through an effective technique of selected only modified test cases that appropriate to the modifications within the given time frame.Earlier,several test case selection approaches have been introduced,but either these techniques were not sufficient according to the requirements of software tester experts or they are ineffective and cannot be used for available test suite specifications and architecture.To address these limitations,we recommend an improved and efficient test case selection(TCS)algorithm for RT.Our proposed technique decreases the execution time and redundancy of the duplicate test cases(TC)and detects onlymodified changes that appropriate to themodifications in test cases.To reduce execution time for TCS,evaluation results of our proposed approach are established on fault detection,redundancy and already executed test case.Results indicate that proposed technique decreases the inclusive testing time of TCS to execute modified test cases by,on average related to a method of Hybrid Whale Algorithm(HWOA),which is a progressive TCS approach in regression testing for a single product.
基金National Natural Science Foundation of China(No.51274150)Tianjin Major Project of Application Foundation and Advanced Technology,China(No.12JCZDJC27800)
文摘Aiming to improve the Structured Query Language( SQL) injection penetration test accuracy through the formalismguided test case generation,an attack purpose based attack tree model of SQL injection is proposed,and then under the guidance of this model, the formal descriptions for the SQL injection vulnerability feature and SQL injection attack inputs are established. Moreover,according to new coverage criteria,these models are instantiated and the executable test cases are generated.Experiments show that compared with the random enumerated test case used in other works,the test case generated by our method can detect the SQL injection vulnerability more effectively. Therefore,the false negative is reduced and the test accuracy is improved.
基金the Aviation Science Foundation of China( Grant No.02F15001)the National Natural Science Foundation of China( Grant No.60674100)
文摘Software testing is an important means to assure the software quality. This paper presents a practicable method to generate test cases of software testing, which is operational and high efficient. We discuss the identification of software specification categories and choices and make a classification tree. Based on the orthogonal array, it is easy to generate test cases. The number of this method is less than that of all combination of the choices.
