Evaluation is an essential part of the teaching process,especially in the programming course.Both students and teachers can benefit significantly from automatic program evaluation.It shortens the time required for ass...Evaluation is an essential part of the teaching process,especially in the programming course.Both students and teachers can benefit significantly from automatic program evaluation.It shortens the time required for assessment so that students can get immediate feedback.At the same time,it can also significantly reduce the workload of teachers.Currently,the automated program assessment system mainly uses a combination of static and dynamic analysis methods.The system is faced with two crucial problems of the unfinished code evaluation and the template code construction.This paper proposes a method of combining deep learning with static analysis.The syntax tree repair is used to solve the problem that the code with compiling errors cannot generate the correct syntax tree.Moreover,the target code is converted to a subset of solution space through the syntax tree standardization,which reduces the number of template code needed.Based on deep learning,the embedded token vector keeps the code’s context all the time,which ensures that the lexical-semantic remains unchanged as much as possible after the syntax tree changes.Finally,the standardized tree is represented as a vector by the recursive neural network.Cosine similarity between target and template code vectors is used as an evaluation score.The experiment shows that the similarity scores obtained by this method are consistent with the expert scores.This method can provide support for future research,such as difficult feedback and has great significance.展开更多
Fault localization techniques are originally proposed to assist in manual debugging by generally producing a rank list of suspicious locations.With the increasing popularity of automated program repair,the fault local...Fault localization techniques are originally proposed to assist in manual debugging by generally producing a rank list of suspicious locations.With the increasing popularity of automated program repair,the fault localization techniques have been introduced to effectively reduce the search space of automated program repair.Unlike developers who mainly focus on the rank information,current automated program repair has two strategies to use the fault localization information:suspiciousness-first algorithm(SFA)based on the suspiciousness accuracy and rank-first algorithm(RFA)relying on the rank accuracy.However,despite the fact that the two different usages are widely adopted by current automated program repair and may result in different repair results,little is known about the impacts of the two strategies on automated program repair.In this paper we empirically compare the performance of SFA and RFA in the context of automated program repair.Specifically,we implement the two strategies and six well-studied fault localization techniques into four state-of-the-art automated program repair tools,and then use these tools to perform repair experiments on 60 real-world bugs from Defects4J.Our study presents a number of interesting findings:RFA outperforms SFA in 70.02%of cases when measured by the number of candidate patches generated before a valid patch is found(NCP),while SFA performs better in parallel repair and patch diversity;the performance of SFA can be improved by increasing the suspiciousness accuracy of fault localization techniques;finally,we use SimFix that deploys SFA to successfully repair four extra Defects4J bugs which cannot be repaired by SimFix originally using RFA.These observations provide a new perspective for future research on the usage and improvement of fault localization in automated program repair.展开更多
基金supported by the 2018-2020 Higher Education Talent Training Quality and Teaching Reform Project of Sichuan Province(Grant No.JG2018-46)the Science and Technology Planning Program of Sichuan University and Luzhou(Grant No.2017CDLZG30)the Postdoctoral Science fund of Sichuan University(Grant No.2019SCU12058).
文摘Evaluation is an essential part of the teaching process,especially in the programming course.Both students and teachers can benefit significantly from automatic program evaluation.It shortens the time required for assessment so that students can get immediate feedback.At the same time,it can also significantly reduce the workload of teachers.Currently,the automated program assessment system mainly uses a combination of static and dynamic analysis methods.The system is faced with two crucial problems of the unfinished code evaluation and the template code construction.This paper proposes a method of combining deep learning with static analysis.The syntax tree repair is used to solve the problem that the code with compiling errors cannot generate the correct syntax tree.Moreover,the target code is converted to a subset of solution space through the syntax tree standardization,which reduces the number of template code needed.Based on deep learning,the embedded token vector keeps the code’s context all the time,which ensures that the lexical-semantic remains unchanged as much as possible after the syntax tree changes.Finally,the standardized tree is represented as a vector by the recursive neural network.Cosine similarity between target and template code vectors is used as an evaluation score.The experiment shows that the similarity scores obtained by this method are consistent with the expert scores.This method can provide support for future research,such as difficult feedback and has great significance.
基金This research was supported in part by the National Natural Science Foundation of China(Grant Nos.61672529,61379054,61602504,61502015)the Fundamental Research Funds for the Central Universities(2019CDXYRJ0011)National Defense Science Foundation of China(3001010).
文摘Fault localization techniques are originally proposed to assist in manual debugging by generally producing a rank list of suspicious locations.With the increasing popularity of automated program repair,the fault localization techniques have been introduced to effectively reduce the search space of automated program repair.Unlike developers who mainly focus on the rank information,current automated program repair has two strategies to use the fault localization information:suspiciousness-first algorithm(SFA)based on the suspiciousness accuracy and rank-first algorithm(RFA)relying on the rank accuracy.However,despite the fact that the two different usages are widely adopted by current automated program repair and may result in different repair results,little is known about the impacts of the two strategies on automated program repair.In this paper we empirically compare the performance of SFA and RFA in the context of automated program repair.Specifically,we implement the two strategies and six well-studied fault localization techniques into four state-of-the-art automated program repair tools,and then use these tools to perform repair experiments on 60 real-world bugs from Defects4J.Our study presents a number of interesting findings:RFA outperforms SFA in 70.02%of cases when measured by the number of candidate patches generated before a valid patch is found(NCP),while SFA performs better in parallel repair and patch diversity;the performance of SFA can be improved by increasing the suspiciousness accuracy of fault localization techniques;finally,we use SimFix that deploys SFA to successfully repair four extra Defects4J bugs which cannot be repaired by SimFix originally using RFA.These observations provide a new perspective for future research on the usage and improvement of fault localization in automated program repair.
