Unsupervised methods based on density representation have shown their abilities in anomaly detection,but detection performance still needs to be improved.Specifically,approaches using normalizing flows can accurately ...Unsupervised methods based on density representation have shown their abilities in anomaly detection,but detection performance still needs to be improved.Specifically,approaches using normalizing flows can accurately evaluate sample distributions,mapping normal features to the normal distribution and anomalous features outside it.Consequently,this paper proposes a Normalizing Flow-based Bidirectional Mapping Residual Network(NF-BMR).It utilizes pre-trained Convolutional Neural Networks(CNN)and normalizing flows to construct discriminative source and target domain feature spaces.Additionally,to better learn feature information in both domain spaces,we propose the Bidirectional Mapping Residual Network(BMR),which maps sample features to these two spaces for anomaly detection.The two detection spaces effectively complement each other’s deficiencies and provide a comprehensive feature evaluation from two perspectives,which leads to the improvement of detection performance.Comparative experimental results on the MVTec AD and DAGM datasets against the Bidirectional Pre-trained Feature Mapping Network(B-PFM)and other state-of-the-art methods demonstrate that the proposed approach achieves superior performance.On the MVTec AD dataset,NF-BMR achieves an average AUROC of 98.7%for all 15 categories.Especially,it achieves 100%optimal detection performance in five categories.On the DAGM dataset,the average AUROC across ten categories is 98.7%,which is very close to supervised methods.展开更多
Nowadays,web systems and servers are constantly at great risk from cyberattacks.This paper proposes a novel approach to detecting abnormal network traffic using a bidirectional long short-term memory(LSTM)network in c...Nowadays,web systems and servers are constantly at great risk from cyberattacks.This paper proposes a novel approach to detecting abnormal network traffic using a bidirectional long short-term memory(LSTM)network in combination with the ensemble learning technique.First,the binary classification module was used to detect the current abnormal flow.Then,the abnormal flows were fed into the multilayer classification module to identify the specific type of flow.In this research,a deep learning bidirectional LSTM model,in combination with the convolutional neural network and attention technique,was deployed to identify a specific attack.To solve the real-time intrusion-detecting problem,a stacking ensemble-learning model was deployed to detect abnormal intrusion before being transferred to the attack classification module.The class-weight technique was applied to overcome the data imbalance between the attack layers.The results showed that our approach gained good performance and the F1 accuracy on the CICIDS2017 data set reached 99.97%,which is higher than the results obtained in other research.展开更多
It has long been a challenging task to detect an anomaly in a crowded scene.In this paper,a selfsupervised framework called the abnormal event detection network(AED-Net),which is composed of a principal component anal...It has long been a challenging task to detect an anomaly in a crowded scene.In this paper,a selfsupervised framework called the abnormal event detection network(AED-Net),which is composed of a principal component analysis network(PCAnet)and kernel principal component analysis(kPCA),is proposed to address this problem.Using surveillance video sequences of different scenes as raw data,the PCAnet is trained to extract high-level semantics of the crowd’s situation.Next,kPCA,a one-class classifier,is trained to identify anomalies within the scene.In contrast to some prevailing deep learning methods,this framework is completely self-supervised because it utilizes only video sequences of a normal situation.Experiments in global and local abnormal event detection are carried out on Monitoring Human Activity dataset from University of Minnesota(UMN dataset)and Anomaly Detection dataset from University of California,San Diego(UCSD dataset),and competitive results that yield a better equal error rate(EER)and area under curve(AUC)than other state-of-the-art methods are observed.Furthermore,by adding a local response normalization(LRN)layer,we propose an improvement to the original AED-Net.The results demonstrate that this proposed version performs better by promoting the framework’s generalization capacity.展开更多
To improve the detection accuracy and robustness of crowd anomaly detection,especially crowd emergency evacuation detection,the abnormal crowd behavior detection method is proposed.This method is based on the improved...To improve the detection accuracy and robustness of crowd anomaly detection,especially crowd emergency evacuation detection,the abnormal crowd behavior detection method is proposed.This method is based on the improved statistical global optical flow entropy which can better describe the degree of chaos of crowd.First,the optical flow field is extracted from the video sequences and a 2D optical flow histogram is gained.Then,the improved optical flow entropy,combining information theory with statistical physics is calculated from 2D optical flow histograms.Finally,the anomaly can be detected according to the abnormality judgment formula.The experimental results show that the detection accuracy achieved over 95%in three public video datasets,which indicates that the proposed algorithm outperforms other state-of-the-art algorithms.展开更多
To detect effectively unknown anomalous attack behaviors of network traffic,an Unsupervised Anomaly Detection approach for network flow using Immune Network based K-means clustering(UADINK)is proposed.In UADINK,artifi...To detect effectively unknown anomalous attack behaviors of network traffic,an Unsupervised Anomaly Detection approach for network flow using Immune Network based K-means clustering(UADINK)is proposed.In UADINK,artificial immune network based K-means clustering algorithm(aiNet_KMC)is introduced to cluster network flow,i.e.extracting abstract internal images from network flows and obtaining an optimizing parameter K of K-means by aiNet model,and network flows are clustered by K-means algorithm.The cluster labeling algorithm(clusLA)and the network flow anomaly detection algorithm(NFAD)are introduced to detect anomalous attack behaviors of network flows,where the clusLA algorithm is used for labeling whether each cluster belongs to malicious,and the labeled clusters are regarded as detectors to identify anomaly network flows by NFAD.To evaluate the effectiveness of UADINK,the ISCX 2012 IDS dataset is considered as the simulating experimental dataset.Compared with the NDM based K-means anomaly detection approach,the results show that UADINK is a radical anomaly detection approach in order to detect anomalies of network flows.展开更多
In recent years, network traffic data have become larger and more complex, leading to higher possibilities of network intrusion. Traditional intrusion detection methods face difficulty in processing high-speed network...In recent years, network traffic data have become larger and more complex, leading to higher possibilities of network intrusion. Traditional intrusion detection methods face difficulty in processing high-speed network data and cannot detect currently unknown attacks. Therefore, this paper proposes a network attack detection method combining a flow calculation and deep learning. The method consists of two parts: a real-time detection algorithm based on flow calculations and frequent patterns and a classification algorithm based on the deep belief network and support vector machine(DBN-SVM). Sliding window(SW) stream data processing enables real-time detection, and the DBN-SVM algorithm can improve classification accuracy. Finally, to verify the proposed method, a system is implemented.Based on the CICIDS2017 open source data set, a series of comparative experiments are conducted. The method's real-time detection efficiency is higher than that of traditional machine learning algorithms. The attack classification accuracy is 0.7 percentage points higher than that of a DBN, which is 2 percentage points higher than that of the integrated algorithm boosting and bagging methods. Hence, it is suitable for the real-time detection of high-speed network intrusions.展开更多
Distributed Denial-of-Service(DDoS)has caused great damage to the network in the big data environment.Existing methods are characterized by low computational efficiency,high false alarm rate and high false alarm rate....Distributed Denial-of-Service(DDoS)has caused great damage to the network in the big data environment.Existing methods are characterized by low computational efficiency,high false alarm rate and high false alarm rate.In this paper,we propose a DDoS attack detection method based on network flow grayscale matrix feature via multi-scale convolutional neural network(CNN).According to the different characteristics of the attack flow and the normal flow in the IP protocol,the seven-tuple is defined to describe the network flow characteristics and converted into a grayscale feature by binary.Based on the network flow grayscale matrix feature(GMF),the convolution kernel of different spatial scales is used to improve the accuracy of feature segmentation,global features and local features of the network flow are extracted.A DDoS attack classifier based on multi-scale convolution neural network is constructed.Experiments show that compared with correlation methods,this method can improve the robustness of the classifier,reduce the false alarm rate and the missing alarm rate.展开更多
Because of an explosive growth of the intrusions, necessity of anomaly-based Intrusion Detection Systems (IDSs) which are capable of detecting novel attacks, is increasing. Among those systems, flow-based detection sy...Because of an explosive growth of the intrusions, necessity of anomaly-based Intrusion Detection Systems (IDSs) which are capable of detecting novel attacks, is increasing. Among those systems, flow-based detection systems which use a series of packets exchanged between two terminals as a unit of observation, have an advantage of being able to detect anomaly which is included in only some specific sessions. However, in large-scale networks where a large number of communications takes place, analyzing every flow is not practical. On the other hand, a timeslot-based detection systems need not to prepare a number of buffers although it is difficult to specify anomaly communications. In this paper, we propose a multi-stage anomaly detection system which is combination of timeslot-based and flow-based detectors. The proposed system can reduce the number of flows which need to be subjected to flow-based analysis but yet exhibits high detection accuracy. Through experiments using data set, we present the effectiveness of the proposed method.展开更多
针对传统的基于自编码器的无监督异常声音检测方法存在特征表达能力不足的问题,提出一种基于注意力-跳跃自编码器-生成对抗网络的无监督异常声音检测方法ASAE-GAN(Attentional Skip-connected Auto Encoder and Generative Adversarial ...针对传统的基于自编码器的无监督异常声音检测方法存在特征表达能力不足的问题,提出一种基于注意力-跳跃自编码器-生成对抗网络的无监督异常声音检测方法ASAE-GAN(Attentional Skip-connected Auto Encoder and Generative Adversarial Network)。ASAE-GAN在跳跃自编码器和生成对抗网络的基础上,引入通道间注意力机制和时间注意力机制,增强模型的特征表达能力。使用MIMII数据集中的pump声音数据进行实验,评价指标使用AUC分数。结果表明:ASAE-GAN的平均AUC分数相比较于AE、UNET和Skip-GANomaly分别提升了16.27%、14.23%和6.55%,验证了其在无监督异常声音检测方面的优越性。展开更多
基金This work was supported in part by the National Key R&D Program of China 2021YFE0110500in part by the National Natural Science Foundation of China under Grant 62062021in part by the Guiyang Scientific Plan Project[2023]48-11.
文摘Unsupervised methods based on density representation have shown their abilities in anomaly detection,but detection performance still needs to be improved.Specifically,approaches using normalizing flows can accurately evaluate sample distributions,mapping normal features to the normal distribution and anomalous features outside it.Consequently,this paper proposes a Normalizing Flow-based Bidirectional Mapping Residual Network(NF-BMR).It utilizes pre-trained Convolutional Neural Networks(CNN)and normalizing flows to construct discriminative source and target domain feature spaces.Additionally,to better learn feature information in both domain spaces,we propose the Bidirectional Mapping Residual Network(BMR),which maps sample features to these two spaces for anomaly detection.The two detection spaces effectively complement each other’s deficiencies and provide a comprehensive feature evaluation from two perspectives,which leads to the improvement of detection performance.Comparative experimental results on the MVTec AD and DAGM datasets against the Bidirectional Pre-trained Feature Mapping Network(B-PFM)and other state-of-the-art methods demonstrate that the proposed approach achieves superior performance.On the MVTec AD dataset,NF-BMR achieves an average AUROC of 98.7%for all 15 categories.Especially,it achieves 100%optimal detection performance in five categories.On the DAGM dataset,the average AUROC across ten categories is 98.7%,which is very close to supervised methods.
文摘Nowadays,web systems and servers are constantly at great risk from cyberattacks.This paper proposes a novel approach to detecting abnormal network traffic using a bidirectional long short-term memory(LSTM)network in combination with the ensemble learning technique.First,the binary classification module was used to detect the current abnormal flow.Then,the abnormal flows were fed into the multilayer classification module to identify the specific type of flow.In this research,a deep learning bidirectional LSTM model,in combination with the convolutional neural network and attention technique,was deployed to identify a specific attack.To solve the real-time intrusion-detecting problem,a stacking ensemble-learning model was deployed to detect abnormal intrusion before being transferred to the attack classification module.The class-weight technique was applied to overcome the data imbalance between the attack layers.The results showed that our approach gained good performance and the F1 accuracy on the CICIDS2017 data set reached 99.97%,which is higher than the results obtained in other research.
基金This work is partially supported by the National Key Research and Development Program of China(2016YFE0204200)the National Natural Science Foundation of China(61503017)+3 种基金the Fundamental Research Funds for the Central Universities(YWF-18-BJ-J-221)the Aeronautical Science Foundation of China(2016ZC51022)the Platform CAPSEC(capteurs pour la sécurité)funded by Région Champagne-ArdenneFEDER(fonds européen de développement régional).
文摘It has long been a challenging task to detect an anomaly in a crowded scene.In this paper,a selfsupervised framework called the abnormal event detection network(AED-Net),which is composed of a principal component analysis network(PCAnet)and kernel principal component analysis(kPCA),is proposed to address this problem.Using surveillance video sequences of different scenes as raw data,the PCAnet is trained to extract high-level semantics of the crowd’s situation.Next,kPCA,a one-class classifier,is trained to identify anomalies within the scene.In contrast to some prevailing deep learning methods,this framework is completely self-supervised because it utilizes only video sequences of a normal situation.Experiments in global and local abnormal event detection are carried out on Monitoring Human Activity dataset from University of Minnesota(UMN dataset)and Anomaly Detection dataset from University of California,San Diego(UCSD dataset),and competitive results that yield a better equal error rate(EER)and area under curve(AUC)than other state-of-the-art methods are observed.Furthermore,by adding a local response normalization(LRN)layer,we propose an improvement to the original AED-Net.The results demonstrate that this proposed version performs better by promoting the framework’s generalization capacity.
基金National Natural Science Foundation of China(61701029)。
文摘To improve the detection accuracy and robustness of crowd anomaly detection,especially crowd emergency evacuation detection,the abnormal crowd behavior detection method is proposed.This method is based on the improved statistical global optical flow entropy which can better describe the degree of chaos of crowd.First,the optical flow field is extracted from the video sequences and a 2D optical flow histogram is gained.Then,the improved optical flow entropy,combining information theory with statistical physics is calculated from 2D optical flow histograms.Finally,the anomaly can be detected according to the abnormality judgment formula.The experimental results show that the detection accuracy achieved over 95%in three public video datasets,which indicates that the proposed algorithm outperforms other state-of-the-art algorithms.
文摘To detect effectively unknown anomalous attack behaviors of network traffic,an Unsupervised Anomaly Detection approach for network flow using Immune Network based K-means clustering(UADINK)is proposed.In UADINK,artificial immune network based K-means clustering algorithm(aiNet_KMC)is introduced to cluster network flow,i.e.extracting abstract internal images from network flows and obtaining an optimizing parameter K of K-means by aiNet model,and network flows are clustered by K-means algorithm.The cluster labeling algorithm(clusLA)and the network flow anomaly detection algorithm(NFAD)are introduced to detect anomalous attack behaviors of network flows,where the clusLA algorithm is used for labeling whether each cluster belongs to malicious,and the labeled clusters are regarded as detectors to identify anomaly network flows by NFAD.To evaluate the effectiveness of UADINK,the ISCX 2012 IDS dataset is considered as the simulating experimental dataset.Compared with the NDM based K-means anomaly detection approach,the results show that UADINK is a radical anomaly detection approach in order to detect anomalies of network flows.
基金supported by the National Key Research and Development Program of China(2017YFB1401300,2017YFB1401304)the National Natural Science Foundation of China(61702211,L1724007,61902203)+3 种基金Hubei Provincial Science and Technology Program of China(2017AKA191)the Self-Determined Research Funds of Central China Normal University(CCNU)from the Colleges’Basic Research(CCNU17QD0004,CCNU17GF0002)the Natural Science Foundation of Shandong Province(ZR2017QF015)the Key Research and Development Plan–Major Scientific and Technological Innovation Projects of Shandong Province(2019JZZY020101)。
文摘In recent years, network traffic data have become larger and more complex, leading to higher possibilities of network intrusion. Traditional intrusion detection methods face difficulty in processing high-speed network data and cannot detect currently unknown attacks. Therefore, this paper proposes a network attack detection method combining a flow calculation and deep learning. The method consists of two parts: a real-time detection algorithm based on flow calculations and frequent patterns and a classification algorithm based on the deep belief network and support vector machine(DBN-SVM). Sliding window(SW) stream data processing enables real-time detection, and the DBN-SVM algorithm can improve classification accuracy. Finally, to verify the proposed method, a system is implemented.Based on the CICIDS2017 open source data set, a series of comparative experiments are conducted. The method's real-time detection efficiency is higher than that of traditional machine learning algorithms. The attack classification accuracy is 0.7 percentage points higher than that of a DBN, which is 2 percentage points higher than that of the integrated algorithm boosting and bagging methods. Hence, it is suitable for the real-time detection of high-speed network intrusions.
基金This work was supported by the Hainan Provincial Natural Science Foundation of China[2018CXTD333,617048]National Natural Science Foundation of China[61762033,61702539]+1 种基金Hainan University Doctor Start Fund Project[kyqd1328]Hainan University Youth Fund Project[qnjj1444].
文摘Distributed Denial-of-Service(DDoS)has caused great damage to the network in the big data environment.Existing methods are characterized by low computational efficiency,high false alarm rate and high false alarm rate.In this paper,we propose a DDoS attack detection method based on network flow grayscale matrix feature via multi-scale convolutional neural network(CNN).According to the different characteristics of the attack flow and the normal flow in the IP protocol,the seven-tuple is defined to describe the network flow characteristics and converted into a grayscale feature by binary.Based on the network flow grayscale matrix feature(GMF),the convolution kernel of different spatial scales is used to improve the accuracy of feature segmentation,global features and local features of the network flow are extracted.A DDoS attack classifier based on multi-scale convolution neural network is constructed.Experiments show that compared with correlation methods,this method can improve the robustness of the classifier,reduce the false alarm rate and the missing alarm rate.
文摘Because of an explosive growth of the intrusions, necessity of anomaly-based Intrusion Detection Systems (IDSs) which are capable of detecting novel attacks, is increasing. Among those systems, flow-based detection systems which use a series of packets exchanged between two terminals as a unit of observation, have an advantage of being able to detect anomaly which is included in only some specific sessions. However, in large-scale networks where a large number of communications takes place, analyzing every flow is not practical. On the other hand, a timeslot-based detection systems need not to prepare a number of buffers although it is difficult to specify anomaly communications. In this paper, we propose a multi-stage anomaly detection system which is combination of timeslot-based and flow-based detectors. The proposed system can reduce the number of flows which need to be subjected to flow-based analysis but yet exhibits high detection accuracy. Through experiments using data set, we present the effectiveness of the proposed method.
文摘针对传统的基于自编码器的无监督异常声音检测方法存在特征表达能力不足的问题,提出一种基于注意力-跳跃自编码器-生成对抗网络的无监督异常声音检测方法ASAE-GAN(Attentional Skip-connected Auto Encoder and Generative Adversarial Network)。ASAE-GAN在跳跃自编码器和生成对抗网络的基础上,引入通道间注意力机制和时间注意力机制,增强模型的特征表达能力。使用MIMII数据集中的pump声音数据进行实验,评价指标使用AUC分数。结果表明:ASAE-GAN的平均AUC分数相比较于AE、UNET和Skip-GANomaly分别提升了16.27%、14.23%和6.55%,验证了其在无监督异常声音检测方面的优越性。
