A biased sampling algorithm for the restricted Boltzmann machine(RBM) is proposed, which allows generating configurations with a conserved quantity. To validate the method, a study of the short-range order in binary a...A biased sampling algorithm for the restricted Boltzmann machine(RBM) is proposed, which allows generating configurations with a conserved quantity. To validate the method, a study of the short-range order in binary alloys with positive and negative exchange interactions is carried out. The network is trained on the data collected by Monte–Carlo simulations for a simple Ising-like binary alloy model and used to calculate the Warren–Cowley short-range order parameter and other thermodynamic properties. We demonstrate that the proposed method allows us not only to correctly reproduce the order parameters for the alloy concentration at which the network was trained, but can also predict them for any other concentrations.展开更多
This paper proposes a new power grid investment prediction model based on the deep restricted Boltzmann machine(DRBM)optimized by the Lion algorithm(LA).Firstly,two factors including transmission and distribution pric...This paper proposes a new power grid investment prediction model based on the deep restricted Boltzmann machine(DRBM)optimized by the Lion algorithm(LA).Firstly,two factors including transmission and distribution price reform(TDPR)and 5G station construction were comprehensively incorporated into the consideration of influencing factors,and the fuzzy threshold method was used to screen out critical influencing factors.Then,the LA was used to optimize the parameters of the DRBM model to improve the model’s prediction accuracy,and the model was trained with the selected influencing factors and investment.Finally,the LA-DRBM model was used to predict the investment of a power grid enterprise,and the final prediction result was obtained by modifying the initial result with the modifying factors.The LA-DRBMmodel compensates for the deficiency of the singlemodel,and greatly improves the investment prediction accuracy of the power grid.In this study,a power grid enterprise was taken as an example to carry out an empirical analysis to prove the validity of the model,and a comparison with the RBM,support vector machine(SVM),back propagation neural network(BPNN),and regression model was conducted to verify the superiority of the model.The conclusion indicates that the proposed model has a strong generalization ability and good robustness,is able to abstract the combination of low-level features into high-level features,and can improve the efficiency of the model’s calculations for investment prediction of power grid enterprises.展开更多
This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learn...This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learning.Viewing it as a spin glass model and exhibiting various links with other models of statistical physics,we gather recent results dealing with mean-field theory in this context.First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM,leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns.Then we discuss recent works either able to devise mean-field based learning algorithms;either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.展开更多
In recent years,a laser-induced breakdown spectrometer(LIBS)combined with machine learning has been widely developed for steel classification.However,the much redundant information of LIBS spectra increases the comput...In recent years,a laser-induced breakdown spectrometer(LIBS)combined with machine learning has been widely developed for steel classification.However,the much redundant information of LIBS spectra increases the computation complexity for classification.In this work,restricted Boltzmann machines(RBM)and principal component analysis(PCA)were used for dimension reduction of datasets,respectively.Then,a support vector machine(SVM)was adopted to process feature information.Two models(RBM-SVM and PCA-SVM)are compared in terms of performance.After optimization,the accuracy of the RBM-SVM model can achieve 100%,and the maximum dimension reduction time is 33.18 s,which is nearly half of that of the PCA model(53.19 s).These results preliminarily indicate that LIBS combined with RBM-SVM has great potential in the real-time classification of steel.展开更多
Restricted Boltzmann machine(RBM)has been proposed as a powerful variational ansatz to represent the ground state of a given quantum many-body system.On the other hand,as a shallow neural network,it is found that the ...Restricted Boltzmann machine(RBM)has been proposed as a powerful variational ansatz to represent the ground state of a given quantum many-body system.On the other hand,as a shallow neural network,it is found that the RBM is still hardly able to capture the characteristics of systems with large sizes or complicated interactions.In order to find a way out of the dilemma,here,we propose to adopt the Green's function Monte Carlo(GFMC)method for which the RBM is used as a guiding wave function.To demonstrate the implementation and effectiveness of the proposal,we have applied the proposal to study the frustrated J_(1)-J_(2)Heisenberg model on a square lattice,which is considered as a typical model with sign problem for quantum Monte Carlo simulations.The calculation results demonstrate that the GFMC method can significantly further reduce the relative error of the ground-state energy on the basis of the RBM variational results.This encourages to combine the GFMC method with other neural networks like convolutional neural networks for dealing with more models with sign problem in the future.展开更多
Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture...Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.展开更多
Online monitoring and diagnosis of production processes face great challenges due to the nonlinearity and multivariate of complex industrial processes.Traditional process monitoring methods employ kernel function or m...Online monitoring and diagnosis of production processes face great challenges due to the nonlinearity and multivariate of complex industrial processes.Traditional process monitoring methods employ kernel function or multilayer neural networks to solve the nonlinear mapping problem of data.However,the above methods increase the model complexity and are not interpretable,leading to difficulties in subsequent fault recognition/diagnosis/location.A process monitoring and diagnosis method based on the free energy of Gaussian-Bernoulli restricted Boltzmann machine(GBRBM-FE)was proposed.Firstly,a GBRBM network was established to make the probability distribution of the reconstructed data as close as possible to the probability distribution of the raw data.On this basis,the weights and biases in GBRBM network were used to construct F statistics,which represents the free energy of the sample.The smaller the energy of the sample is,the more normal the sample is.Therefore,F statistics can be used to monitor the production process.To diagnose fault variables,the F statistic for each sample was decomposed to obtain the Fv statistic for each variable.By analyzing the deviation degree between the corresponding variables of abnormal samples and normal samples,the cause of process abnormalities can be accurately located.The application of converter steelmaking process demonstrates that the proposed method outperforms the traditional methods,in terms of fault monitoring and diagnosis performance.展开更多
The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-b...The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-based restricted Boltzmann machine(RBM) approach for CF and use the deep multilayer RBM network structure, which alleviates the data sparsity problem and has excellent ability to extract features. Each item is treated as a single RBM, and different items share the same weights and biases. The parameters are learned layer by layer in the deep network. The batch gradient descent algorithm with minibatch is used to increase the convergence speed. The new feature vector discovered by the multilayer RBM network structure is very effective in predicting a rating and achieves a better result. Experimental results on the data set of MovieL ens show that the item-based multilayer RBM approach achieves the best performance, with a mean absolute error of 0.6424 and a root-mean-square error of 0.7843.展开更多
Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificia...Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificial neural network （ANN）, restricted Boltzmann machines （RBM）, anddiscrete wavelet transform （DWT） is presented in this paper. In the proposed model, DWT firstdecomposes time series into approximation and detail. Then Khashei and Bijari＇s model, which is anensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their bothlinear and nonlinear components and fit the relationship between the components as a function insteadof additive relationship. Furthermore, RBM is used to perform pre-training for generating initialweights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detailare combined to obtain final forecasting. The forecasting capability of the proposed model is testedwith three well-known time series： sunspot, Canadian lynx, exchange rate time series. The predictionperformance is compared to the other six forecasting models. The results indicate that the proposedmodel gives the best performance in all three data sets and all three measures （i.e. MSE, MAE andMAPE）.展开更多
Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameter...Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there are no image features specially designed for thermal infrared images. In this paper, we propose using the deep Boltzmann machine to learn thermal features for emotion recognition from thermal infrared facial images. First, the face is located and normalized from the thermal infrared im- ages. Then, a deep Boltzmann machine model composed of two layers is trained. The parameters of the deep Boltzmann machine model are further fine-tuned for emotion recognition after pre-tralning of feature learning. Comparative experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, eye, and mouth are more effective for discriminating valence dimension of emotion than other facial areas. In addition, our study shows that adding unlabeled data from other database during training can also improve feature learning performance.展开更多
Identification and classification of DC faults are considered as fundamentals of DC grid protection.A sudden rise of DC fault current must be identified and classified to immediately operate the corresponding interrup...Identification and classification of DC faults are considered as fundamentals of DC grid protection.A sudden rise of DC fault current must be identified and classified to immediately operate the corresponding interrupting mechanism.In this paper,the Boltzmann machine learning(BML)approach is proposed for identification and classification of DC faults using travelling waves generated at fault point in voltage source converter based high-voltage direct current(VSC-HVDC)transmission system.An unsupervised way of feature extraction is performed on the frequency spectrum of the travelling waves.Binomial class logistic regression(BCLR)classifies the HVDC transmission system into faulty and healthy states.The proposed technique reduces the time for fault identification and classification because of reduced tagged data with few characteristics.Therefore,the faults near or at converter stations are readily identified and classified.The performance of the proposed technique is assessed via simulations developed in MATLAB/Simulink and tested for pre-fault and post-fault data both at VSC1 and VSC2,respectively.Moreover,the proposed technique is supported by analyzing the root mean square error to show practicality and realization with reduced computations.展开更多
Quantum machine learning based on quantum algorithms may achieve an exponential speedup over classical algorithms in dealing with some problems such as clustering.In this paper,we use the method of training the lower ...Quantum machine learning based on quantum algorithms may achieve an exponential speedup over classical algorithms in dealing with some problems such as clustering.In this paper,we use the method of training the lower bound of the average log likelihood function on the quantum Boltzmann machine(QBM)to recognize the handwritten number datasets and compare the training results with classical models.We find that,when the QBM is semi-restricted,the training results get better with fewer computing resources.This shows that it is necessary to design a targeted algorithm to speed up computation and save resources.展开更多
The original restricted Boltzmann machines(RBMs)are extended by replacing the binary visible and hidden variables with clusters of binary units,and a new learning algorithm for training deep Boltzmann machine of this ...The original restricted Boltzmann machines(RBMs)are extended by replacing the binary visible and hidden variables with clusters of binary units,and a new learning algorithm for training deep Boltzmann machine of this new variant is proposed.The sum of binary units of each cluster is approximated by a Gaussian distribution.Experiments demonstrate that the proposed Boltzmann machines can achieve good performance in the MNIST handwritten digital recognition task.展开更多
This paper focuses on the intrusion classification of huge amounts of data in a network intrusion detection system. An intrusion detection model based on deep belief nets （DBN） is proposed to conduct intrusion detec...This paper focuses on the intrusion classification of huge amounts of data in a network intrusion detection system. An intrusion detection model based on deep belief nets （DBN） is proposed to conduct intrusion detection,and the principles regarding DBN are discussed.The DBN is composed of a multiple unsupervised restricted Boltzmann machine （RBM） and a supervised back propagation （BP）network.First,the DBN in the proposed model is pre-trained in a fast and greedy way,and each RBM is trained by the contrastive divergence algorithm.Secondly,the whole network is fine-tuned by the supervised BP algorithm,which is employed for classifying the low-dimensional features of the intrusion data generated by the last RBM layer simultaneously.The experimental results on the KDD CUP 1999 dataset demonstrate that the DBN using the RBM network with three or more layers outperforms the self-organizing maps （SOM）and neural network （NN）in intrusion classification.Therefore,the DBN is an efficient approach for intrusion detection in high-dimensional space.展开更多
To improve the accuracy and anti-noise ability of the structural damage identification method,a bridge damage identification method is proposed based on a deep belief network(DBN).The output vector is used to establis...To improve the accuracy and anti-noise ability of the structural damage identification method,a bridge damage identification method is proposed based on a deep belief network(DBN).The output vector is used to establish the nonlinear mapping relationship between the mode shape and structural damage.The hidden layer of the DBN is trained through a layer-by-layer pre-training.Finally,the backpropagation algorithm is used to fine-tune the entire network.The method is validated using a numerical model of a steel truss bridge.The results show that under the influence of noise and modeling uncertainty,the damage identification method based on the DBN can identify the accurate damage location and degree identification compared with the traditional damage identification method based on an artificial neural network.展开更多
One of the severe health problems and the most common types of heartdisease (HD) is Coronary heart disease (CHD). Due to the lack of a healthy lifestyle, HD would cause frequent mortality worldwide. If the heart atta...One of the severe health problems and the most common types of heartdisease (HD) is Coronary heart disease (CHD). Due to the lack of a healthy lifestyle, HD would cause frequent mortality worldwide. If the heart attack occurswithout any symptoms, it cannot be cured by an intelligent detection system.An effective diagnosis and detection of CHD should prevent human casualties.Moreover, intelligent systems employ clinical-based decision support approachesto assist physicians in providing another option for diagnosing and detecting HD.This paper aims to introduce a heart disease prediction model including phaseslike (i) Feature extraction, (ii) Feature selection, and (iii) Classification. At first,the feature extraction process is carried out, where the features like a time-domainindex, frequency-domain index, geometrical domain features, nonlinear features,WT features, signal energy, skewness, entropy, kurtosis features are extractedfrom the input ECG signal. The curse of dimensionality becomes a severe issue.This paper provides the solution for this issue by introducing a new ModifiedPrincipal Component Analysis known as Multiple Kernel-based PCA for dimensionality reduction. Furthermore, the dimensionally reduced feature set is thensubjected to a classification process, where the hybrid classifier combining bothRecurrent Neural Network (RNN) and Restricted Boltzmann Machine (RBM)is used. At last, the performance analysis of the adopted scheme is compared overother existing schemes in terms of specific measures.展开更多
Model performance assessment is a key procedure for mineral potential mapping, but the correspond-ing research achievements are seldom reported in literature. Cumulative gain and lift charts are well known in the data...Model performance assessment is a key procedure for mineral potential mapping, but the correspond-ing research achievements are seldom reported in literature. Cumulative gain and lift charts are well known in the data mining community specialized in marketing and sales applications and widely used in customer chum prediction for model performance assessment. In this paper, they are introduced into the field of mineral poten-tial mapping for model performance assessment. These two charts can be viewed as a graphic representation of the advantage of using a predictive model to choose mineral targets. A cumulative gain curve can represent how much a predictive model is superior to a random guess in mineral target prediction. A lift chart can express how much more likely the mineral targets predicted by a model are deposit-bearing ones than those by a random se-lection. As an illustration, the cumulative gain and lift charts are applied to measure the performance of weights of evidence, logistic regression,restricted Boltzmann machine, and multilayer perceptron in mineral potential mapping in the Altay district in northern Xinjiang in China. The results show that the cumulative gain and lift charts can visually reveal that the first three models perform well while the last one performs poorly. Thus, the cumulative gain and lift charts can serve as a graphic tool for model performance assessment in mineral potential mapping.展开更多
This paper presents the mathematical model of mixed integer bilevel programming problem. The properties of the problem are discussed. An algorithm which combines the Boltzmann machine and two-level iterative computat...This paper presents the mathematical model of mixed integer bilevel programming problem. The properties of the problem are discussed. An algorithm which combines the Boltzmann machine and two-level iterative computation is also proposed to solve it. It has been shown by the initial simulated results that the algorithm is efficient.展开更多
Vehicle velocity forecast is an important clue in improving the performance of energy management in hybrid electric vehicles(HEV). This paper presents a new combined model for predicting vehicle’s velocity time serie...Vehicle velocity forecast is an important clue in improving the performance of energy management in hybrid electric vehicles(HEV). This paper presents a new combined model for predicting vehicle’s velocity time series. The main features of the model are to combine the feature extraction capability of deep restricted Boltzmann machines(DBM) and sequence pattern predicting capability of bidirectional long short-term memory(BLSTM). Hence, the model is named as DBMBLSTM. In addition, the DRMBLSTM model utilizes the vehicle driving information and roadside infrastructure information provided respectively through vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communication channels to predict vehicle velocity at various length of prediction horizon. Furthermore, the predictions results of this study are compared with the state of the art of vehicle velocity forecasts. The root mean square error(RMSE) is used as an evaluation criteria of predictions accuracy. Finally,these compared prediction model are applied in model predictive control(MPC) energy management strategy for the verifications of fuel economy improvement of a HEV. Simulation results confirm that the proposed combined deep learning model performs better than other five prediction methods. Therefore, it is a means of arriving at a reliable forecast model for HEV.展开更多
Purpose–Many strategies have been put forward for training deep network models,however,stacking of several layers of non-linearities typically results in poor propagation of gradients and activations.The purpose of t...Purpose–Many strategies have been put forward for training deep network models,however,stacking of several layers of non-linearities typically results in poor propagation of gradients and activations.The purpose of this paper is to explore the use of two steps strategy where initial deep learning model is obtained first by unsupervised learning and then optimizing the initial deep learning model by fine tuning.A number of fine tuning algorithms are explored in this work for optimizing deep learning models.This includes proposing a new algorithm where Backpropagation with adaptive gain algorithm is integrated with Dropout technique and the authors evaluate its performance in the fine tuning of the pretrained deep network.Design/methodology/approach–The parameters of deep neural networks are first learnt using greedy layer-wise unsupervised pretraining.The proposed technique is then used to perform supervised fine tuning of the deep neural network model.Extensive experimental study is performed to evaluate the performance of the proposed fine tuning technique on three benchmark data sets:USPS,Gisette and MNIST.The authors have tested the approach on varying size data sets which include randomly chosen training samples of size 20,50,70 and 100 percent from the original data set.Findings–Through extensive experimental study,it is concluded that the two steps strategy and the proposed fine tuning technique significantly yield promising results in optimization of deep network models.Originality/value–This paper proposes employing several algorithms for fine tuning of deep network model.A new approach that integrates adaptive gain Backpropagation(BP)algorithm with Dropout technique is proposed for fine tuning of deep networks.Evaluation and comparison of various algorithms proposed for fine tuning on three benchmark data sets is presented in the paper.展开更多
基金supported by the financing program AAAA-A16-116021010082-8。
文摘A biased sampling algorithm for the restricted Boltzmann machine(RBM) is proposed, which allows generating configurations with a conserved quantity. To validate the method, a study of the short-range order in binary alloys with positive and negative exchange interactions is carried out. The network is trained on the data collected by Monte–Carlo simulations for a simple Ising-like binary alloy model and used to calculate the Warren–Cowley short-range order parameter and other thermodynamic properties. We demonstrate that the proposed method allows us not only to correctly reproduce the order parameters for the alloy concentration at which the network was trained, but can also predict them for any other concentrations.
基金the National Key Research and Development Program of China(Grant No.2020YFB1707804)the 2018 Key Projects of Philosophy and Social Sciences Research(Grant No.18JZD032)Natural Science Foundation of Hebei Province(Grant No.G2020403008).
文摘This paper proposes a new power grid investment prediction model based on the deep restricted Boltzmann machine(DRBM)optimized by the Lion algorithm(LA).Firstly,two factors including transmission and distribution price reform(TDPR)and 5G station construction were comprehensively incorporated into the consideration of influencing factors,and the fuzzy threshold method was used to screen out critical influencing factors.Then,the LA was used to optimize the parameters of the DRBM model to improve the model’s prediction accuracy,and the model was trained with the selected influencing factors and investment.Finally,the LA-DRBM model was used to predict the investment of a power grid enterprise,and the final prediction result was obtained by modifying the initial result with the modifying factors.The LA-DRBMmodel compensates for the deficiency of the singlemodel,and greatly improves the investment prediction accuracy of the power grid.In this study,a power grid enterprise was taken as an example to carry out an empirical analysis to prove the validity of the model,and a comparison with the RBM,support vector machine(SVM),back propagation neural network(BPNN),and regression model was conducted to verify the superiority of the model.The conclusion indicates that the proposed model has a strong generalization ability and good robustness,is able to abstract the combination of low-level features into high-level features,and can improve the efficiency of the model’s calculations for investment prediction of power grid enterprises.
基金supported by the Comunidad de Madrid and the Complutense University of Madrid (Spain) through the Atracción de Talento program (Ref. 2019-T1/TIC-13298)
文摘This review deals with restricted Boltzmann machine(RBM) under the light of statistical physics.The RBM is a classical family of machine learning(ML) models which played a central role in the development of deep learning.Viewing it as a spin glass model and exhibiting various links with other models of statistical physics,we gather recent results dealing with mean-field theory in this context.First the functioning of the RBM can be analyzed via the phase diagrams obtained for various statistical ensembles of RBM,leading in particular to identify a compositional phase where a small number of features or modes are combined to form complex patterns.Then we discuss recent works either able to devise mean-field based learning algorithms;either able to reproduce generic aspects of the learning process from some ensemble dynamics equations or/and from linear stability arguments.
基金supported by National Natural Science Foundation of China(No.61705064)the Natural Science Foundation of Hubei Province(No.2021CFB607)+1 种基金the Natural Science Foundation of Xiaogan City(No.XGKJ2021010003)the Project of the Hubei Provincial Department of Education(No.T201617)。
文摘In recent years,a laser-induced breakdown spectrometer(LIBS)combined with machine learning has been widely developed for steel classification.However,the much redundant information of LIBS spectra increases the computation complexity for classification.In this work,restricted Boltzmann machines(RBM)and principal component analysis(PCA)were used for dimension reduction of datasets,respectively.Then,a support vector machine(SVM)was adopted to process feature information.Two models(RBM-SVM and PCA-SVM)are compared in terms of performance.After optimization,the accuracy of the RBM-SVM model can achieve 100%,and the maximum dimension reduction time is 33.18 s,which is nearly half of that of the PCA model(53.19 s).These results preliminarily indicate that LIBS combined with RBM-SVM has great potential in the real-time classification of steel.
基金Project supported by the National Natural Science Foundation of China(Grant Nos.11934020 and 11874421)the Natural Science Foundation of Beijing(Grant No.Z180013)。
文摘Restricted Boltzmann machine(RBM)has been proposed as a powerful variational ansatz to represent the ground state of a given quantum many-body system.On the other hand,as a shallow neural network,it is found that the RBM is still hardly able to capture the characteristics of systems with large sizes or complicated interactions.In order to find a way out of the dilemma,here,we propose to adopt the Green's function Monte Carlo(GFMC)method for which the RBM is used as a guiding wave function.To demonstrate the implementation and effectiveness of the proposal,we have applied the proposal to study the frustrated J_(1)-J_(2)Heisenberg model on a square lattice,which is considered as a typical model with sign problem for quantum Monte Carlo simulations.The calculation results demonstrate that the GFMC method can significantly further reduce the relative error of the ground-state energy on the basis of the RBM variational results.This encourages to combine the GFMC method with other neural networks like convolutional neural networks for dealing with more models with sign problem in the future.
文摘Restricted Boltzmann Machines (RBMs) are an effective model for machine learning;however, they require a significant amount of processing time. In this study, we propose a highly parallel, highly flexible architecture that combines small and completely parallel RBMs. This proposal addresses problems associated with calculation speed and exponential increases in circuit scale. We show that this architecture can optionally respond to the trade-offs between these two problems. Furthermore, our FPGA implementation performs at a 134 times processing speed up factor with respect to a conventional CPU.
基金the financial support from the National Key R&D Program of China(Grant No.2020YFA0405700).
文摘Online monitoring and diagnosis of production processes face great challenges due to the nonlinearity and multivariate of complex industrial processes.Traditional process monitoring methods employ kernel function or multilayer neural networks to solve the nonlinear mapping problem of data.However,the above methods increase the model complexity and are not interpretable,leading to difficulties in subsequent fault recognition/diagnosis/location.A process monitoring and diagnosis method based on the free energy of Gaussian-Bernoulli restricted Boltzmann machine(GBRBM-FE)was proposed.Firstly,a GBRBM network was established to make the probability distribution of the reconstructed data as close as possible to the probability distribution of the raw data.On this basis,the weights and biases in GBRBM network were used to construct F statistics,which represents the free energy of the sample.The smaller the energy of the sample is,the more normal the sample is.Therefore,F statistics can be used to monitor the production process.To diagnose fault variables,the F statistic for each sample was decomposed to obtain the Fv statistic for each variable.By analyzing the deviation degree between the corresponding variables of abnormal samples and normal samples,the cause of process abnormalities can be accurately located.The application of converter steelmaking process demonstrates that the proposed method outperforms the traditional methods,in terms of fault monitoring and diagnosis performance.
基金Project supported by the National Science and Technology Suppor Plan(No.2013BAH21B02-01)the Beijing Natural Science Foundation(No.4153058)
文摘The collaborative filtering(CF) technique has been widely used recently in recommendation systems. It needs historical data to give predictions. However, the data sparsity problem still exists. We propose a new item-based restricted Boltzmann machine(RBM) approach for CF and use the deep multilayer RBM network structure, which alleviates the data sparsity problem and has excellent ability to extract features. Each item is treated as a single RBM, and different items share the same weights and biases. The parameters are learned layer by layer in the deep network. The batch gradient descent algorithm with minibatch is used to increase the convergence speed. The new feature vector discovered by the multilayer RBM network structure is very effective in predicting a rating and achieves a better result. Experimental results on the data set of MovieL ens show that the item-based multilayer RBM approach achieves the best performance, with a mean absolute error of 0.6424 and a root-mean-square error of 0.7843.
文摘Time series forecasting research area mainly focuses on developing effective forecasting models toimprove prediction accuracy. An ensemble model composed of autoregressive integrated movingaverage （ARIMA）, artificial neural network （ANN）, restricted Boltzmann machines （RBM）, anddiscrete wavelet transform （DWT） is presented in this paper. In the proposed model, DWT firstdecomposes time series into approximation and detail. Then Khashei and Bijari＇s model, which is anensemble model of ARIMA and ANN, is applied to the approximation and detail to extract their bothlinear and nonlinear components and fit the relationship between the components as a function insteadof additive relationship. Furthermore, RBM is used to perform pre-training for generating initialweights and biases based on inputs feature for ANN. Finally, the forecasted approximation and detailare combined to obtain final forecasting. The forecasting capability of the proposed model is testedwith three well-known time series： sunspot, Canadian lynx, exchange rate time series. The predictionperformance is compared to the other six forecasting models. The results indicate that the proposedmodel gives the best performance in all three data sets and all three measures （i.e. MSE, MAE andMAPE）.
基金This paper was supported by the National Natural Science Foundation of China （Grant Nos. 61175037, 61228304）, Special Innovation Project on Speech of Anhui Province （11010202192）, Project from Anhui Science and Technology Agency （1106c0805008） and the Fundamental Research Funds for the Central Universities. We also acknowledge partial support from the US National Science Foundation （1205664）.
文摘Facial expression and emotion recognition from thermal infrared images has attracted more and more attentions in recent years. However, the features adopted in current work are either temperature statistical parameters extracted from the facial regions of interest or several hand-crafted features that are commonly used in visible spectrum. Till now there are no image features specially designed for thermal infrared images. In this paper, we propose using the deep Boltzmann machine to learn thermal features for emotion recognition from thermal infrared facial images. First, the face is located and normalized from the thermal infrared im- ages. Then, a deep Boltzmann machine model composed of two layers is trained. The parameters of the deep Boltzmann machine model are further fine-tuned for emotion recognition after pre-tralning of feature learning. Comparative experimental results on the NVIE database demonstrate that our approach outperforms other approaches using temperature statistic features or hand-crafted features borrowed from visible domain. The learned features from the forehead, eye, and mouth are more effective for discriminating valence dimension of emotion than other facial areas. In addition, our study shows that adding unlabeled data from other database during training can also improve feature learning performance.
文摘Identification and classification of DC faults are considered as fundamentals of DC grid protection.A sudden rise of DC fault current must be identified and classified to immediately operate the corresponding interrupting mechanism.In this paper,the Boltzmann machine learning(BML)approach is proposed for identification and classification of DC faults using travelling waves generated at fault point in voltage source converter based high-voltage direct current(VSC-HVDC)transmission system.An unsupervised way of feature extraction is performed on the frequency spectrum of the travelling waves.Binomial class logistic regression(BCLR)classifies the HVDC transmission system into faulty and healthy states.The proposed technique reduces the time for fault identification and classification because of reduced tagged data with few characteristics.Therefore,the faults near or at converter stations are readily identified and classified.The performance of the proposed technique is assessed via simulations developed in MATLAB/Simulink and tested for pre-fault and post-fault data both at VSC1 and VSC2,respectively.Moreover,the proposed technique is supported by analyzing the root mean square error to show practicality and realization with reduced computations.
基金supported by the National Natural Science Foundation of China under Grant No.11725524the Hubei Provincal Natural Science Foundation of China under Grant No.2019CFA003
文摘Quantum machine learning based on quantum algorithms may achieve an exponential speedup over classical algorithms in dealing with some problems such as clustering.In this paper,we use the method of training the lower bound of the average log likelihood function on the quantum Boltzmann machine(QBM)to recognize the handwritten number datasets and compare the training results with classical models.We find that,when the QBM is semi-restricted,the training results get better with fewer computing resources.This shows that it is necessary to design a targeted algorithm to speed up computation and save resources.
文摘The original restricted Boltzmann machines(RBMs)are extended by replacing the binary visible and hidden variables with clusters of binary units,and a new learning algorithm for training deep Boltzmann machine of this new variant is proposed.The sum of binary units of each cluster is approximated by a Gaussian distribution.Experiments demonstrate that the proposed Boltzmann machines can achieve good performance in the MNIST handwritten digital recognition task.
基金The National Key Technology R&D Program during the 12th Five-Year Plan Period(No.2013BAK01B02)the National Natural Science Foundation of China(No.61373176)the Scientific Research Projects of Shaanxi Educational Committee(No.14JK1693)
文摘This paper focuses on the intrusion classification of huge amounts of data in a network intrusion detection system. An intrusion detection model based on deep belief nets （DBN） is proposed to conduct intrusion detection,and the principles regarding DBN are discussed.The DBN is composed of a multiple unsupervised restricted Boltzmann machine （RBM） and a supervised back propagation （BP）network.First,the DBN in the proposed model is pre-trained in a fast and greedy way,and each RBM is trained by the contrastive divergence algorithm.Secondly,the whole network is fine-tuned by the supervised BP algorithm,which is employed for classifying the low-dimensional features of the intrusion data generated by the last RBM layer simultaneously.The experimental results on the KDD CUP 1999 dataset demonstrate that the DBN using the RBM network with three or more layers outperforms the self-organizing maps （SOM）and neural network （NN）in intrusion classification.Therefore,the DBN is an efficient approach for intrusion detection in high-dimensional space.
基金The National Natural Science Foundation of China(No.51378104)。
文摘To improve the accuracy and anti-noise ability of the structural damage identification method,a bridge damage identification method is proposed based on a deep belief network(DBN).The output vector is used to establish the nonlinear mapping relationship between the mode shape and structural damage.The hidden layer of the DBN is trained through a layer-by-layer pre-training.Finally,the backpropagation algorithm is used to fine-tune the entire network.The method is validated using a numerical model of a steel truss bridge.The results show that under the influence of noise and modeling uncertainty,the damage identification method based on the DBN can identify the accurate damage location and degree identification compared with the traditional damage identification method based on an artificial neural network.
文摘One of the severe health problems and the most common types of heartdisease (HD) is Coronary heart disease (CHD). Due to the lack of a healthy lifestyle, HD would cause frequent mortality worldwide. If the heart attack occurswithout any symptoms, it cannot be cured by an intelligent detection system.An effective diagnosis and detection of CHD should prevent human casualties.Moreover, intelligent systems employ clinical-based decision support approachesto assist physicians in providing another option for diagnosing and detecting HD.This paper aims to introduce a heart disease prediction model including phaseslike (i) Feature extraction, (ii) Feature selection, and (iii) Classification. At first,the feature extraction process is carried out, where the features like a time-domainindex, frequency-domain index, geometrical domain features, nonlinear features,WT features, signal energy, skewness, entropy, kurtosis features are extractedfrom the input ECG signal. The curse of dimensionality becomes a severe issue.This paper provides the solution for this issue by introducing a new ModifiedPrincipal Component Analysis known as Multiple Kernel-based PCA for dimensionality reduction. Furthermore, the dimensionally reduced feature set is thensubjected to a classification process, where the hybrid classifier combining bothRecurrent Neural Network (RNN) and Restricted Boltzmann Machine (RBM)is used. At last, the performance analysis of the adopted scheme is compared overother existing schemes in terms of specific measures.
基金Supported by Project of the National Natural Science Foundation of China(Nos.41272360,41472299,61133011)
文摘Model performance assessment is a key procedure for mineral potential mapping, but the correspond-ing research achievements are seldom reported in literature. Cumulative gain and lift charts are well known in the data mining community specialized in marketing and sales applications and widely used in customer chum prediction for model performance assessment. In this paper, they are introduced into the field of mineral poten-tial mapping for model performance assessment. These two charts can be viewed as a graphic representation of the advantage of using a predictive model to choose mineral targets. A cumulative gain curve can represent how much a predictive model is superior to a random guess in mineral target prediction. A lift chart can express how much more likely the mineral targets predicted by a model are deposit-bearing ones than those by a random se-lection. As an illustration, the cumulative gain and lift charts are applied to measure the performance of weights of evidence, logistic regression,restricted Boltzmann machine, and multilayer perceptron in mineral potential mapping in the Altay district in northern Xinjiang in China. The results show that the cumulative gain and lift charts can visually reveal that the first three models perform well while the last one performs poorly. Thus, the cumulative gain and lift charts can serve as a graphic tool for model performance assessment in mineral potential mapping.
文摘This paper presents the mathematical model of mixed integer bilevel programming problem. The properties of the problem are discussed. An algorithm which combines the Boltzmann machine and two-level iterative computation is also proposed to solve it. It has been shown by the initial simulated results that the algorithm is efficient.
基金supported by the National Natural Science Foundation of China(Grant No.61703318)Natural Science Foundation of Hubei Province(Grant No.2017CFB130)
文摘Vehicle velocity forecast is an important clue in improving the performance of energy management in hybrid electric vehicles(HEV). This paper presents a new combined model for predicting vehicle’s velocity time series. The main features of the model are to combine the feature extraction capability of deep restricted Boltzmann machines(DBM) and sequence pattern predicting capability of bidirectional long short-term memory(BLSTM). Hence, the model is named as DBMBLSTM. In addition, the DRMBLSTM model utilizes the vehicle driving information and roadside infrastructure information provided respectively through vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communication channels to predict vehicle velocity at various length of prediction horizon. Furthermore, the predictions results of this study are compared with the state of the art of vehicle velocity forecasts. The root mean square error(RMSE) is used as an evaluation criteria of predictions accuracy. Finally,these compared prediction model are applied in model predictive control(MPC) energy management strategy for the verifications of fuel economy improvement of a HEV. Simulation results confirm that the proposed combined deep learning model performs better than other five prediction methods. Therefore, it is a means of arriving at a reliable forecast model for HEV.
文摘Purpose–Many strategies have been put forward for training deep network models,however,stacking of several layers of non-linearities typically results in poor propagation of gradients and activations.The purpose of this paper is to explore the use of two steps strategy where initial deep learning model is obtained first by unsupervised learning and then optimizing the initial deep learning model by fine tuning.A number of fine tuning algorithms are explored in this work for optimizing deep learning models.This includes proposing a new algorithm where Backpropagation with adaptive gain algorithm is integrated with Dropout technique and the authors evaluate its performance in the fine tuning of the pretrained deep network.Design/methodology/approach–The parameters of deep neural networks are first learnt using greedy layer-wise unsupervised pretraining.The proposed technique is then used to perform supervised fine tuning of the deep neural network model.Extensive experimental study is performed to evaluate the performance of the proposed fine tuning technique on three benchmark data sets:USPS,Gisette and MNIST.The authors have tested the approach on varying size data sets which include randomly chosen training samples of size 20,50,70 and 100 percent from the original data set.Findings–Through extensive experimental study,it is concluded that the two steps strategy and the proposed fine tuning technique significantly yield promising results in optimization of deep network models.Originality/value–This paper proposes employing several algorithms for fine tuning of deep network model.A new approach that integrates adaptive gain Backpropagation(BP)algorithm with Dropout technique is proposed for fine tuning of deep networks.Evaluation and comparison of various algorithms proposed for fine tuning on three benchmark data sets is presented in the paper.
