A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively ...A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.展开更多
There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement an...There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.展开更多
To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with...To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.展开更多
A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a force...A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.展开更多
In today’s world, there are many people suffering from mentalhealth problems such as depression and anxiety. If these conditions are notidentified and treated early, they can get worse quickly and have far-reachingne...In today’s world, there are many people suffering from mentalhealth problems such as depression and anxiety. If these conditions are notidentified and treated early, they can get worse quickly and have far-reachingnegative effects. Unfortunately, many people suffering from these conditions,especially depression and hypertension, are unaware of their existence until theconditions become chronic. Thus, this paper proposes a novel approach usingBi-directional Long Short-Term Memory (Bi-LSTM) algorithm and GlobalVector (GloVe) algorithm for the prediction and treatment of these conditions.Smartwatches and fitness bands can be equipped with these algorithms whichcan share data with a variety of IoT devices and smart systems to betterunderstand and analyze the user’s condition. We compared the accuracy andloss of the training dataset and the validation dataset of the two modelsnamely, Bi-LSTM without a global vector layer and with a global vector layer.It was observed that the model of Bi-LSTM without a global vector layer hadan accuracy of 83%,while Bi-LSTMwith a global vector layer had an accuracyof 86% with a precision of 86.4%, and an F1 score of 0.861. In addition toproviding basic therapies for the treatment of identified cases, our model alsohelps prevent the deterioration of associated conditions, making our methoda real-world solution.展开更多
Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on w...Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.展开更多
In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is es...In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.展开更多
In this paper,the recurrent neural network structure of a bidirectional long shortterm memory network(Bi-LSTM)with special memory cells that store information is used to characterize the deep features of the variation...In this paper,the recurrent neural network structure of a bidirectional long shortterm memory network(Bi-LSTM)with special memory cells that store information is used to characterize the deep features of the variation pattern between logging and seismic data.A mapping relationship model between high-frequency logging data and low-frequency seismic data is established via nonlinear mapping.The seismic waveform is infinitely approximated using the logging curve in the low-frequency band to obtain a nonlinear mapping model of this scale,which then stepwise approach the logging curve in the high-frequency band.Finally,a seismic-inversion method of nonlinear mapping multilevel well–seismic matching based on the Bi-LSTM network is developed.The characteristic of this method is that by applying the multilevel well–seismic matching process,the seismic data are stepwise matched to the scale range that is consistent with the logging curve.Further,the matching operator at each level can be stably obtained to effectively overcome the problems that occur in the well–seismic matching process,such as the inconsistency in the scale of two types of data,accuracy in extracting the seismic wavelet of the well-side seismic traces,and multiplicity of solutions.Model test and practical application demonstrate that this method improves the vertical resolution of inversion results,and at the same time,the boundary and the lateral characteristics of the sand body are well maintained to improve the accuracy of thin-layer sand body prediction and achieve an improved practical application effect.展开更多
Purpose-To optimize train operations,dispatchers currently rely on experience for quick adjustments when delays occur.However,delay predictions often involve imprecise shifts based on known delay times.Real-time and a...Purpose-To optimize train operations,dispatchers currently rely on experience for quick adjustments when delays occur.However,delay predictions often involve imprecise shifts based on known delay times.Real-time and accurate train delay predictions,facilitated by data-driven neural network models,can significantly reduce dispatcher stress and improve adjustment plans.Leveraging current train operation data,these models enable swift and precise predictions,addressing challenges posed by train delays in high-speed rail networks during unforeseen events.Design/methodology/approach-This paper proposes CBLA-net,a neural network architecture for predicting late arrival times.It combines CNN,Bi-LSTM,and attention mechanisms to extract features,handle time series data,and enhance information utilization.Trained on operational data from the Beijing-Tianjin line,it predicts the late arrival time of a target train at the next station using multidimensional input data from the target and preceding trains.Findings-This study evaluates our model’s predictive performance using two data approaches:one considering full data and another focusing only on late arrivals.Results show precise and rapid predictions.Training with full data achieves aMAEof approximately 0.54 minutes and a RMSEof 0.65 minutes,surpassing the model trained solely on delay data(MAE:is about 1.02 min,RMSE:is about 1.52 min).Despite superior overall performance with full data,the model excels at predicting delays exceeding 15 minutes when trained exclusively on late arrivals.For enhanced adaptability to real-world train operations,training with full data is recommended.Originality/value-This paper introduces a novel neural network model,CBLA-net,for predicting train delay times.It innovatively compares and analyzes the model’s performance using both full data and delay data formats.Additionally,the evaluation of the network’s predictive capabilities considers different scenarios,providing a comprehensive demonstration of the model’s predictive performance.展开更多
Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural ...Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural networks for rainfall-runoff modeling in the Zou River basin at Atchérigbé outlet. To this end, we used daily precipitation data over the period 1988-2010 as input of the models, such as the Long Short-Term Memory (LSTM) and Recurrent Gate Networks (GRU) to simulate river discharge in the study area. The investigated models give good results in calibration (R2 = 0.888, NSE = 0.886, and RMSE = 0.42 for LSTM;R2 = 0.9, NSE = 0.9 and RMSE = 0.397 for GRU) and in validation (R2 = 0.865, NSE = 0.851, and RMSE = 0.329 for LSTM;R2 = 0.9, NSE = 0.865 and RMSE = 0.301 for GRU). This good performance of LSTM and GRU models confirms the importance of models based on machine learning in modeling hydrological phenomena for better decision-making.展开更多
Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning netwo...Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.展开更多
The healthy condition of the milling tool has a very high impact on the machining quality of the titanium components.Therefore,it is important to recognize the healthy condition of the tool and replace the damaged cut...The healthy condition of the milling tool has a very high impact on the machining quality of the titanium components.Therefore,it is important to recognize the healthy condition of the tool and replace the damaged cutter at the right time.In order to recognize the health condition of the milling cutter,a method based on the long short term memory(LSTM)was proposed to recognize tool health state in this paper.The various signals collected in the tool wear experiments were analyzed by time-domain statistics,and then the extracted data were generated by principal component analysis(PCA)method.The preprocessed data extracted by PCA is transmitted to the LSTM model for recognition.Compared with back propagation neural network(BPNN)and support vector machine(SVM),the proposed method can effectively utilize the time-domain regulation in the data to achieve higher recognition speed and accuracy.展开更多
In terms of the modular fuzzy neural network (MFNN) combining fuzzy c-mean (FCM) cluster and single-layer neural network, a short-term climate prediction model is developed. It is found from modeling results that the ...In terms of the modular fuzzy neural network (MFNN) combining fuzzy c-mean (FCM) cluster and single-layer neural network, a short-term climate prediction model is developed. It is found from modeling results that the MFNN model for short-term climate prediction has advantages of simple structure, no hidden layer and stable network parameters because of the assembling of sound functions of the self-adaptive learning, association and fuzzy information processing of fuzzy mathematics and neural network methods. The case computational results of Guangxi flood season (JJA) rainfall show that the mean absolute error (MAE) and mean relative error (MRE) of the prediction during 1998-2002 are 68.8 mm and 9.78%, and in comparison with the regression method, under the conditions of the same predictors and period they are 97.8 mm and 12.28% respectively. Furthermore, it is also found from the stability analysis of the modular model that the change of the prediction results of independent samples with training times in the stably convergent interval of the model is less than 1.3 mm. The obvious oscillation phenomenon of prediction results with training times, such as in the common back-propagation neural network (BPNN) model, does not occur, indicating a better practical application potential of the MFNN model.展开更多
BACKGROUND Because of the powerful abilities of self-learning and handling complex biological information,artificial neural network(ANN)models have been widely applied to disease diagnosis,imaging analysis,and prognos...BACKGROUND Because of the powerful abilities of self-learning and handling complex biological information,artificial neural network(ANN)models have been widely applied to disease diagnosis,imaging analysis,and prognosis prediction.However,there has been no trained preoperative ANN(preope-ANN)model to preoperatively predict the prognosis of patients with gastric cancer(GC).AIM To establish a neural network model that can predict long-term survival of GC patients before surgery to evaluate the tumor condition before the operation.METHODS The clinicopathological data of 1608 GC patients treated from January 2011 to April 2015 at the Department of Gastric Surgery,Fujian Medical University Union Hospital were analyzed retrospectively.The patients were randomly divided into a training set(70%)for establishing a preope-ANN model and a testing set(30%).The prognostic evaluation ability of the preope-ANN model was compared with that of the American Joint Commission on Cancer(8th edition)clinical TNM(cTNM)and pathological TNM(pTNM)staging through the receiver operating characteristic curve,Akaike information criterion index,Harrell's C index,and likelihood ratio chi-square.RESULTS We used the variables that were statistically significant factors for the 3-year overall survival as input-layer variables to develop a preope-ANN in the training set.The survival curves within each score of the preope-ANN had good discrimination(P<0.05).Comparing the preope-ANN model,cTNM,and pTNM in both the training and testing sets,the preope-ANN model was superior to cTNM in predictive discrimination(C index),predictive homogeneity(likelihood ratio chi-square),and prediction accuracy(area under the curve).The prediction efficiency of the preope-ANN model is similar to that of pTNM.CONCLUSION The preope-ANN model can accurately predict the long-term survival of GC patients,and its predictive efficiency is not inferior to that of pTNM stage.展开更多
Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial featur...Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial features among the consecutive speech frames become highly correlated such that it is helpful for speaker separation by providing additional spatial information.To fully exploit this information,we design a separation system on Recurrent Neural Network(RNN)with long short-term memory(LSTM)which effectively learns the temporal dynamics of spatial features.In detail,a LSTM-based speaker separation algorithm is proposed to extract the spatial features in each time-frequency(TF)unit and form the corresponding feature vector.Then,we treat speaker separation as a supervised learning problem,where a modified ideal ratio mask(IRM)is defined as the training function during LSTM learning.Simulations show that the proposed system achieves attractive separation performance in noisy and reverberant environments.Specifically,during the untrained acoustic test with limited priors,e.g.,unmatched signal to noise ratio(SNR)and reverberation,the proposed LSTM based algorithm can still outperforms the existing DNN based method in the measures of PESQ and STOI.It indicates our method is more robust in untrained conditions.展开更多
In recent years, introduction of a renewable energy source such as solar energy is expected. However, solar radiation is not constant and power output of photovoltaic (PV) system is influenced by weather conditions. I...In recent years, introduction of a renewable energy source such as solar energy is expected. However, solar radiation is not constant and power output of photovoltaic (PV) system is influenced by weather conditions. It is difficult for getting to know accurate power output of PV system. In order to forecast the power output of PV system as accurate as possible, this paper proposes a decision technique of forecasting model for short-term-ahead power output of PV system based on solar radiation prediction. Application of Recurrent Neural Network (RNN) is shown for solar radiation prediction in this paper. The proposed method in this paper does not require complicated calculation, but mathematical model with only useful weather data. The validity of the proposed RNN is confirmed by comparing simulation results of solar radiation forecasting with that obtained from other展开更多
The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process par...The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.展开更多
Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. ...Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. the deep remembrance for the initial impression.. The eternal term memory vector is determined only by the initial vector that establishes category neuron node and is used to keep the remembrance for this vector for ever. Two times of vigilance algorithm are put forward, and the posterior input vector must first pass the first vigilance of this eternal term memory vector, only succeeded has it the qualification to begin the second vigilance of long term memory vector. The long term memory vector can be revised only when both of the vigilances are passed. Results of recognition examples show that the improved ART-2 overcomes the defect of traditional ART-2 and can recognize a gradually changing course effectively.展开更多
The fuzzy neural network is applied to the short-term load forecasting. The fuzzy rules and fuzzy membership functions of the network are obtained through fuzzy neural network learming. Three inference algorithms, i.e...The fuzzy neural network is applied to the short-term load forecasting. The fuzzy rules and fuzzy membership functions of the network are obtained through fuzzy neural network learming. Three inference algorithms, i.e. themultiplicative inference, the maximum inference and the minimum inference, are used for comparison. The learningalgorithms corresponding to the inference methods are derived from back-propagation algorithm. To validate the fuzzyneural network model, the network is used to Predict short-term load by compaing the network output against the realload data from a local power system supplying electricity to a large steel manufacturer. The experimental results aresatisfactory.展开更多
文摘A correct and timely fault diagnosis is important for improving the safety and reliability of chemical processes. With the advancement of big data technology, data-driven fault diagnosis methods are being extensively used and still have considerable potential. In recent years, methods based on deep neural networks have made significant breakthroughs, and fault diagnosis methods for industrial processes based on deep learning have attracted considerable research attention. Therefore, we propose a fusion deeplearning algorithm based on a fully convolutional neural network(FCN) to extract features and build models to correctly diagnose all types of faults. We use long short-term memory(LSTM) units to expand our proposed FCN so that our proposed deep learning model can better extract the time-domain features of chemical process data. We also introduce the attention mechanism into the model, aimed at highlighting the importance of features, which is significant for the fault diagnosis of chemical processes with many features. When applied to the benchmark Tennessee Eastman process, our proposed model exhibits impressive performance, demonstrating the effectiveness of the attention-based LSTM FCN in chemical process fault diagnosis.
文摘There are two technical challenges in predicting slope deformation.The first one is the random displacement,which could not be decomposed and predicted by numerically resolving the observed accumulated displacement and time series of a landslide.The second one is the dynamic evolution of a landslide,which could not be feasibly simulated simply by traditional prediction models.In this paper,a dynamic model of displacement prediction is introduced for composite landslides based on a combination of empirical mode decomposition with soft screening stop criteria(SSSC-EMD)and deep bidirectional long short-term memory(DBi-LSTM)neural network.In the proposed model,the time series analysis and SSSC-EMD are used to decompose the observed accumulated displacements of a slope into three components,viz.trend displacement,periodic displacement,and random displacement.Then,by analyzing the evolution pattern of a landslide and its key factors triggering landslides,appropriate influencing factors are selected for each displacement component,and DBi-LSTM neural network to carry out multi-datadriven dynamic prediction for each displacement component.An accumulated displacement prediction has been obtained by a summation of each component.For accuracy verification and engineering practicability of the model,field observations from two known landslides in China,the Xintan landslide and the Bazimen landslide were collected for comparison and evaluation.The case study verified that the model proposed in this paper can better characterize the"stepwise"deformation characteristics of a slope.As compared with long short-term memory(LSTM)neural network,support vector machine(SVM),and autoregressive integrated moving average(ARIMA)model,DBi-LSTM neural network has higher accuracy in predicting the periodic displacement of slope deformation,with the mean absolute percentage error reduced by 3.063%,14.913%,and 13.960%respectively,and the root mean square error reduced by 1.951 mm,8.954 mm and 7.790 mm respectively.Conclusively,this model not only has high prediction accuracy but also is more stable,which can provide new insight for practical landslide prevention and control engineering.
基金The National Key R&D Program of China under contract No.2016YFC1402103
文摘To explore new operational forecasting methods of waves,a forecasting model for wave heights at three stations in the Bohai Sea has been developed.This model is based on long short-term memory(LSTM)neural network with sea surface wind and wave heights as training samples.The prediction performance of the model is evaluated,and the error analysis shows that when using the same set of numerically predicted sea surface wind as input,the prediction error produced by the proposed LSTM model at Sta.N01 is 20%,18%and 23%lower than the conventional numerical wave models in terms of the total root mean square error(RMSE),scatter index(SI)and mean absolute error(MAE),respectively.Particularly,for significant wave height in the range of 3–5 m,the prediction accuracy of the LSTM model is improved the most remarkably,with RMSE,SI and MAE all decreasing by 24%.It is also evident that the numbers of hidden neurons,the numbers of buoys used and the time length of training samples all have impact on the prediction accuracy.However,the prediction does not necessary improve with the increase of number of hidden neurons or number of buoys used.The experiment trained by data with the longest time length is found to perform the best overall compared to other experiments with a shorter time length for training.Overall,long short-term memory neural network was proved to be a very promising method for future development and applications in wave forecasting.
基金supported by the Ministry of Trade,Industry & Energy(MOTIE,Korea) under Industrial Technology Innovation Program (No.10063424,'development of distant speech recognition and multi-task dialog processing technologies for in-door conversational robots')
文摘A Long Short-Term Memory(LSTM) Recurrent Neural Network(RNN) has driven tremendous improvements on an acoustic model based on Gaussian Mixture Model(GMM). However, these models based on a hybrid method require a forced aligned Hidden Markov Model(HMM) state sequence obtained from the GMM-based acoustic model. Therefore, it requires a long computation time for training both the GMM-based acoustic model and a deep learning-based acoustic model. In order to solve this problem, an acoustic model using CTC algorithm is proposed. CTC algorithm does not require the GMM-based acoustic model because it does not use the forced aligned HMM state sequence. However, previous works on a LSTM RNN-based acoustic model using CTC used a small-scale training corpus. In this paper, the LSTM RNN-based acoustic model using CTC is trained on a large-scale training corpus and its performance is evaluated. The implemented acoustic model has a performance of 6.18% and 15.01% in terms of Word Error Rate(WER) for clean speech and noisy speech, respectively. This is similar to a performance of the acoustic model based on the hybrid method.
基金This research is funded by Vellore Institute of Technology,Chennai,India.
文摘In today’s world, there are many people suffering from mentalhealth problems such as depression and anxiety. If these conditions are notidentified and treated early, they can get worse quickly and have far-reachingnegative effects. Unfortunately, many people suffering from these conditions,especially depression and hypertension, are unaware of their existence until theconditions become chronic. Thus, this paper proposes a novel approach usingBi-directional Long Short-Term Memory (Bi-LSTM) algorithm and GlobalVector (GloVe) algorithm for the prediction and treatment of these conditions.Smartwatches and fitness bands can be equipped with these algorithms whichcan share data with a variety of IoT devices and smart systems to betterunderstand and analyze the user’s condition. We compared the accuracy andloss of the training dataset and the validation dataset of the two modelsnamely, Bi-LSTM without a global vector layer and with a global vector layer.It was observed that the model of Bi-LSTM without a global vector layer hadan accuracy of 83%,while Bi-LSTMwith a global vector layer had an accuracyof 86% with a precision of 86.4%, and an F1 score of 0.861. In addition toproviding basic therapies for the treatment of identified cases, our model alsohelps prevent the deterioration of associated conditions, making our methoda real-world solution.
基金support of national natural science foundation of China(No.52067021)natural science foundation of Xinjiang(2022D01C35)+1 种基金excellent youth scientific and technological talents plan of Xinjiang(No.2019Q012)major science&technology special project of Xinjiang Uygur Autonomous Region(2022A01002-2)。
文摘Wind power volatility not only limits the large-scale grid connection but also poses many challenges to safe grid operation.Accurate wind power prediction can mitigate the adverse effects of wind power volatility on wind power grid connections.For the characteristics of wind power antecedent data and precedent data jointly to determine the prediction accuracy of the prediction model,the short-term prediction of wind power based on a combined neural network is proposed.First,the Bi-directional Long Short Term Memory(BiLSTM)network prediction model is constructed,and the bi-directional nature of the BiLSTM network is used to deeply mine the wind power data information and find the correlation information within the data.Secondly,to avoid the limitation of a single prediction model when the wind power changes abruptly,the Wavelet Transform-Improved Adaptive Genetic Algorithm-Back Propagation(WT-IAGA-BP)neural network based on the combination of the WT-IAGA-BP neural network and BiLSTM network is constructed for the short-term prediction of wind power.Finally,comparing with LSTM,BiLSTM,WT-LSTM,WT-BiLSTM,WT-IAGA-BP,and WT-IAGA-BP&LSTM prediction models,it is verified that the wind power short-term prediction model based on the combination of WT-IAGA-BP neural network and BiLSTM network has higher prediction accuracy.
文摘In dense pedestrian tracking,frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories.In this study,a conditional random field tracking model is established by using a visual long short term memory network in the three-dimensional(3D)space and the motion estimations jointly performed on object trajectory segments.Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation.To address the uncertainty of the length and interval of trajectory segments,a multimode long short term memory network is proposed for the object motion estimation.The tracking performance is evaluated using the PETS2009 dataset.The experimental results show that the proposed method achieves better performance than the tracking methods based on the independent motion estimation.
基金supported by the National Major Science and Technology Special Project(No.2016ZX05026-002).
文摘In this paper,the recurrent neural network structure of a bidirectional long shortterm memory network(Bi-LSTM)with special memory cells that store information is used to characterize the deep features of the variation pattern between logging and seismic data.A mapping relationship model between high-frequency logging data and low-frequency seismic data is established via nonlinear mapping.The seismic waveform is infinitely approximated using the logging curve in the low-frequency band to obtain a nonlinear mapping model of this scale,which then stepwise approach the logging curve in the high-frequency band.Finally,a seismic-inversion method of nonlinear mapping multilevel well–seismic matching based on the Bi-LSTM network is developed.The characteristic of this method is that by applying the multilevel well–seismic matching process,the seismic data are stepwise matched to the scale range that is consistent with the logging curve.Further,the matching operator at each level can be stably obtained to effectively overcome the problems that occur in the well–seismic matching process,such as the inconsistency in the scale of two types of data,accuracy in extracting the seismic wavelet of the well-side seismic traces,and multiplicity of solutions.Model test and practical application demonstrate that this method improves the vertical resolution of inversion results,and at the same time,the boundary and the lateral characteristics of the sand body are well maintained to improve the accuracy of thin-layer sand body prediction and achieve an improved practical application effect.
基金supported in part by the National Natural Science Foundation of China under Grant 62203468in part by the Technological Research and Development Program of China State Railway Group Co.,Ltd.under Grant Q2023X011+1 种基金in part by the Young Elite Scientist Sponsorship Program by China Association for Science and Technology(CAST)under Grant 2022QNRC001in part by the Youth Talent Program Supported by China Railway Society,and in part by the Research Program of China Academy of Railway Sciences Corporation Limited under Grant 2023YJ112.
文摘Purpose-To optimize train operations,dispatchers currently rely on experience for quick adjustments when delays occur.However,delay predictions often involve imprecise shifts based on known delay times.Real-time and accurate train delay predictions,facilitated by data-driven neural network models,can significantly reduce dispatcher stress and improve adjustment plans.Leveraging current train operation data,these models enable swift and precise predictions,addressing challenges posed by train delays in high-speed rail networks during unforeseen events.Design/methodology/approach-This paper proposes CBLA-net,a neural network architecture for predicting late arrival times.It combines CNN,Bi-LSTM,and attention mechanisms to extract features,handle time series data,and enhance information utilization.Trained on operational data from the Beijing-Tianjin line,it predicts the late arrival time of a target train at the next station using multidimensional input data from the target and preceding trains.Findings-This study evaluates our model’s predictive performance using two data approaches:one considering full data and another focusing only on late arrivals.Results show precise and rapid predictions.Training with full data achieves aMAEof approximately 0.54 minutes and a RMSEof 0.65 minutes,surpassing the model trained solely on delay data(MAE:is about 1.02 min,RMSE:is about 1.52 min).Despite superior overall performance with full data,the model excels at predicting delays exceeding 15 minutes when trained exclusively on late arrivals.For enhanced adaptability to real-world train operations,training with full data is recommended.Originality/value-This paper introduces a novel neural network model,CBLA-net,for predicting train delay times.It innovatively compares and analyzes the model’s performance using both full data and delay data formats.Additionally,the evaluation of the network’s predictive capabilities considers different scenarios,providing a comprehensive demonstration of the model’s predictive performance.
文摘Hydrological models are developed to simulate river flows over a watershed for many practical applications in the field of water resource management. The present paper compares the performance of two recurrent neural networks for rainfall-runoff modeling in the Zou River basin at Atchérigbé outlet. To this end, we used daily precipitation data over the period 1988-2010 as input of the models, such as the Long Short-Term Memory (LSTM) and Recurrent Gate Networks (GRU) to simulate river discharge in the study area. The investigated models give good results in calibration (R2 = 0.888, NSE = 0.886, and RMSE = 0.42 for LSTM;R2 = 0.9, NSE = 0.9 and RMSE = 0.397 for GRU) and in validation (R2 = 0.865, NSE = 0.851, and RMSE = 0.329 for LSTM;R2 = 0.9, NSE = 0.865 and RMSE = 0.301 for GRU). This good performance of LSTM and GRU models confirms the importance of models based on machine learning in modeling hydrological phenomena for better decision-making.
文摘Hand gestures are a natural way for human-robot interaction.Vision based dynamic hand gesture recognition has become a hot research topic due to its various applications.This paper presents a novel deep learning network for hand gesture recognition.The network integrates several well-proved modules together to learn both short-term and long-term features from video inputs and meanwhile avoid intensive computation.To learn short-term features,each video input is segmented into a fixed number of frame groups.A frame is randomly selected from each group and represented as an RGB image as well as an optical flow snapshot.These two entities are fused and fed into a convolutional neural network(Conv Net)for feature extraction.The Conv Nets for all groups share parameters.To learn longterm features,outputs from all Conv Nets are fed into a long short-term memory(LSTM)network,by which a final classification result is predicted.The new model has been tested with two popular hand gesture datasets,namely the Jester dataset and Nvidia dataset.Comparing with other models,our model produced very competitive results.The robustness of the new model has also been proved with an augmented dataset with enhanced diversity of hand gestures.
基金National Natural Science Foundation of China(No.51805079)Shanghai Natural Science Foundation,China(No.17ZR1400600)Fundamental Research Funds for the Central Universities,China(No.16D110309)
文摘The healthy condition of the milling tool has a very high impact on the machining quality of the titanium components.Therefore,it is important to recognize the healthy condition of the tool and replace the damaged cutter at the right time.In order to recognize the health condition of the milling cutter,a method based on the long short term memory(LSTM)was proposed to recognize tool health state in this paper.The various signals collected in the tool wear experiments were analyzed by time-domain statistics,and then the extracted data were generated by principal component analysis(PCA)method.The preprocessed data extracted by PCA is transmitted to the LSTM model for recognition.Compared with back propagation neural network(BPNN)and support vector machine(SVM),the proposed method can effectively utilize the time-domain regulation in the data to achieve higher recognition speed and accuracy.
基金This reasearch was supported by the Science Foundation of Guangxi under grant No.0339025the Natural Sciences Foundation of China under grant No.40075021.
文摘In terms of the modular fuzzy neural network (MFNN) combining fuzzy c-mean (FCM) cluster and single-layer neural network, a short-term climate prediction model is developed. It is found from modeling results that the MFNN model for short-term climate prediction has advantages of simple structure, no hidden layer and stable network parameters because of the assembling of sound functions of the self-adaptive learning, association and fuzzy information processing of fuzzy mathematics and neural network methods. The case computational results of Guangxi flood season (JJA) rainfall show that the mean absolute error (MAE) and mean relative error (MRE) of the prediction during 1998-2002 are 68.8 mm and 9.78%, and in comparison with the regression method, under the conditions of the same predictors and period they are 97.8 mm and 12.28% respectively. Furthermore, it is also found from the stability analysis of the modular model that the change of the prediction results of independent samples with training times in the stably convergent interval of the model is less than 1.3 mm. The obvious oscillation phenomenon of prediction results with training times, such as in the common back-propagation neural network (BPNN) model, does not occur, indicating a better practical application potential of the MFNN model.
基金the Scientific and Technological Innovation JointCapital Projects of Fujian Province,No.2016Y9031the Construction Project of Fujian Province Minimally Invasive Medical Center,No.[2017]171+4 种基金the General Project of Miaopu Scientific Research Fund of Fujian Medical University,No.2015MP021the Youth Project of Fujian Provincial Health and Family Planning Commission,No.2016-1-41the Fujian Province Medical Innovation ProjectChinese Physicians Association Young Physician Respiratory Research Fund,No.2015-CXB-16the Fujian Science and Technology Innovation Joint Fund Project,No.2017Y9004
文摘BACKGROUND Because of the powerful abilities of self-learning and handling complex biological information,artificial neural network(ANN)models have been widely applied to disease diagnosis,imaging analysis,and prognosis prediction.However,there has been no trained preoperative ANN(preope-ANN)model to preoperatively predict the prognosis of patients with gastric cancer(GC).AIM To establish a neural network model that can predict long-term survival of GC patients before surgery to evaluate the tumor condition before the operation.METHODS The clinicopathological data of 1608 GC patients treated from January 2011 to April 2015 at the Department of Gastric Surgery,Fujian Medical University Union Hospital were analyzed retrospectively.The patients were randomly divided into a training set(70%)for establishing a preope-ANN model and a testing set(30%).The prognostic evaluation ability of the preope-ANN model was compared with that of the American Joint Commission on Cancer(8th edition)clinical TNM(cTNM)and pathological TNM(pTNM)staging through the receiver operating characteristic curve,Akaike information criterion index,Harrell's C index,and likelihood ratio chi-square.RESULTS We used the variables that were statistically significant factors for the 3-year overall survival as input-layer variables to develop a preope-ANN in the training set.The survival curves within each score of the preope-ANN had good discrimination(P<0.05).Comparing the preope-ANN model,cTNM,and pTNM in both the training and testing sets,the preope-ANN model was superior to cTNM in predictive discrimination(C index),predictive homogeneity(likelihood ratio chi-square),and prediction accuracy(area under the curve).The prediction efficiency of the preope-ANN model is similar to that of pTNM.CONCLUSION The preope-ANN model can accurately predict the long-term survival of GC patients,and its predictive efficiency is not inferior to that of pTNM stage.
基金This work is supported by the National Nature Science Foundation of China(NSFC)under Grant Nos.61571106,61501169,41706103the Fundamental Research Funds for the Central Universities under Grant No.2242013K30010.
文摘Speaker separation in complex acoustic environment is one of challenging tasks in speech separation.In practice,speakers are very often unmoving or moving slowly in normal communication.In this case,the spatial features among the consecutive speech frames become highly correlated such that it is helpful for speaker separation by providing additional spatial information.To fully exploit this information,we design a separation system on Recurrent Neural Network(RNN)with long short-term memory(LSTM)which effectively learns the temporal dynamics of spatial features.In detail,a LSTM-based speaker separation algorithm is proposed to extract the spatial features in each time-frequency(TF)unit and form the corresponding feature vector.Then,we treat speaker separation as a supervised learning problem,where a modified ideal ratio mask(IRM)is defined as the training function during LSTM learning.Simulations show that the proposed system achieves attractive separation performance in noisy and reverberant environments.Specifically,during the untrained acoustic test with limited priors,e.g.,unmatched signal to noise ratio(SNR)and reverberation,the proposed LSTM based algorithm can still outperforms the existing DNN based method in the measures of PESQ and STOI.It indicates our method is more robust in untrained conditions.
文摘In recent years, introduction of a renewable energy source such as solar energy is expected. However, solar radiation is not constant and power output of photovoltaic (PV) system is influenced by weather conditions. It is difficult for getting to know accurate power output of PV system. In order to forecast the power output of PV system as accurate as possible, this paper proposes a decision technique of forecasting model for short-term-ahead power output of PV system based on solar radiation prediction. Application of Recurrent Neural Network (RNN) is shown for solar radiation prediction in this paper. The proposed method in this paper does not require complicated calculation, but mathematical model with only useful weather data. The validity of the proposed RNN is confirmed by comparing simulation results of solar radiation forecasting with that obtained from other
基金funded by Fujian Science and Technology Key Project(No.2016H6022,2018J01099,2017H0037)
文摘The fraction defective of semi-finished products is predicted to optimize the process of relay production lines, by which production quality and productivity are increased, and the costs are decreased. The process parameters of relay production lines are studied based on the long-and-short-term memory network. Then, the Keras deep learning framework is utilized to build up a short-term relay quality prediction algorithm for the semi-finished product. A simulation model is used to study prediction algorithm. The simulation results show that the average prediction absolute error of the fraction is less than 5%. This work displays great application potential in the relay production lines.
基金Sponsored by the National Natural Science Foundation of China (Grant No. 50305005)
文摘Aimed at the problem that the traditional ART-2 neural network can not recognize a gradually changing course, an eternal term memory (ETM) vector is introduced into ART-2 to simulate the function of human brain, i.e. the deep remembrance for the initial impression.. The eternal term memory vector is determined only by the initial vector that establishes category neuron node and is used to keep the remembrance for this vector for ever. Two times of vigilance algorithm are put forward, and the posterior input vector must first pass the first vigilance of this eternal term memory vector, only succeeded has it the qualification to begin the second vigilance of long term memory vector. The long term memory vector can be revised only when both of the vigilances are passed. Results of recognition examples show that the improved ART-2 overcomes the defect of traditional ART-2 and can recognize a gradually changing course effectively.
文摘The fuzzy neural network is applied to the short-term load forecasting. The fuzzy rules and fuzzy membership functions of the network are obtained through fuzzy neural network learming. Three inference algorithms, i.e. themultiplicative inference, the maximum inference and the minimum inference, are used for comparison. The learningalgorithms corresponding to the inference methods are derived from back-propagation algorithm. To validate the fuzzyneural network model, the network is used to Predict short-term load by compaing the network output against the realload data from a local power system supplying electricity to a large steel manufacturer. The experimental results aresatisfactory.
