摘要
循环神经网络(RNN)是一类非常强大的用于处理和预测序列数据的神经网络模型。循环结构的神经网络克服了传统机器学习方法对输入和输出数据的许多限制,使其成为深度学习领域中一类非常重要的模型。RNN及其变体网络已经被成功应用于多种任务,尤其是当数据中存在一定时间依赖性的时候。语音识别、机器翻译、语言模型、文本分类、词向量生成、信息检索等,都需要一个模型能够将具有序列性质的数据作为输入进行学习;然而,RNN通常难以训练,循环多次之后,大多数情况下梯度往往倾向于消失,也有较少情况会发生梯度爆炸的问题。针对RNN在实际应用中存在的问题,长短期记忆(LSTM)网络被提出,它能够保持信息的长期存储而备受关注,关于LSTM结构的改进工作也陆续出现。然后,主要针对循环结构的神经网络的发展进行详细阐述,对目前流行的几种变体模型进行详细的讨论和对比。最后,对循环结构的神经网络的发展趋势进行了探讨。
Recurrent Neural Network(RNN)is a kind of very powerful neural network for processing and predicting sequence data.The neural network of recurrent structure overcomes many limitations of traditional machine learning methods on input and output,making it a very important model in the field of deep learning.RNN and its variants have been successfully applied to a variety of tasks,especially with time-dependent data.Speech recognition,machine translation,language model,text classification,word vector generation,information retrieval and so on,all need a model that can learn sequence data.However,Learning with RNN has long been considered to be difficult,the gradient tends to vanish in most cases,and there are less cases where exploding will occur when backpropagating errors across many time steps.Due to problems in practical application of RNN,LSTM has been proposed for their long-term storage of information,and improvements have also been made to the structure of LSTM.Then,the development of the neural network based on recurrent structure was mainly analyzed.Several popular variants were discussed and compared in detail.Finally,the development trend of neural network with recurrent structure was discussed.
作者
杨丽
吴雨茜
王俊丽
刘义理
YANG Li;WU Yuxi;WANG Junli;LIU Yili(College of Electronics and Information Engineering,Tongji University,Shanghai 201804,China;College of Economics and Management,Tongji University,Shanghai 201804,China)
出处
《计算机应用》
CSCD
北大核心
2018年第A02期1-6,26,共7页
journal of Computer Applications
基金
同济大学中央高校基本科研业务费资助项目
关键词
循环神经网络
长短期记忆网络
深度学习
神经网络
序列数据
Recurrent Neural Network(RNN)
Long-Short Term Memory(LSTM)network
deep learning
neural network
sequence data
