With the popularity of deep learning tools in image decomposition and natural language processing,how to support and store a large number of parameters required by deep learning algorithms has become an urgent problem...With the popularity of deep learning tools in image decomposition and natural language processing,how to support and store a large number of parameters required by deep learning algorithms has become an urgent problem to be solved.These parameters are huge and can be as many as millions.At present,a feasible direction is to use the sparse representation technique to compress the parameter matrix to achieve the purpose of reducing parameters and reducing the storage pressure.These methods include matrix decomposition and tensor decomposition.To let vector take advance of the compressing performance of matrix decomposition and tensor decomposition,we use reshaping and unfolding to let vector be the input and output of Tensor-Factorized Neural Networks.We analyze how reshaping can get the best compress ratio.According to the relationship between the shape of tensor and the number of parameters,we get a lower bound of the number of parameters.We take some data sets to verify the lower bound.展开更多
Neural Machine Translation(NMT)is an end-to-end learning approach for automated translation,overcoming the weaknesses of conventional phrase-based translation systems.Although NMT based systems have gained their popul...Neural Machine Translation(NMT)is an end-to-end learning approach for automated translation,overcoming the weaknesses of conventional phrase-based translation systems.Although NMT based systems have gained their popularity in commercial translation applications,there is still plenty of room for improvement.Being the most popular search algorithm in NMT,beam search is vital to the translation result.However,traditional beam search can produce duplicate or missing translation due to its target sequence selection strategy.Aiming to alleviate this problem,this paper proposed neural machine translation improvements based on a novel beam search evaluation function.And we use reinforcement learning to train a translation evaluation system to select better candidate words for generating translations.In the experiments,we conducted extensive experiments to evaluate our methods.CASIA corpus and the 1,000,000 pairs of bilingual corpora of NiuTrans are used in our experiments.The experiment results prove that the proposed methods can effectively improve the English to Chinese translation quality.展开更多
基金This work was supported by National Natural Science Foundation of China(Nos.61802030,61572184)the Science and Technology Projects of Hunan Province(No.2016JC2075)the International Cooperative Project for“Double First-Class”,CSUST(No.2018IC24).
文摘With the popularity of deep learning tools in image decomposition and natural language processing,how to support and store a large number of parameters required by deep learning algorithms has become an urgent problem to be solved.These parameters are huge and can be as many as millions.At present,a feasible direction is to use the sparse representation technique to compress the parameter matrix to achieve the purpose of reducing parameters and reducing the storage pressure.These methods include matrix decomposition and tensor decomposition.To let vector take advance of the compressing performance of matrix decomposition and tensor decomposition,we use reshaping and unfolding to let vector be the input and output of Tensor-Factorized Neural Networks.We analyze how reshaping can get the best compress ratio.According to the relationship between the shape of tensor and the number of parameters,we get a lower bound of the number of parameters.We take some data sets to verify the lower bound.
基金This work is supported by the National Natural Science Foundation of China(61872231,61701297).
文摘Neural Machine Translation(NMT)is an end-to-end learning approach for automated translation,overcoming the weaknesses of conventional phrase-based translation systems.Although NMT based systems have gained their popularity in commercial translation applications,there is still plenty of room for improvement.Being the most popular search algorithm in NMT,beam search is vital to the translation result.However,traditional beam search can produce duplicate or missing translation due to its target sequence selection strategy.Aiming to alleviate this problem,this paper proposed neural machine translation improvements based on a novel beam search evaluation function.And we use reinforcement learning to train a translation evaluation system to select better candidate words for generating translations.In the experiments,we conducted extensive experiments to evaluate our methods.CASIA corpus and the 1,000,000 pairs of bilingual corpora of NiuTrans are used in our experiments.The experiment results prove that the proposed methods can effectively improve the English to Chinese translation quality.
