摘要
针对深度学习用于处理带有个性化特征的广域分布式数据时,处理精度、通信代价和响应速度等性能难以进一步提升的问题,本论文提议了一种适用于广域网络的粗粒度分布式深度学习方法及系统。分布式深度学习方法一般分为数据分布式和模型分布式,还可以在网络范畴上分为局域分布式和广域分布式。其中,数据分布式比模型分布式更易实现,但模型分布式在参数规模的扩展性上更具优势;相比于局域分布式,广域分布式在通信代价上更具挑战性,但它可以距离用户更近,从而在响应速度上更具优势。在论文中,分布式深度学习方法被进一步分为细粒度分布式和粗粒度分布式。相比于细粒度分布式,粗粒度分布式的相对通信时间更短,从而更适合于广域网络。论文所提议的系统可以作为粗粒度分布式深度学习方法的一个范例,适用于在广域网络上处理具有个性化特征的分布式数据。评价结果表明,粗粒度分布式系统不仅自然保证了更好的通信代价和响应速度,而且提升了个性化数据的处理精度。
It is hard for personal data processing to improve the performance including accuracy, response time and communication cost in a global area network even using the latest distributed system of deep learning. In this paper, we propose a coarse - grained distribu- ted system of deep learning to effectively process personal data in the global area network. As we know, the exiting distributed systems of deep learning could be classified as data - based and models - based. In this paper, the distributed systems of deep learning are fur- ther classified as fine - grained and coarse - grained by redefining a concept of granularity. A coarse - grained distributed system has relatively short communication time and long computation time. Furthermore, a typical deep learning model is used to realize the coarse - grained distributed system. The model is pre - trained based on the public data in the core of Internet. Then, the model is downloaded by each terminal device and fine - tuned based on the personal data on the edge of Internet. As a result, each user owns a personalized model. Evaluation shows the proposal is superior to a state - of - the - art model in an application of personalized media mining.
出处
《网络新媒体技术》
2016年第6期1-6,共6页
Network New Media Technology
基金
中国科学院先导专项(XDA06040501)
关键词
粗粒度
细粒度
分布式深度学习
个性化数据挖掘
媒体挖掘
通信代价
精度
coarse granularity, fine granularity, distributed deep learning, personal data mining, media mining, communication cost, accuracy
