摘要
图像序列光流计算是图像处理与计算机视觉等领域的重要研究方向.随着深度学习技术的快速发展,以卷积神经网络为代表的深度学习理论与方法成为光流计算技术研究的热点.本文主要对深度学习光流计算技术研究进行综述,首先介绍了有监督学习、无监督学习和半监督学习的光流计算网络模型与训练策略,然后重点阐述并分析了不同网络模型优化方法.针对光流计算模型的评估问题,分别介绍了Middlebury、MPI-Sintel和KITTI等数据库及评价基准,并对不同类型深度学习和传统变分光流模型进行对比与分析.最后,总结了深度学习光流计算技术在模型复杂度与泛化性、光流估计鲁棒性、小样本训练准确性等方面的关键技术问题,并指出了可能的解决方案与研究思路.
Optical flow computation is an important research direction in image processing and computer vision.With the rapid development of the deep learning technology,the convolutional neural network based deep learning theories and methodologies have been the research focus of optical flow computation.This article mainly reviews the research progress of the deep learning based optical flow estimation technologies.First,the typical models and training strategies of the optical flow computing networks with supervised learning,unsupervised learning and semi-supervised learning are introduced.Second,the optimization methods of various network models are described and analyzed.Third,the evaluation benchmarks of Middlebury,MPI-Sintel and KITTI databases are summarized,and the experimental comparison results and analysis between the different deep-learning and variational optical flow methods are conducted.Finally,we discuss some issues of the deep learning based optical flow computation technology including the model complexity and generalization,the robustness of optical flow estimation and the accuracy of the small sample training.Afterwards,we point out several possible solutions and research ideas to address the above mentioned issues.
作者
张聪炫
周仲凯
陈震
葛利跃
黎明
江少锋
陈昊
ZHANG Cong-xuan;ZHOU Zhong-kai;CHEN Zhen;GE Li-yue;LI Ming;JIANG Shao-feng;CHEN Hao(Key Laboratory of Nondestructive Testing,Ministry of Education,Nanchang Hangkong University,Nanchang,Jiangxi 330063,China;Institute of Automation,Chinese Academy of Sciences,Beijing 100190,China)
出处
《电子学报》
EI
CAS
CSCD
北大核心
2020年第9期1841-1849,共9页
Acta Electronica Sinica
基金
国家自然科学基金(No.61866026,No.61772255,No.61866025)
江西省优势科技创新团队计划(No.20165BCB19007)
江西省科技创新杰出青年人才计划(No.20192BCB23011)
航空科学基金(No.2018ZC56008)
中国博士后科学基金(No.2019M650894)
江西省重点研发计划(No.20171BBG70052)
江西省研究生创新专项资金项目资助(No.YC2018049)。
关键词
光流计算
深度学习
卷积神经网络
训练策略
优化方法
评价基准
optical flow computation
deep learning
convolutional neural network
training strategy
optimization method
evaluation benchmark
