摘要
深度神经网络在众多视觉表征领域取得了显著的成功,如目标检测、识别等.然而,需要大量良好标记的数据进行训练是它们最普遍的限制之一.在实际应用中,为每一个要学习的新任务建立庞大的标记数据集是极其昂贵,甚至是不可行的.半监督深度学习,通过在有限标记数据的条件下充分挖掘大量的未标记数据信息,从而达到与监督学习相媲美的分类精度.然而,当标记数据极其稀少时,现有半监督算法的性能会受到严重影响.因此,本文提出了一种可靠标签选择与学习(Reliable Label Selection and Learning,ReLSL)算法,以解决在仅有极少量标签图像数据时半监督深度学习所面临的问题.具体地,本文首先运用无监督学习方法提取样本特征,并应用基于图的标签传染算法得到无标签样本的伪标签.而后,为了筛选出更为可靠、有更多信息的样本,本文提出了一种综合考虑样本输出均值和一致性的伪标签学习与标定策略.在获得具有扩展标签的数据集后,考虑到训练样本中引入一定比例的标签噪声无可避免,因此本文提出两种策略来训练高鲁棒半监督深度模型:标签平滑策略(Label Smoothing Strategy,LS),用以避免标签过于尖锐;均值偏移校正策略(Mean Shifting Correction Strategy,MSC),用以降低样本输出偏移风险.实验结果表明,在CNN-13、WRN-28-2及ResNet-18各种网络结构下,本文所提出的ReLSL算法在CIFAR-10/100、SVHN、STL-10和Mini-ImageNet数据集上均表现出先进的性能.特别地,本文算法在WRN-28-2网络结构下仅有10个标记数据的CIFAR-10上,相较于最新算法具有6.78%的准确率提升;在CNN-13网络下仅有100个标记数据时,可以达到目前主流半监督学习算法4000标记时的测试误差6.39±0.47%.
Deep neural networks have achieved remarkable success in many visual representation fields,such as object detection,recognition,etc.However,requiring the large quantity of well labeled data for training is one of their most prevalent limitations.Many real-world classification applications are concerned with samples that are not presented in standard benchmark datasets,and building large labeled dataset for each new task to be learned is not practically feasible.Although enormous quantities of unlabeled data are accessible and can be collected with minimal effort,the data labeling process is still extremely expensive.Semi-supervised learning(SSL)provides a way to improve a model’s performance with the surplus of unlabeled data when only limited labeled data are available.However,when the labeled data is extremely scarce,the performance of the existing SSL algorithms can be severely affected.For example,on the prevalent CIFAR-10 dataset,when each class is supported by only one label sample,the accuracy of most SSL algorithms degrades seriously.The problem is mainly manifested as:the initial informative information for classification is extremely limited,the model faces cold-start problem;in the process of training,the proportion of pseudo-label noise is difficult to control and the model has a much larger potential risk to be collapsed.In this paper,we propose a Reliable Label Selection and Learning(ReLSL)framework,which tackles the problem semi-supervised deep learning facing when only few-shot labeled image data is available.In brief,we exploit synergies among unsupervised learning,SSL and robust learning to bootstrap additional reliable labels for robust network training.For the unsupervised learning,it is used to ease the problem of cold-start under scarce labeled conditions.For SSL and robust learning,they are used to obtain good learning performance in the presence of noise labels.To be specific,for our whole ReLSL,we first implement Anchor Neighborhood Discovery(AND),an unsupervised learning algorithm to extract features of all training samples,and then obtain their pseudo-label by applying graph-based label propagation algorithm.Then,in order to screen out more reliable and informative samples,a pseudo-label learning and calibration strategy is proposed that comprehensively considers the mean and consistency of the sample’s output,and conduct effective screening of samples through Small-Loss theory.After obtaining the dataset with extended labels,considering that a certain proportion of label noise is inevitably introduced into the training set,we therefore propose two strategies to train a robust SSL model,namely,a Label-Smoothing strategy(LS)for regularizing labels from being too sharp,thus reducing noise label interference to loss function;Mean-Shifting Correction strategy(MSC)for reducing the risk of sample output deviation.As a result,the proposed ReLSL achieves state-of-the-art performance on CIFAR-10/100,SVHN,STL-10 and Mini-ImageNet across a variety of SSL conditions with the CNN-13,WRN-28-2 and ResNet-18 networks.In particular,our framework achieves a 6.78%accuracy boosting on CIFAR-10 with only 10 labeled data under WRN-28-2.Moreover,our algorithm can achieve the test error of 6.39±0.47%with only 100 labeled data under CNN-13,which is comparable to the one with typical SSL under 4000 labeled conditions.
作者
魏翔
王靖杰
张顺利
张迪
张健
魏小涛
WEI Xiang;WANG Jing-Jie;ZHANG Shun-Li;ZHANG Di;ZHANG Jian;WEI Xiao-Tao(School of Software Engineering,Beijing Jiaotong University,Beijing 100044)
出处
《计算机学报》
EI
CAS
CSCD
北大核心
2022年第6期1147-1160,共14页
Chinese Journal of Computers
基金
国家自然科学基金(61906014,61976017,61902019)
北京市自然科学基金(4202056)资助.
关键词
半监督深度学习
极少标签
鲁棒性
标签传播
特征提取
semi-supervised learning
few-shot labels
robustness
label propagation
feature extraction
