摘要
基于差分隐私的深度学习隐私保护方法中,训练周期的长度以及隐私预算的分配方式直接制约着深度学习模型的效用.针对现有深度学习结合差分隐私的方法中模型训练周期有限、隐私预算分配不合理导致模型安全性与可用性差的问题,提出一种基于数据特征相关性和自适应差分隐私的深度学习方法(deep learning methods based on data feature Relevance and Adaptive Differential Privacy,RADP).首先,该方法利用逐层相关性传播算法在预训练模型上计算出原始数据集上每个特征的平均相关性;然后,使用基于信息熵的方法计算每个特征平均相关性的隐私度量,根据隐私度量对特征平均相关性自适应地添加拉普拉斯噪声;在此基础上,根据加噪保护后的每个特征平均相关性,合理分配隐私预算,自适应地对特征添加拉普拉斯噪声;最后,理论分析该方法(RADP)满足ε-差分隐私,并且兼顾安全性与可用性.同时,在三个真实数据集(MNIST,Fashion-MNIST,CIFAR-10)上的实验结果表明,RADP方法的准确率以及平均损失均优于AdLM(Adaptive Laplace Mechanism)方法、DPSGD(Differential Privacy with Stochastic Gradient Descent)方法和DPDLIGDO(Differentially Private Deep Learning with Iterative Gradient Descent Optimization)方法,并且RADP方法的稳定性仍能保持良好.
In the deep learning privacy protection based on differential privacy,the length of the training period and the allocation of the privacy budget directly restrict the utility of the deep learning model.In the existing methods of deep learning combined with differential privacy,the model training cycle is limited and the budget allocation of a large number of feature privacy is unreasonable,which leads to poor security and availability of the model.We propose a method of deep learning methods based on data feature relevance and adaptive differential privacy(RADP).First,the method uses the layerby-layer correlation propagation algorithm to calculate the average correlation of each feature parameter and the output result on the original data set on the pre-trained model and uses the information entropy-based method to calculate the average correlation of each feature parameter.According to the privacy metric,the Laplace noise is adaptively added to the average correlation;on this basis,according to the average correlation of each feature parameter,the privacy budget is allocated reasonably,Laplace noise is added to the feature parameters;finally,theoretical analysis shows that the method proposed in this paper satisfiesε-differential privacy and take into account security and availability.Based on the experimental results on 3 real datasets MNIST,Fashion-MNIST,and CIFAR-10,the accuracy and average loss of RADP are better than those of the AdLM(Adaptive Laplace Mechanism)method,the DPSGD(Differential Privacy with Stochastic Gradient Descent)method and the DPDLIGDO(Differentially Private Deep Learning with Iterative Gradient Descent Optimization)method.Moreover,the stability of RADP method can still be maintained well.
作者
康海燕
王骁识
KANG Hai-yan;WANG Xiao-shi(Information Security Department,Beijing Information Science and Technology University,Beijing 100192,China)
出处
《电子学报》
EI
CAS
CSCD
北大核心
2024年第6期1963-1976,共14页
Acta Electronica Sinica
基金
国家社科基金(No.21BTQ079)
教育部人文社科项目(No.20YJAZH046)
国家自然科学基金(No.61370139)。
关键词
差分隐私
深度学习
逐层相关性传播
信息熵
隐私度量
隐私预算
拉普拉斯机制
differential privacy
deep learning
layer-wise relevance propagation
entropy of information
privacy Metrics
privacy budget
laplace mechanism
