摘要
基于深度学习的点击率预估模型多数通过建模各个域的特征之间的交互关系提升预估准确率。特征嵌入向量对模型效果具有重要影响,而现有的CTR模型中不同特征的嵌入向量学习过程相互独立,且由于特征长尾分布导致大部分低频特征不能学习到较好的向量表示,严重影响模型的预测效果。基于域内特征间存在隐含的相似性,提出两种分别基于特征间共现概率和游走概率的相似度定义和对应的相似性图构建方法,并给出结合剪枝策略的广度优先遍历算法实现相似特征的高效计算。在此基础上,基于域内特征相似性图,设计一种嵌入生成器,对于低频特征,在域内特征相似性图上通过图神经网络聚合与其相似的特征信息,生成新的特征嵌入,作为预处理过程对特征嵌入向量进行数据增强,提升嵌入向量的表示学习质量。在公开数据集Criteo、Avazu上的实验结果表明,该方法明显提升点击率预估模型的预测准确率,其中对代表性点击率预估模型xDeepFM和AutoInt,AUC指标分别提升了0.007和0.008,LogLoss则下降了0.009和0.006,证明了嵌入生成模型的有效性。
There exist numerous deep learning-based Click-Through Rate(CTR) models;however,most of them improve prediction accuracy by modeling feature interaction between different fields.The feature embedding vectors have a significant impact on model performance;existing CTR models independently learn the embedding vectors of different features in a field.Consequently,most low-frequency features cannot attain sufficiently good embeddings because of the long-tail feature distribution,which seriously affects model accuracy.Noticing that implicit similarity exists between features of the same field,this study proposes a two intra-field feature similarity based on co-occurrence probability and walk probability,respectively.Moreover,the proposed method develops the corresponding similarity graph construction method and designs a breadth first traversal algorithm combined with pruning strategy to efficiently calculate similar features.Based on the intra-field feature similarity graph,an embedding generator is also proposed.For low-frequency features,information of similar features is aggregated on the similarity graph through a graph neural network.This data augmentation method is used as a preprocessing step to improve the learning quality of feature embedding vectors.Extensive experiments conducted on the public data sets Criteo and Avazu demonstrate that the proposed method improves the prediction accuracy of several classical CTR models.Regarding the representative CTR models xDeepFM and AutoInt,the AUC is increased by 0.007 and 0.008,respectively,while the LogLoss is decreased by 0.009 and 0.006,respectively,which proves the effectiveness of the embedding generator.
作者
雷李想
武志昊
刘钰
周子站
LEI Lixiang;WU Zhihao;LIU Yu;ZHOU Zizhan(School of Computer and Information Technology,Beijing Jiaotong University,Beijing 100044,China;Institute of Network Science and Intelligent System,Beijing Jiaotong University,Beijing 100044,China;TravelSky Technology Limited,Beijing 101318,China;Key Laboratory of Intelligent Application Technology for Civil Aviation Passenger Services,Beijing 101318,China)
出处
《计算机工程》
CAS
CSCD
北大核心
2023年第2期238-245,共8页
Computer Engineering
基金
国家自然科学基金(61603028)
中国民航信息网络股份有限公司和民航旅客服务智能化应用技术重点实验室基金项目(K20L00070)。
关键词
点击率预估
稀疏特征
特征嵌入
特征相似性
图神经网络
Click-Through Rate(CTR)prediction
sparse feature
feature embedding
feature similarity
graph neural network
