摘要
为确保源图像中的显著区域在融合图像保持显著,提出了一种自注意力引导的红外与可见光图像融合方法。在特征学习层引入自注意力学习机制获取源图像的特征图和自注意力图,利用自注意力图可以捕获到图像中长距离依赖的特性,设计平均加权融合策略对源图像的特征图进行融合,最后将融合后的特征图进行重构获得融合图像。通过生成对抗网络实现了图像特征编码、自注意力学习、融合规则和融合特征解码的学习。TNO真实数据上的实验表明,学习到注意力单元体现了图像中显著的区域,能够较好地引导融合规则的生成,提出的算法在客观和主观评价上优于当前主流红外与可见光图像融合算法,较好地保留了可见光图像的细节信息和红外图像的红外目标信息。
Due to the lack of image saliency preserving in the existing fusion rules,a self-attention-guided infrared and visible light image fusion method is proposed.First,the feature map and self-attention map of the source images are learnt by the self-attention learning mechanism in the feature learning layer.Next,the selfattention map which can capture the long-distance dependent characteristics of the image is used to design average weighted fusion strategy.Finally,the fused feature maps are reconstructed to obtain the fused image,and the learning of image feature coding,self-attention mechanism,fusion rule,and fused feature decoding are realized by generative adversarial network.Experiments on TNO real-world data show that the learned self-attention unit can represent the salient region and benefit the fusion rule design,the proposed algorithm is better than SOAT infrared and visible image fusion algorithms in objective and subjective evaluation,and it retains the detailed information of visible images and infrared target information of infrared images.
作者
王天元
罗晓清
张战成
WANG Tianyuan;LUO Xiaoqing;ZHANG Zhancheng(School of Artificial Intelligence and Computer Science,Jiangnan University,Wuxi 214122,China;School of Electronic&Information Engineering,Suzhou University of Science and Technology,Suzhou 215009,China)
出处
《红外技术》
CSCD
北大核心
2023年第2期171-177,共7页
Infrared Technology
基金
国家自然科学基金(61772237),江苏省六大人才高峰项目(XYDXX-030)。
关键词
图像融合
自注意力
生成对抗网络
红外图像
深度学习
image fusion
self-attention
generative adversarial network
infrared images
deep learning
