CSFusionNet: An End-to-end Image Fusion Network Based on Cascade-Skip Learning

• Bang Cheng,
• Jianghua Cheng,
• Tong Liu,
• Xiaobing Luo,
• Xiangyu Du,
• Liang Zhang,
• Tao Wang,

Vol. 18, No. 11, November 30, 2024
10.3837/tiis.2024.11.007, Download Paper (Free):

• end-to-end image fusion network
• cascade-skip learning
• visual saliency features
• Feature Maps

Abstract

In this paper, we proposed an end-to-end image fusion network based on cascade-skip learning, named CSFusionNet. Inspired by the Unet++ network model, the motivation is to reduce the number of network parameters, make full use of the features extracted by middle network layers, and improve the efficiency of training. Firstly, the input of the proposed method is two-way of source images, and the images are respectively extracted features. The feature maps are generated through their respective encoding and are fused in the decoding part. Secondly, the method proposed is to cascade the feature maps in different layers, and then output them through up-sampling. The first three input layers extract high-dimensional information from the source images and connect to the features of the last three layers before the output layer by cascade-skip learning. Therefore, the details of features can be well preserved in the fused image. Finally, a large number of experiments were carried out in two types of fusion tasks (Vis-Ir image fusion and Multi-Focus image fusion) with three datasets. With the comparison of the state-of-the-art image fusion algorithms, the visual saliency features of the target in the fused image are more prominent in the proposed method, the quantitative results also verify that our proposed network has achieved comparable or even better performance. Among the four widely used evaluation metrics, our method can obtain the optimal or suboptimal value.

Statistics

Cite this article