ISGM-Fus: Internal structure-guided model for multispectral and hyperspectral image fusion

Fusing a high-resolution multispectral image (HR-MSI) and a low-resolution hyperspectral image (LR-HSI) is an important way to reconstruct a high-spatial-resolution hyperspectral image (HR-HSI). Most of the deep learning-based and model-based fusion methods have achieved remarkable success in recent years. However, the former usually requires massive training samples, which is restricted because of the limited training samples. The latter usually shows limited performance due to the absence of a training phase. To solve this problem, in this paper, we propose an internal structure-guided model for HR-MSI and LR-HSI fusion (ISGM-Fus) to increase the fusion accuracy without requiring additional training samples. Firstly, we design two spatial internal structure autoregressive models to capture the internal spatial information from both the LR-HSI and HR-MSI, including a local autoregressive model and a non-local autoregressive model. Secondly, we design a split-band low-rank prior to capture the internal spectral information from the LR-HSI. Finally, we apply the spatial and spectral information learned to design a new fusion model to increase the spatial resolution of HSI. The experimental results on three sets of HSIs show the superiority of ISGM-Fus to some other traditional and advanced fusion methods. The source code is available at https://github.com/Q-J-M/ISSLM-Fus.