MSCSCformer: Multiscale Convolutional Sparse Coding-Based Transformer for Pansharpening

With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution multispectral images (LRMSs) with high-resolution panchromatic (PAN) images, has gained considerable attention. However, current deep-learning (DL) methods have limitations in capturing global long-range dependencies and incorporating spectral characteristics across different spectral bands of multispectral (MS) images. Additionally, model-based approaches do not effectively utilize the multiscale information between LRMS and HRMS data, limiting their further performance enhancement. To address these limitations, we propose a new observation model based on multiscale convolutional sparse coding (MS-CSC) and design a novel multiscale hybrid spatial-spectral transformer (MSHST) for the unfolding networks. The MS-CSC-based observation model aims to fuse multiscale information, while the MSHST incorporates spatial self-attention to capture global long-range dependencies and spectral self-attention to capture the interband correlation. Experimental results demonstrate the superiority of our method over other state-of-the-art approaches in both reduced-resolution and full-resolution evaluations. Ablation experiments further validate the effectiveness of the proposed multiscale model and MSHST. Code is available at https://github.com/Eternityyx/MSCSCformer.