Dual-modality image feature fusion network for gastric precancerous lesions classification

Gastric precancerous conditions are closely linked to the development of gastric cancer. However, the detection of gastric precancerous lesions (GPL) is limited by the indistinct symptoms and the low detection rate of microscope images. This paper proposes an RGB and Hyperspectral Dual-modality imaging Feature Fusion Network (DuFF-Net) to improve the classification accuracy of GPL. To fully exploit information of different modality images, we customize a dual-stream ResNet-based model for feature sharing and fusion. Skip-Connections are added between inter-path of networks to achieve information interaction. In the decision step, we adopt the SE-based attention module and Pearson Correlation to highlight and select effective features. Experimental results show that the DuFF-Net increases the screening accuracy to 96.15 % for two types of gastric precancerous tissues with high morphological similarity. Furthermore, our approach reduces the labeling workload for classification tasks by approximately 50 %. These findings provide valuable guidance for the screening and subsequent lesion segmentation of GPLs.