Multimodal Sensor Fusion and Interpretable Deep Learning for Shallow Water Hazardous Geomorphology Features Recognition

Detailed and accurate identification of disaster geomorphology is important in the governance, development, and emergency decision-making of shallow water areas in estuaries. However, traditional disaster geomorphology monitoring processes are constrained by the low efficiency of multisource heterogeneous data fusion and the poor stability of recognition models. In this study, we seamlessly acquired shallow water bathymetry and topographic data using unmanned aerial vehicle (UAV) light detection and ranging (LiDAR) and unmanned surface vehicle (USV) multibeam sonar, and proposed an attention-aware multimodal multiscale cross-fusion geomorphology recognition network called AMMNet. First, we adopted a three-stream encoder-decoder structure and designed an attention mechanism-based multimodal fusion module (AMFM) for riverbank morphology and texture to better utilize the fused enhanced features at multiple extraction stages. Second, we added an atrous spatial pyramid pooling (ASPP) module between the encoder and decoder to aggregate feature information of disaster geomorphology at different scales. Finally, an integrated approach combining gradient-weighted class activation mapping (Grad-CAM) and Shapley additive explanations (SHAPs) was employed to analyze the decision-making process of the model and disaster mechanisms. To evaluate the performance of the AMMNet model, it was compared comprehensively with other typical recognition models using an UAV-USV dataset containing 16 types of morphological and textural features. The results of this study demonstrate the superiority of the AMMNet model in recognizing hazard features compared with other models. Specifically, the intersection over union (IOU) for riverbank collapse, scarps, scour pits, and dunes reached 93.7%, 92.2%, 87.2%, and 94.2%, respectively. Furthermore, this study found that low entropy and a high topographic position index (TPI) were the main triggering factors for riverbank collapse. In addition, low elevation and high contrast have been found to increase the probability of dune-related hazards. This study provides a significant reference for accurate shallow water geomorphological identification and timely disaster risk warnings.