Precipitation Retrieval Integrating Multiple Satellite Observations: A Dataset and a Framework

Multimodal satellite observations have been widely used for precipitation retrieval. Numerous retrieval algorithms and precipitation products have been developed based on these data. However, the integrated retrieval of multimodal data remains challenging due to the modality heterogeneity caused by different data characteristics reflecting precipitation patterns. To address these issues, effectively integrating multimodal data is crucial. We propose a framework named precipitation retrieval integrating multiple satellite observations and geographical information (PRMG), consisting of two networks for precipitation identification and estimation, respectively. Specifically, PRMG includes a multibranch fusion (MBF) module for integrating three types of satellite observations: infrared (IR), passive microwave (PMW), and spaceborne precipitation radar (PR), and a geographical information correction (GIC) module to incorporate the geographical information to calibrate precipitation features. We also collect a new precipitation retrieval dataset for multimodal precipitation retrieval, called Precipitation-MG, which includes satellite observations, corresponding geographical information, and precipitation products. Extensive experiments on Precipitation-MG demonstrate the effectiveness of the multimodal fusion method and the geographic correction method. The retrieval performance of PRMG achieves significant improvements compared to the Global Precipitation Measurement (GPM) currently in operation, i.e., Level-2 dual-frequency precipitation radar (DPR) and GPM microwave imager (GMI) combined (2B-CMB) product. The source code and dataset are publicly available at https://github.com/Zjut-MultimediaPlus/PRMG