MAG-Net: Physics-Aware Multi-Modal Fusion of Geostationary Satellite and Radar for Severe Convective Precipitation Nowcasting

Abstract: Radar-based convective precipitation nowcasting suffers from rapid performance degradation beyond 30 minutes due to missing thermodynamic variables. Existing deep learning models also face blurring effects, training instability, and limited interpretability. To address this, we propose MAG-Net, a Physics-Aware Multi-modal Attention-guided Generator Network. It integrates radar dynamics with selected geostationary satellite channels (IR 10.8, WV 7.1, BTD) to incorporate thermodynamic and microphysical precursors. MAG-Net features a Dual-Stream Encoder for heterogeneous modalities and a Symmetric Dual-Head Decoder optimizing reflectivity regression and event probability via an uncertainty-weighted multi-task strategy. Furthermore, an inference-time Gradient-Preserving Fusion (GPF) strategy combines probabilistic constraints with regression details for better high-frequency texture retention. Experiments on a large-scale dataset (2018-2023) over southeastern China show MAG-Net outperforms deterministic (e.g., CPrecNet) and generative (e.g., DGMR) baselines. Specifically, it improves CSI40 by 0.083 (0.172 to 0.255) over CPrecNet, enhancing intense convective echo detection. Finally, Integrated Gradients (IG) analysis reveals the model's reliance on satellite inputs increases with forecast lead time and convective intensity, confirming that satellite data captures critical precursors for severe weather prediction.

• The paper introduces MAG-Net, a physics-aware multi-modal framework that fuses satellite channels and radar data to improve severe convective precipitation nowcasting.
• It employs a dual-stream hierarchical encoder with cross-modal attention and a symmetric dual-head decoder, ensuring both pixel-wise regression and categorical event prediction.
• Experimental results show significant gains over radar-only methods, reflected in improved CSI40, RMSE/MAE metrics and enhanced forecasting of convective initiation and dissipation.

Physics-Aware Multi-Modal Fusion for Severe Convective Precipitation Nowcasting: An Authoritative Analysis of MAG-Net

Introduction

The MAG-Net architecture presents a comprehensive approach to convective precipitation nowcasting by synergistically leveraging multi-modal geostationary satellite and radar data. It directly targets a longstanding limitation in operational nowcasting: radar-based extrapolation, though effective for short lead times, is fundamentally blind to thermodynamic precursors of convective initiation and incapable of capturing dissipation without explicit environmental information. The underlying philosophy driving MAG-Net is the physics-principled selection and fusion of satellite channels that provide unique thermodynamic and microphysical insights, enforced via architectural innovations that ensure model interpretability and mitigate regression blurring.

Figure 1: Schematic breakdown of MAG-Net’s dual-head architecture and gradient-preserving multi-modal fusion strategy.

Methodological Innovations

Physics-Aware Multi-Modal Fusion

MAG-Net’s design begins with the targeted selection of three satellite channels: IR 10.8 μm (cloud-top temperature and updraft proxy), WV 7.1 μm (mid-tropospheric humidity), and BTD (10.8–12.0 μm, cloud phase discrimination). The fusion is conducted via a Dual-Stream Hierarchical Encoder, with radar and satellite streams aligned through temporal synchrony and processed independently to preserve their statistical heterogeneity. At the latent bottleneck, a cross-modal attention mechanism allows radar features to query satellite information, thus focusing the combined representation on meteorologically relevant regions.

Symmetric Dual-Head and Gradient-Preserving Inference

A symmetric dual-head decoder enables the joint optimization of pixel-wise regression (intensity) and categorical classification (event probability), utilizing an uncertainty-weighted multi-task loss. Critically, at inference time, a Gradient-Preserving Fusion (GPF) strategy decomposes outputs into frequency bands—low-frequency structure from classification is fused with high-frequency texture from regression—yielding both structural coherence and fine-scale detail absent in MSE-centric regression models.

Figure 2: Quantitative performance: MAG-Net (red stars) exhibits substantial skill gains, especially at high reflectivity thresholds, and optimizes the POD–FAR trade-off.

Experimental Analysis

Quantitative and Categorical Skill

Evaluated on a six-year, high-resolution dataset from southeastern China, MAG-Net demonstrates skill improvements over both deterministic (CPrecNet, SimVPv2) and generative (DGMR) baselines. The architecture achieves a CSI$_{40}$ of 0.255, an absolute gain of 0.083 over the best radar-only reference, and maintains competitive RMSE/MAE. While pure regression variants reduce pixel error, they underperform for convective extremes, affirming the efficacy of the dual-head, GPF approach for structural fidelity.

Figure 3: MAE and RMSE statistics show the superior stability and reduced error accumulation for MAG-Net versus radar-only and naive fusion models.

Figure 4: Spectral analysis reveals MAG-Net maintains high-frequency (small-scale) energy, improving agreement with ground truth compared to regression blurring.

Qualitative and Mechanistic Interpretability

MAG-Net outperforms baselines in capturing convective initiation and dissipation, validated visually during challenging scenarios where single-modality or regression-centric models consistently miss or diffuse emergent storm cores.

Figure 5: Visual comparison during a convective initiation event: MAG-Net captures emerging cores and their intensification aligned with satellite precursors.

Channel ablation reveals that IR 10.8 μm is indispensable for high-intensity echo capture, BTD is critical for false alarm suppression, and WV 7.1 μm grants broader environmental context. These findings are rigorously borne out in the regression error and categorical skill experiments.

Figure 6: Channel ablation quantifies the unique contribution of each satellite channel to both detection and error suppression, consistent with meteorological physics.

Attributive Interpretation via Integrated Gradients

A temporal breakdown of Integrated Gradients (IG) attributions demonstrates MAG-Net’s increasing reliance on satellite predictors as forecast lead time grows or target intensity rises, confirming its learned physical alignment: radar is dominant for short, low-intensity, and advection-driven events; satellite dominates for long lead times and severe convection.

Figure 7: Integrated Gradients analysis: satellite attributions rise with lead time and reflectivity, with IR 10.8 μm consistently prioritized for intense events.

Spatial explanations during event evolution illuminate the model’s identification of early IR/BRT and BTD patterns, aligning predictive attention with key physical regimes of developing or decaying convection.

Figure 8: Convective initiation case study. Only MAG-Net, by leveraging multi-modal cues, anticipates the nascent cell in region “A”.

Figure 9: Attribution heatmaps highlight spectral and spatial regions that drive initiation prediction, dominated by IR gradients and BTD regimes tied to convective physics.

Broader Implications

The demonstrated gains have significant operational and conceptual ramifications. MAG-Net’s parallel fusion and GPF framework enables sub–100 ms inference (single GPU, 90-min, 9-frame forecasts, batch size 16), making it deployable for real-time systems. The physics-aware channel selection ensures robust diurnal performance and cross-domain generalization potential. The explicit architectural interpretability—via multi-task learning and post-hoc IG analysis—addresses a longstanding barrier to deep learning adoption in mission-critical meteorological forecasting.

Theoretically, MAG-Net’s success reiterates the limitations of advection-only and regression-blurring models, establishing that optimal nowcasting demands multimodal, physics-aligned representation learning. It also provides a rigorous template for future hybrid architectures seeking to blend remote-sensing physics with modern deep sequence modeling.

Conclusion

MAG-Net introduces a rigorous, physics-constrained multi-modal approach to severe convective precipitation nowcasting, outperforming prior SOTA models in both categorical skill and the preservation of physically meaningful details. The explicit use of meteorologically grounded satellite channels, enforced via dual-head learning and GPF, enables MAG-Net to anticipate convective initiation and dissipation scenarios that have remained elusive to radar-only models. Importantly, the interpretability and computational tractability of the framework make it immediately relevant for operational integration, while its methodological paradigm sets a foundation for further physical–deep learning convergence in environmental AI.