Precipitation nowcasting of satellite data using physically-aligned neural networks (2511.05471v2)
Abstract: Accurate short-term precipitation forecasts predominantly rely on dense weather-radar networks, limiting operational value in places most exposed to climate extremes. We present TUPANN (Transferable and Universal Physics-Aligned Nowcasting Network), a satellite-only model trained on GOES-16 RRQPE. Unlike most deep learning models for nowcasting, TUPANN decomposes the forecast into physically meaningful components: a variational encoder-decoder infers motion and intensity fields from recent imagery under optical-flow supervision, a lead-time-conditioned MaxViT evolves the latent state, and a differentiable advection operator reconstructs future frames. We evaluate TUPANN on both GOES-16 and IMERG data, in up to four distinct climates (Rio de Janeiro, Manaus, Miami, La Paz) at 10-180min lead times using the CSI and HSS metrics over 4-64 mm/h thresholds. Comparisons against optical-flow, deep learning and hybrid baselines show that TUPANN achieves the best or second-best skill in most settings, with pronounced gains at higher thresholds. Training on multiple cities further improves performance, while cross-city experiments show modest degradation and occasional gains for rare heavy-rain regimes. The model produces smooth, interpretable motion fields aligned with numerical optical flow and runs in near real time due to the low latency of GOES-16. These results indicate that physically aligned learning can provide nowcasts that are skillful, transferable and global.
Sponsor
Summary
- The paper introduces TUPANN, an innovative neural network that integrates satellite rainfall data with explicit physical conditioning, enhancing interpretability in precipitation nowcasting.
- It employs a novel Variational Encoder-Decoder and a MaxViT Transformer to produce smooth, physically plausible motion fields for multi-horizon forecasts.
- Evaluation across diverse climate regimes shows TUPANN outperforms baseline models in CSI and HSS, demonstrating its transferability and operational relevance.
Physically-Aligned Neural Networks for Satellite Precipitation Nowcasting
Introduction and Motivation
The paper introduces TUPANN (Transferable and Universal Physics-Aligned Nowcasting Network), a neural architecture tailored for precipitation nowcasting using only satellite-based rainfall estimates. The motivation is twofold: (i) radar data, commonly used for high-precision nowcasting, are geographically limited, excluding many regions at risk from extreme precipitation; (ii) deep learning models for nowcasting often lack explicit physical structure, impeding interpretability and trustworthiness in operational forecasting scenarios.
TUPANN attempts to bridge these gaps through satellite data integration and explicit physical conditioning. By utilizing GOES-16 Rain Rate Quantitative Precipitation Estimation (RRQPE) and IMERG datasets, TUPANN enables nowcasting across diverse climate regimes, using modular neural components supervised to respect physically plausible motion fields as derived from numerical optical flow.
Dataset Characterization and Regional Analysis
The study leverages satellite-based datasets (GOES-16 RRQPE and IMERG) covering South and North America, focusing on four distinct climate regions: Rio de Janeiro (coastal, orographic convection), Manaus (rainforest, monsoon), Miami (coastal, sea-breeze thunderstorms), and La Paz (highland, convective storms). The satellite products differ both spatially and temporally: RRQPE provides 2 km resolution at 10 min intervals and near-real-time latency; IMERG offers 10 km resolution every 30 min, but is designed for retrospective validation.
Analysis of precipitation intensity distributions shows substantial regional variability, with Manaus experiencing the highest frequency of heavy-rain events, and seasonal cycles most pronounced in La Paz and Rio de Janeiro.
Figure 1: Proportion of observations above different precipitation thresholds in the GOES-16 RRQPE dataset highlights the heavy rainfall dominance in Manaus and intermediate levels in Rio de Janeiro and La Paz.
Figure 2: Accumulated precipitation in GOES-16 RRQPE reveals marked seasonal variability across regions and delineates train/validation/test splits.
TUPANN Architecture and Physical Alignment
TUPANN operates by decomposing the forecasting task into physically interpretable components:
- Variational Encoder-Decoder (VED): Instead of reconstructing raw frames, the VED infers spatially-resolved motion fields (2D optical flow) and intensity correction fields from past satellite images. Supervision is enforced via explicit comparison to motion fields computed by numerical optical flow methods (DARTS for GOES-16, Lucas–Kanade for IMERG).
- Lead-Time-Conditioned MaxViT Transformer: The latent state output from the VED is evolved in time via a MaxViT-based vision transformer, with lead time embedded as a conditioning signal, enabling direct multi-horizon forecasting within a single model.
- Differentiable Advection Operator: The decoder outputs are coupled to a differentiable warp operator, advecting precipitation forward according to learned motion and intensity fields.
Figure 3: TUPANN architecture comprises VED and MaxViT modules, with motion and intensity fields extrapolated by a differentiable warp function.
Ground truth for supervision is derived by numerically advecting past images via optical flow and then deriving the intensity error field for each pair.
Figure 4: Ground truth motion fields and intermediate advected frames are computed using optical flow, ensuring physically meaningful targets for the VED output.
Losses at each module are physically motivated: the VED is penalized via L1, cosine similarity, and KL regularizations against numerical motion and intensity fields; MaxViT is optimized on the L1 residual in original image space, with gradients traversing through the decoder and transformer modules.
Experimental Design and Baseline Comparisons
TUPANN is evaluated against a suite of state-of-the-art nowcasting models:
- PySTEPS (Lucas–Kanade/DARTS): Optical flow-based numerical baselines.
- Earthformer: Space-time transformer (Cuboid Attention) without explicit physical conditioning.
- NowcastNet: Physics-inspired GAN with implicit motion estimation.
- CasCast: Cascaded latent diffusion model refining coarse deterministic predictions.
Critical Success Index (CSI) and Heidke Skill Score (HSS) over 4–64 mm/h thresholds serve as primary metrics. TUPANN and Earthformer are tuned via validation maximization on CSI in Rio de Janeiro; other model hyperparameters remain consistent with published defaults.
Results: Skill, Transferability, and Interpretability
TUPANN achieves competitive or superior CSI and HSS across all tested regions and thresholds, with particular advantage at higher precipitation intensities. In Rio de Janeiro and Miami, TUPANN systematically outperforms Earthformer, NowcastNet, and CasCast, especially on pooled and extreme thresholds.
Figure 5: Mean CSI across lead times shows TUPANN's sustained skill versus baselines in all climate regimes.
TUPANN's explicit motion field supervision results in smoother, physically plausible flow fields compared to the patchy, less coherent estimates of NowcastNet.
Figure 6: Comparative visualization of motion fields shows TUPANN's outputs closely match numerical DARTS flows, whereas NowcastNet/Evolution Network fields deviate from physical intuition.
Visualization confirms that although generative models add textural sharpness, they can introduce spurious artifacts. TUPANN and Earthformer yield smoother fields and superior timing/location prediction for heavy-rain events.
Figure 7: Model predictions for a Manaus rain event over 3h intervals illustrate TUPANN's superior skill in rain movement and intensity, and the trade-offs between sharpness and accuracy.
Ablation experiments reveal:
- GAN-TUPANN: Adding adversarial training improves visual sharpness but produces inconsistent or degraded CSI at higher thresholds—sharper does not mean more skillful.
- Cross-City Training: Models trained on Rio de Janeiro generalize modestly to other cities, with performance declines ≤20%. In Miami, the Rio-trained model matches or exceeds local models at extreme thresholds.
- Multi-City Training: Pooled training across all cities enhances skill and transferability, particularly for rare heavy-rain regimes.
Operational deployment is feasible on commodity GPUs (RTX 3080, 16GB RAM), with end-to-end latency <3 minutes for near-real-time forecasts, enabling direct integration in disaster early warning workflows. The model is currently operational for Rio de Janeiro's INEA agency.
Extension to IMERG and Broader Satellite Products
When transferred to the coarser, more delayed IMERG dataset, TUPANN maintains leadership in unpooled CSI at all thresholds and most lead times, generalizing robustly despite spatial/temporal resolution change.
Figure 8: For the IMERG dataset in Rio de Janeiro, TUPANN sustains skill across most lead times, exceeding NowcastNet except at 150 min.
Figure 9: IMERG prediction sample illustrates differences in spatial resolution, reinforcing TUPANN's adaptability to diverse satellite products.
Limitations and Future Work
Current limitations center on:
- Dependency on geostationary satellites (retraining for Himawari, Meteosat, etc. is needed for global coverage).
- Reliance on numerical optical flow, which may not fully capture non-Lagrangian or multi-layer precipitation dynamics.
- CSI/HSS metric degradation at lead times >2 hours suggests strong predictability limits; calibrated probabilistic post-processing could be explored for uncertainty quantification.
- GAN-driven adversarial improvements do not systematically translate to operational skill.
Future development directions include assimilation of additional inputs (cloud imagery, microwave retrievals), direct transfer learning for new continents, ensemble approaches for uncertainty quantification, and deeper integration with operational meteorological infrastructure.
Conclusion
TUPANN demonstrates that explicit physical alignment in neural architectures, when combined with satellite-only precipitation data, delivers interpretable, skillful, and operationally scalable nowcasting across diverse climatic regimes. Its modular design—VED supervised by numerical flow and MaxViT temporal embedding—supports global transferability and addresses key operational constraints in radar-sparse regions. Strong CSI/HSS performance, physical interpretability, and demonstrated operational deployment suggest that physically-aligned neural nowcasting is a promising frontier for equitable, global disaster risk reduction.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What is this paper about?
This paper is about “nowcasting,” which means making short-term weather forecasts (from 10 minutes up to 3 hours ahead). The authors built a new computer model, called TUPANN, that predicts rain using only satellite data. Unlike many other models, TUPANN is designed to follow real-world physics about how rain clouds move and change, which makes its predictions clearer and more trustworthy—especially in places that don’t have good weather radar.
The main questions the researchers asked
- Can a satellite-only model make accurate short-term rain forecasts without relying on radar, which many regions lack?
- Will adding physical understanding (like how clouds move) into a neural network improve accuracy and make the model’s behavior easier to interpret?
- Can one model work across different climates and still do well at predicting heavy rain?
How did they do it? Methods in simple terms
Think of forecasting rain like predicting how a moving, changing picture will look a few frames ahead. TUPANN breaks this problem into three easier pieces:
- Learning how the rain pattern moves
- The model watches recent satellite images and figures out how the rain “flows” across the map.
- This uses “optical flow,” a technique like tracking how dots in a video shift from one frame to the next. Imagine watching a swarm of fireflies and following where each one goes—optical flow does that for pixels.
- TUPANN is trained to match motion fields from trusted optical-flow algorithms (like DARTS or Lucas–Kanade), so its idea of “movement” aligns with real physics.
- Summarizing what’s happening inside the images
- The model uses a “variational encoder–decoder” (VED).
- Encoder–decoder is like compressing a story into a short summary (the “latent state”) and then rebuilding useful parts from that summary. Here, the summary helps predict two things for the next frame:
- A motion field (how the rain shifts)
- An intensity correction (how the rain gets stronger or weaker)
- Evolving the summary forward in time
- The model passes the summary into a “Transformer” called MaxViT, which is good at spotting patterns across the whole image efficiently.
- It is “lead-time conditioned,” meaning the model is told how far ahead it needs to predict (like 20 min vs. 90 min), so one network can handle many future times.
- Finally, a “warp” (advection) step slides the last image along the predicted motion and adjusts the intensity—like moving a transparent sheet over a map and darkening or lightening parts to match where the rain will go.
Training and checking performance
- The model is trained in two stages: first the motion/intensity learner (VED), then the time-step predictor (MaxViT).
- It’s tested in four different places and climates: Rio de Janeiro (Brazil), Manaus (Brazil), Miami (USA), and La Paz (Bolivia).
- It uses satellite rainfall products: GOES‑16 RRQPE (every 10 minutes, higher resolution, near real-time) and IMERG (every 30 minutes, lower resolution, delayed but widely used).
- Accuracy is measured with CSI and HSS scores:
- CSI (Critical Success Index): how well predicted “rainy pixels” overlap the actual rainy pixels. Higher is better.
- HSS (Heidke Skill Score): compares the forecast to random chance. Higher is better.
- They check multiple rain rates (from 4 to 64 mm/h) and multiple lead times (10 to 180 minutes).
What did they find?
TUPANN performed strongly across cities, rain intensities, and lead times.
- Overall accuracy: TUPANN is the best or second-best model in most tests, beating or matching strong baselines like Earthformer, NowcastNet, PySTEPS, and CasCast.
- Heavy rain: TUPANN often does better at higher rain rates (like 32–64 mm/h), which are the most important for flood warnings.
- Different climates: It works well in very different places—a rainforest (Manaus), a high-altitude city (La Paz), a coastal monsoon region (Rio), and subtropical storms (Miami).
- Interpretable physics: The “motion fields” TUPANN produces are smooth and match what physics-based optical-flow methods show. This makes the model easier to trust because we can see how it thinks rain is moving.
- Real-time potential: Because GOES‑16 satellites have low delay (about 5 minutes), TUPANN can run near real time.
- Training across cities: Training on multiple cities usually improves results, helping the model learn more general patterns.
- Transferability: Training in one city and testing in another only modestly reduces performance; sometimes it even helps for rare heavy-rain events.
- Other data source (IMERG): On IMERG (a different satellite dataset), TUPANN remains competitive and often best, showing good generalization beyond its main training source.
Why it matters
Many regions most at risk from floods don’t have good radar coverage. TUPANN shows we can make useful, short-term rain forecasts using satellites alone—and do it in a way that respects physical rules about motion and intensity. That means:
- Better early warnings: More places can get timely, accurate alerts for heavy rain and flash floods.
- Trustworthy forecasts: Because the model’s motion fields align with physical algorithms, forecasters can understand and trust the predictions more.
- Global scalability: Satellites cover huge areas, so this approach can work almost anywhere.
- Practical use: The model runs quickly and uses widely available satellite data, making it promising for real-time operations.
In the future, this kind of physically aligned learning could help build global, reliable nowcasting systems. It might also be combined with other data (like ground sensors or numerical weather models) to improve extreme-event predictions even more, while keeping forecasts interpretable and fast.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
The paper leaves several concrete gaps and open questions that future research could address:
- Ground-truth fidelity: Model training and evaluation primarily use satellite-derived RRQPE (and IMERG) rather than independent ground truth (gauges or gauge-corrected radar). Quantify and correct for label noise/bias and assess how RRQPE/IMERG errors propagate into learned dynamics and verification scores.
- Cross-sensor/domain robustness: Generalization beyond GOES-16 (e.g., Himawari-8/9, Meteosat) is untested. Evaluate transfer under different sensors, scan cadences, viewing geometries, resolutions, and parallax characteristics, including over oceans and high latitudes.
- Optical-flow supervision dependence: The motion “ground truth” comes from Lucas–Kanade or DARTS, which have known failure modes (e.g., large displacements, growth/decay, occlusions, parallax). Perform sensitivity analyses to the chosen optical-flow method and its hyperparameters, quantify error propagation into the VED, and test whether the model can overcome optical-flow biases.
- Physical consistency beyond advection: The framework approximates continuity via advection plus an unconstrained “intensity correction” source term. Evaluate explicit mass/moisture conservation and non-negativity constraints; investigate learning or enforcing PDE terms (e.g., divergence-driven growth/decay) instead of relying on a free correction field.
- Exposure bias from teacher forcing: MaxViT is trained assuming X_{k−1} equals the true frame (teacher forcing), but inference uses recursive predictions. Assess scheduled sampling or multi-step training to mitigate error compounding at long lead times.
- Lack of probabilistic forecasts: Outputs are deterministic. Develop calibrated probabilistic nowcasts (ensembles, Bayesian VED sampling, dropout, diffusion) and verify reliability (Brier score, reliability diagrams, CRPS), especially for extremes.
- Extreme-event skill and rarity: CSI at 64 mm/h remains low across models. Explore extreme-focused training (class/threshold-aware losses, focal or quantile losses), rare-event resampling, and event-based metrics (POD, FAR, ETS) and spatial scores (FSS/SAL) tailored to severe precipitation.
- Lead-time conditioning design: One-hot lead-time embeddings may limit extrapolation and interpolation across unseen horizons or irregular cadences. Test continuous/time-encoding schemes and variable timestep handling.
- Input information content: Only a single satellite-derived rain-rate channel is used. Evaluate adding ABI multispectral channels, cloud-top/IR features, lightning, and static predictors (topography, land–sea mask) to improve convective initiation/growth prediction and orographic effects.
- Environmental/NWP coupling: No environmental predictors (e.g., CAPE, vertical shear, low-level winds) are used. Quantify gains from lightweight NWP or reanalysis conditioning while retaining a satellite-only fallback mode.
- Motion-field validation: Motion interpretability is shown qualitatively. Provide quantitative motion-vector verification (endpoint/angular errors) against trusted references (e.g., radar-tracked echoes where available) and analyze failure modes.
- Joint end-to-end training: VED and MaxViT are trained sequentially with the VED frozen. Test joint fine-tuning or multi-task training to co-optimize latent dynamics, motion fields, and reconstruction.
- Warp operator fidelity: The fixed differentiable advection operator’s numerical properties (mass conservation, stability, boundary handling, non-negativity) are not assessed. Compare fixed vs learnable warping and quantify artifacts at tile boundaries.
- Temporal robustness: Handling of missing frames, latency jitter, and irregular observation intervals is not evaluated. Develop and test robustness strategies for operational data gaps and variable cadences.
- Spatial scalability and tiling: Inference is reported for 512 km tiles; scaling to continental/global domains requires tiling and stitching. Assess boundary effects, context requirements across tiles, and computational throughput on operational hardware.
- Domain shift across climates/seasons: Cross-city tests use four regions; broader seasonal/diurnal and inter-annual shifts are not characterized. Evaluate generalization across seasons, ENSO phases, and rare regimes (e.g., dry spells transitioning to convective outbreaks).
- Label-set curation bias: Event-based sampling and exclusion of light precipitation (IMERG) may bias training/evaluation toward heavier events. Quantify impacts on light-rain skill and consider balanced/resampled training protocols.
- Metrics breadth: Verification focuses on CSI and HSS at fixed thresholds (POOL1/POOL4). Add continuous scores (MAE/RMSE), spatial neighborhood metrics (FSS), displacement-aware scores, and accumulation-based metrics (e.g., 1–3 h totals) relevant to hydrologic impacts.
- Convective initiation and storm lifecycle: The framework captures advection well but explicit modeling of initiation/mergers/splits is not analyzed. Introduce cell-tracking diagnostics and lifecycle-aware evaluation to identify growth/decay performance limits.
- GAN/diffusion alternatives: The GAN head improves sharpness but yields mixed skill. Systematically compare physics-aligned diffusion models and adversarial training schedules for satellite-only inputs, measuring both perceptual realism and deterministic/probabilistic skill.
- Resolution mismatch and bias correction: Transferring between RRQPE (2 km/10 min) and IMERG (10 km/30 min) is nontrivial. Evaluate resolution-aware architectures, super-resolution modules, and cross-product bias correction to avoid conflating resolution with skill.
- Parallax and high terrain effects: Geostationary parallax and orography (e.g., La Paz) can misalign surface precipitation and apparent motion. Quantify parallax impacts and test parallax correction or 3D-informed motion priors.
- Data augmentation and sample efficiency: The impacts of augmentation (rotations, flips, intensity scaling), context length T, and forecast horizon T_f are not explored. Provide ablations to guide practitioners on data/compute trade-offs.
- Code, data, and reproducibility: Operational uptake requires reproducible pipelines. Clarify code availability, preprocessing (e.g., clipping, normalization), event selection thresholds, and random seeds to enable independent verification.
- Downstream utility: No evaluation against impact-relevant outcomes (e.g., flood proxies, warning lead time). Couple predictions with hydrologic models or hazard proxies to assess practical gains in early warning contexts.
Glossary
- Adaptive Fourier Neural Operator (AFNO): A neural-operator architecture that learns solution operators for PDEs using Fourier transforms, enabling efficient global forecasting. "FourCastNet \citep{10.1145/359(2979.35934)12} employs an Adaptive Fourier Neural Operator architecture \citep{Guibas2021AdaptiveFN}"
- Advection operator: A function that transports a field along a velocity (motion) field to predict its future state. "a differentiable advection operator reconstructs future frames."
- Advanced Baseline Imager (ABI): The imaging sensor on the GOES‑R series that provides multi-band observations used to derive precipitation products. "GOESâR Advanced Baseline Imager Rain Rate Quantitative Precipitation Estimation (RRQPE)"
- CasCast: A cascaded latent diffusion framework that refines deterministic forecasts to add small-scale structure and improve extremes. "CasCast \citep{10.5555/369(2070.36927)03} extends this approach through a cascaded LDM framework"
- Continuity equation: A conservation law expressing the continuity of mass (or other conserved quantity) in a flow, used as a physical constraint in learning. "by enforcing the continuity equation through an explicit loss between predicted physical terms and those derived from a numerical optical-flow method"
- Critical Success Index (CSI): A categorical verification metric that measures the overlap between predicted and observed events, ignoring true negatives. "we report critical success indices (CSI) and Heidke skill scores (HSS)"
- Cuboid Attention: An attention mechanism that operates on 3D blocks to capture spatiotemporal interactions efficiently. "Earthformer \citep{10.5555/360(0270.36021)11} adapts the Transformer framework for general Earth-system forecasting through a modified Cuboid Attention mechanism"
- DARTS (optical flow): A spectral optical-flow algorithm that solves the optical-flow equation in Fourier space, tailored to radar imagery. "and DARTS, a spectral method tailored to radar imagery that solves the opticalâflow equation in Fourier space."
- Earthformer: A Transformer-based spatiotemporal model for Earth-system forecasting using specialized attention mechanisms. "Earthformer \citep{10.5555/360(0270.36021)11} adapts the Transformer framework for general Earth-system forecasting"
- Ensemble-based probabilistic forecasts: Forecasts produced by running multiple perturbed simulations to quantify uncertainty. "stochastic perturbations to enable ensemble-based probabilistic forecasts \citep{https://doi.org/10.1002/wrcr.20536}"
- Generative Adversarial Network (GAN): A generative modeling framework with a generator and discriminator trained adversarially to match data distributions. "Generative Adversarial Networks (GANs), including DGMR \citep{Ravuri2021} and NowcastNet \citep{zhang2023}"
- Geostationary satellites: Satellites that remain fixed relative to Earth’s rotation, providing continuous regional coverage with low latency. "by leveraging geostationary satellites, which provide global coverage with near realâtime latency"
- GOES‑16: A NOAA geostationary satellite providing high-frequency imagery over the Americas, used here as the primary data source. "We present TUPANN (Transferable and Universal Physics-Aligned Nowcasting Network), a satellite-only model trained on GOES-16 RRQPE."
- Heidke Skill Score (HSS): A categorical forecast score that compares performance against random chance. "we report critical success indices (CSI) and Heidke skill scores (HSS)"
- IMERG: Integrated Multi‑satellitE Retrievals for GPM, a multi-satellite precipitation product used for evaluation. "we also use the Integrated MultiâsatellitE Retrievals for GPM (IMERG) Final Run product \citep{IMERG}."
- Kullback–Leibler divergence (KL divergence): A measure of discrepancy between probability distributions used to regularize latent variables. "Finally, a Kullback-Leibler divergence term is added to ensure the regularity of the learned latent space."
- Lagrangian extrapolation: Forecasting by advecting observed features along estimated motion pathways. "Early approaches relied primarily on Lagrangian extrapolation of radar echoes"
- Latent Diffusion Model (LDM): A diffusion-based generative model that operates in a compressed latent space to produce high-fidelity samples. "PreDiff \citep{10.5555/3666122.3669561} employs a Latent Diffusion Model (LDM) with a knowledge-alignment mechanism"
- Lead‑time conditioning: Conditioning a model on the forecast horizon so a single network can produce predictions for multiple lead times. "a leadâtimeâconditioned MaxViT evolves the latent state"
- Lucas–Kanade (LK) optical flow: A classical optical-flow method solving local least-squares under small-displacement assumptions. "LucasâKanade (LK), which solves a local leastâsquares problem under the assumption of small displacements"
- MaxViT: A vision transformer combining windowed and grid attention to capture global context with linear complexity in image size. "We adopt MaxViT \citep{Tu2022Nov}, which combines local and grid attention to efficiently capture global context while avoiding quadratic attention cost."
- Mesoscale convection: Organized convective systems on 10–1000 km scales that drive significant precipitation. "The dominant precipitation processes include orographic and mesoscale convection in Rio \citep{silva2022rio}"
- NowcastNet: A GAN-based nowcasting architecture that estimates motion/intensity fields and refines predictions adversarially. "NowcastNet augments the GAN structure with an Evolution Network that estimates motion and intensity fields used to generate intermediate predictions before adversarial refinement."
- Numerical weather prediction (NWP): Physics-based numerical models of the atmosphere used for weather forecasting. "enhanced with downscaled numerical weather prediction input."
- Optical flow: Estimation of pixel-wise apparent motion between images; used to infer precipitation motion fields. "An optical flow algorithm is able to infer motion fields between two images"
- Orographic: Pertaining to terrain-induced lifting that can trigger or enhance precipitation. "orographic and mesoscale convection in Rio \citep{silva2022rio}"
- Physics‑Informed Neural Networks (PINNs): Neural networks trained with PDE-based loss terms to enforce physical laws. "Physics-Informed Neural Networks (PINNs) \citep{Raissi2019-cw} exemplify this approach"
- PreDiff: A diffusion-based nowcasting model that applies knowledge alignment for physically plausible predictions. "PreDiff guides denoising steps toward physically plausible predictions by aligning generated intensities"
- PySTEPS: An open-source library for optical-flow-based precipitation nowcasting and extrapolation. "PySTEPS \citep{gmd-12-4185-2019} has become a widely adopted open-source Python library providing a reproducible platform for numerical nowcasting."
- RRQPE (Rain Rate Quantitative Precipitation Estimation): A GOES‑R product providing satellite-derived rain-rate estimates at high frequency. "GOESâR Advanced Baseline Imager Rain Rate Quantitative Precipitation Estimation (RRQPE)"
- Semi‑Lagrangian advection: A stable numerical scheme that advects fields by following trajectories through the flow. "a U-Netâbased learnable semi-Lagrangian advection (Evolution Network)"
- STEPS model: A stochastic ensemble extrapolation method for short-term precipitation forecasting. "applies the STEPS model \citep{https://doi.org/10.1256/qj.04.100} for probabilistic extrapolation"
- Variational encoder–decoder (VED): A variationally trained encoder–decoder that infers latent states and structured outputs (e.g., motion and intensity fields). "a variational encoderâdecoder infers motion and intensity fields from recent imagery under optical-flow supervision"
- Warp function: A differentiable warping operator that applies motion and intensity corrections to produce the next frame. "\widehat{X}k = \text{warp} \left( \widehat{v}{k-1\rightarrow k}, \widehat{s}{k-1\rightarrow k}, \widehat{X}{k-1}\right)."
Practical Applications
Immediate Applications
The findings and methods in this paper enable several deployable, real-world uses, particularly in radar-sparse regions, due to the model’s satellite-only inputs, low latency (GOES‑16 RRQPE ~5 minutes), near real-time inference (<2 seconds for all lead times), and interpretable motion fields.
- Municipal early warning and public safety (Policy, Government; sectors: emergency management, public works)
- Use case: 10–180 minute precipitation nowcasts to issue targeted flash-flood, landslide, and storm alerts in cities lacking reliable radars (e.g., Rio de Janeiro–like settings).
- Potential tools/workflows: Operational Nowcast API producing gridded GeoTIFF/COG every 10 minutes; threshold exceedance maps; automated CAP (Common Alerting Protocol) messages; dashboards for civil defense.
- Assumptions/dependencies: GOES‑16 coverage (Americas) and product continuity; local calibration of thresholds (e.g., intensity–duration–frequency curves); server/GPU for inference; quality control for satellite QPE biases.
- Flood and landslide operations support (Policy, Industry; sectors: hydrology, civil engineering, utilities)
- Use case: Drive hydrologic/hydraulic models (e.g., SWMM, HEC‑HMS/HEC‑RAS) with short-term nowcasts to anticipate surface runoff, combined sewer overflow, and debris-flow/landslide triggers.
- Potential tools/workflows: Stream of nowcasts ingested by existing flood models; exceedance probability or threshold masks for pump/gate operations; integration with SCADA.
- Assumptions/dependencies: Local model calibration against gauges; satellite QPE bias correction; availability of high-resolution DEM/drainage network.
- Airport and aviation ground operations (Industry; sectors: aviation, transportation)
- Use case: Tactical decisions for ramp closures, de-icing, lightning proximity procedures, and taxiway routing under approaching convective cells.
- Potential tools/workflows: Airport operation center dashboard; integration with A‑CDM tooling; API to feed ramp management software.
- Assumptions/dependencies: 2 km/10 min resolution is adequate for many airport footprints; not a replacement for certified aviation weather products; integration with lightning/wind feeds improves safety.
- Road traffic and logistics routing (Industry; sectors: mobility, logistics, ride‑hailing, delivery)
- Use case: Dynamic re-routing and scheduling to avoid flooded segments or intense downpours; fleet pre-positioning; surge pricing protection.
- Potential tools/workflows: Map services overlay; API scoring of road segments for next 30–120 minutes; notification systems to drivers/customers.
- Assumptions/dependencies: Local hazard maps linking rainfall rates to impacts; uptime SLAs for API; mapping to actual street-level flood risk.
- Grid and field operations for utilities (Industry; sectors: energy, water, telecom)
- Use case: Crew staging for outage response; proactive inspections; adjusting urban drainage pump regimes; protecting vulnerable infrastructure.
- Potential tools/workflows: Alerting and work-order prioritization in outage management systems; storm operations dashboards.
- Assumptions/dependencies: Empirical mapping from predicted rainfall to outage/fault likelihood; interconnection with OMS/ADMS/SCADA.
- Renewable generation and demand-side management (Industry; sectors: energy)
- Use case: Anticipate solar PV ramps (cloud/rain proxy), wind ramp events (convective passage), and hydro inflows for short-term dispatch.
- Potential tools/workflows: Short-horizon forecasts into scheduling/optimization; PV soiling/wash scheduling; hydropower spill/peaking decisions.
- Assumptions/dependencies: Co-modeling of irradiance/cloud cover may be needed for PV; site-specific calibration.
- Precision agriculture field decisions (Industry; sectors: agriculture)
- Use case: Spray/no-spray windows, harvest timing, irrigation pause/resume in next 0–3 hours.
- Potential tools/workflows: Farm management system plugin; mobile app push alerts; geofenced field-level risk scores.
- Assumptions/dependencies: Sub-kilometer heterogeneity and canopy effects; on-farm rain gauges for bias correction.
- Insurance claims triage and client alerts (Industry; sectors: insurance, reinsurance)
- Use case: Early notification to policyholders; claims triage and staffing; event footprint estimation in near real-time.
- Potential tools/workflows: Event monitoring dashboards; automated client messaging; exposure overlay.
- Assumptions/dependencies: Regulatory acceptance of satellite-derived QPE; calibration vs. gauges/radar where available; audited versioning of models.
- Media and consumer weather apps (Daily life; sectors: software, media)
- Use case: Hyperlocal “next 3 hours” rain timing and intensity notifications in radar-poor areas.
- Potential tools/workflows: Mobile SDK/API; map tiles with motion vectors; lightweight inference microservice.
- Assumptions/dependencies: Clear uncertainty communication; load-tested API; localization for different climates.
- Research and teaching on physics-aligned ML (Academia; sectors: AI/ML for Earth, remote sensing, hydrology)
- Use case: Benchmarking optical-flow–supervised motion fields; studying interpretability; comparing physics-aligned vs. purely data-driven architectures.
- Potential tools/workflows: Reproducible training scripts; baselines (PySTEPS, Earthformer, NowcastNet) comparisons; Jupyter notebooks for curricula.
- Assumptions/dependencies: Access to GOES‑16 RRQPE/IMERG; availability of code or faithful reimplementation; compute for training.
Long-Term Applications
These opportunities require additional research, scaling, regionalization, or integration with external systems.
- Global nowcasting across geostationary fleets (Policy, Industry; sectors: meteorology, international development)
- Use case: Extend TUPANN to Meteosat (Europe/Africa), Himawari (Asia-Pacific), GOES‑West; provide equitable nowcasting where radar is scarce.
- Potential tools/workflows: Cross-sensor harmonization, transfer learning, domain adaptation; unified API covering multiple satellites.
- Assumptions/dependencies: Consistent satellite RRQPE proxies per fleet; re-training per sensor; handling retrieval biases and diurnal/seasonal regimes.
- Probabilistic/ensemble nowcasting with uncertainty quantification (Industry, Academia; sectors: risk, operations research)
- Use case: Generate pixelwise exceedance probabilities (e.g., P[rain > X mm/h]) for risk-aware decisions.
- Potential tools/workflows: Stochastic evolution/perturbation of latent states; diffusion-based or Bayesian variants; calibration with reliability diagrams.
- Assumptions/dependencies: Methodological work to maintain physical alignment under stochastic sampling; compute for ensembles.
- Real-time flood inundation mapping and digital twins (Policy, Industry; sectors: smart cities, civil engineering)
- Use case: Couple nowcasts with 2D hydrodynamic solvers to produce street-level inundation depth maps for 0–6 h horizons.
- Potential tools/workflows: City digital twin platforms; GPU-accelerated solvers; live SCADA control optimization.
- Assumptions/dependencies: High-quality terrain, drainage topology, and roughness maps; compute budget; rigorous validation.
- Hybrid satellite–radar–gauge assimilation (Academia, Industry; sectors: meteorology, water resources)
- Use case: Fuse sparse radars and gauges with satellite-only nowcasts to improve accuracy and correct biases.
- Potential tools/workflows: Variational/EnKF assimilation; learned bias-correction layers; crowdsourced rainfall signals.
- Assumptions/dependencies: Access to heterogeneous sensors; robust QC; latency alignment across streams.
- Multi-hazard nowcasting (Policy, Industry; sectors: severe weather, insurance)
- Use case: Joint prediction of convective hazards (lightning, hail proxies), urban wind gusts, and extreme rainfall for integrated warnings.
- Potential tools/workflows: Multi-task architectures; fusion of GLM lightning, IR bands, NWP predictors; hazard-specific thresholds.
- Assumptions/dependencies: Additional labeled datasets; hazard-specific verification frameworks.
- Regulatory adoption and standardization (Policy; sectors: national weather services, WMO members)
- Use case: Transition from research to sanctioned operational products with auditability and governance.
- Potential tools/workflows: MLOps, model cards, data lineage; adherence to WMO verification standards; service-level agreements.
- Assumptions/dependencies: Long-term skill validation; change management; legal and procurement frameworks.
- Parametric and microinsurance product design (Industry; sectors: finance, insurance)
- Use case: Near-real-time triggers for microinsurance and meso-level disaster risk financing programs.
- Potential tools/workflows: Trigger indices based on exceedance duration/area; smart contracts; reinsurance linkage.
- Assumptions/dependencies: Basis risk analysis; regulator acceptance of satellite-based triggers; robust backtesting.
- Edge/on-board deployment (Industry, Space agencies; sectors: edge AI, satellite ops)
- Use case: Run compressed models on edge servers or future satellites to cut latency, serving remote communities with limited connectivity.
- Potential tools/workflows: Quantization/pruning; model distillation; streaming inference at local hubs.
- Assumptions/dependencies: Hardware constraints; robust failover and over-the-air updates; security certification.
- Agricultural robotics and autonomy (Industry; sectors: agri-robotics)
- Use case: Weather-aware path planning and task scheduling for field robots and autonomous implements.
- Potential tools/workflows: Onboard inference modules; geofenced operational constraints; V2X coordination.
- Assumptions/dependencies: Reliable sub-field precipitation mapping; integration with fleet management.
- Climate services and extremes analytics (Academia, Policy; sectors: climate risk)
- Use case: Event catalogs and short-term extremes statistics for adaptation planning and infrastructure design stress tests.
- Potential tools/workflows: Archival of nowcasts/observations; extreme value analysis; integration with resilience planning platforms.
- Assumptions/dependencies: Longitudinal data stewardship; homogenization across satellite generations; bias correction vs. gauges/radar where possible.
Cross-cutting assumptions and dependencies affecting feasibility
- Data availability and transferability: GOES‑16 RRQPE currently covers the Americas; extension to other regions needs analogous products and re-training. Satellite retrieval biases vary by climate/topography and may drift over time.
- Model generalization: Cross-city tests show modest degradation; multi-city training helps. For operational use, local fine-tuning and continuous evaluation are recommended.
- Verification and trust: Interpretable motion fields improve trust, but governance, versioning, and audit trails are essential for policy and regulated uses.
- Compute and MLOps: Near real-time inference is feasible on a single modern GPU; production requires robust pipelines, monitoring, and fallback modes.
- Deterministic vs. probabilistic outputs: Current model is deterministic; risk-sensitive applications benefit from uncertainty-aware extensions.
- Legal and user communication: For consumer and policy applications, clear uncertainty communication and disclaimers are needed; some sectors require certified products.
Collections
Sign up for free to add this paper to one or more collections.