Generative 3D Gaussian Splatting for Arbitrary-ResolutionAtmospheric Downscaling and Forecasting

Abstract: While AI-based numerical weather prediction (NWP) enables rapid forecasting, generating high-resolution outputs remains computationally demanding due to limited multi-scale adaptability and inefficient data representations. We propose the 3D Gaussian splatting-based scale-aware vision transformer (GSSA-ViT), a novel framework for arbitrary-resolution forecasting and flexible downscaling of high-dimensional atmospheric fields. Specifically, latitude-longitude grid points are treated as centers of 3D Gaussians. A generative 3D Gaussian prediction scheme is introduced to estimate key parameters, including covariance, attributes, and opacity, for unseen samples, improving generalization and mitigating overfitting. In addition, a scale-aware attention module is designed to capture cross-scale dependencies, enabling the model to effectively integrate information across varying downscaling ratios and support continuous resolution adaptation. To our knowledge, this is the first NWP approach that combines generative 3D Gaussian modeling with scale-aware attention for unified multi-scale prediction. Experiments on ERA5 show that the proposed method accurately forecasts 87 atmospheric variables at arbitrary resolutions, while evaluations on ERA5 and CMIP6 demonstrate its superior performance in downscaling tasks. The proposed framework provides an efficient and scalable solution for high-resolution, multi-scale atmospheric prediction and downscaling. Code is available at: https://github.com/binbin2xs/weather-GS.

• The paper introduces a novel generative 3D Gaussian Splatting framework that unifies atmospheric downscaling and forecasting.
• It leverages a scale-aware Vision Transformer to generate continuous, high-fidelity predictions across arbitrary resolutions.
• Evaluations on CMIP6 and ERA5 datasets demonstrate state-of-the-art performance with significant LRMSE improvements.

Generative 3D Gaussian Splatting for Arbitrary-Resolution Atmospheric Downscaling and Forecasting

Introduction and Motivation

The challenge of producing high-resolution, flexible atmospheric forecasts with neural weather models is fundamentally limited by traditional grid-based parameterizations and the requirement for fixed-scale architectures. Most modern data-driven NWP models rely on fixed spatial resolutions and require separate models or decoders for different downscaling ratios, resulting in excessive computational demands and inflexibility. The paper "Generative 3D Gaussian Splatting for Arbitrary-Resolution Atmospheric Downscaling and Forecasting" (2604.07928) introduces a paradigm shift by formulating both downscaling and forecasting tasks as generative problems in a continuous Gaussian space, leveraging 3D Gaussian Splatting (3DGS) to fundamentally enable arbitrary-resolution prediction using a unified model.

Figure 1: Comparison between traditional and GSSA-ViT-based forecasting, highlighting single-decoder efficiency and direct arbitrary-resolution rendering.

Methodology: 3D Gaussian Splatting and Scale-Aware Vision Transformer

The core of the proposed framework is the GSSA-ViT, which couples 3DGS-based atmospheric field representation with a scale-aware Vision Transformer. Each grid point on the latitude–longitude mesh is embedded as a center of a 3D Gaussian primitive, parameterized by its covariance, attributes, and opacity. The system generates Gaussian parameters for every point of interest, enforcing a continuous, physically plausible representation, which is essential for flexible high-resolution rendering and efficient information compression.

As opposed to scene-specific overfitting seen in prior 3DGS applications, the approach introduces a generative mechanism, predicting Gaussian parameters conditionally for unseen samples, thus supporting both spatial downscaling and temporal forecasting within the same generative regime. The scale-aware attention mechanism, implemented via embedding of the downscaling ratio into the transformer’s cross-attention, explicitly guides the network to adapt across arbitrary output resolutions.

Figure 2: GSSA-ViT workflow from low-res grid initialization through scale-aware attention to arbitrary-res rendering.

Continuous atmospheric variables across pressure levels and altitudes are thus rendered from the intermediate Gaussian space to any desired spatial resolution by varying the sampling density on the sphere. This operation is fully differentiable, enabling end-to-end optimization of the entire framework for both downscaling and forecasting objectives.

Evaluation: Downscaling and Forecasting at Arbitrary Resolution

Downscaling Performance

The authors extensively benchmark the method on downscaling tasks from coarse CMIP6 outputs to high-resolution ERA5 targets. Quantitative evaluation demonstrates that GSSA-ViT consistently achieves the lowest LRMSE across standard atmospheric variables, significantly outperforming prominent baselines including MINet, MetaSR, and LIIF, especially as the target resolution approaches finer scales (e.g., 1.40625°, 0.703125°, 0.3515625°).

Figure 3: Relative LRMSE across downscaling ratios for several variables, showing GSSA-ViT's consistent superiority even as ratios increase.

Visual comparisons further substantiate the numerical superiority: GSSA-ViT produces sharper, physically coherent reconstructions in both global and polar regions, overcoming the blurriness and information loss typical of learning-based and interpolation-based baselines.

Figure 4: Global visualizations at ×4 downscaling, exposing GSSA-ViT's capability to recover spatial structures across diverse atmospheric variables compared to several baselines.

Figure 5: Qualitative results at ×8 downscaling, with GSSA-ViT maintaining high spatial fidelity where other methods degrade.

Figure 6: Downscaling at ×16, highlighting GSSA-ViT's robustness in ultra-fine reconstructions.

Regional visualizations bolster this finding, demonstrating effective detail capture even in climatically complex subregions.

Figure 7: Regional reconstructions at ×4 downscaling, focusing on challenging mesoscale features.

Arbitrary-Resolution Forecasting

In the arbitrary-resolution forecasting regime using ERA5, GSSA-ViT delivers the best accuracy over a wide range of lead times (from 6h to 120h) and across output scales, outperforming both upsampled low-resolution neural forecasters (e.g., NeuralGCM, Stormer) and deep-learning-based upsampling models.

Figure 8: LRMSE as a function of forecast horizon and target resolution; GSSA-ViT demonstrates stable, superior performance even at fine scales and long lead times.

At each vertical level and for a diverse suite of prognostic variables, the model maintains error stability across resolutions and heights, reflecting strong vertical and scale robustness.

Figure 9: Six-hour lead LRMSE as a function of downscaling ratio and vertical level: GSSA-ViT shows negligible degradation in accuracy with increased scale factor.

Qualitative visualizations of GSSA-ViT’s global predictions reinforce the quantitative findings. Notably, GSSA-ViT avoids the post hoc interpolation artifacts inherent to native low-resolution forecasting models, consistently producing fine-grained, physically plausible spatial details.

Figure 10: Global upper-level prediction at 0.25°: GSSA-ViT directly generates high-res outputs.

Figure 11: Surface variables at 0.25°: GSSA-ViT’s direct predictions are qualitatively richer.

Regional error maps validate the model’s ability to reconstruct mesoscale and local weather phenomena, especially in dynamically active regions.

Figure 12: Regional surface-level visualizations at 0.3515625°, GSSA-ViT preserves structure absent in upsampled baselines.

Figure 13: Additional regional analyses for surface states further demonstrating spatial reconstruction advantages.

Ablation and Model Analysis

Ablative experiments confirm the architectural necessity of fixed spatial priors, learnable Gaussian parameterization (covariance, rotation, opacity), and decoder modularity. Increasing the number of Gaussian primitives above the baseline did not yield further improvement, indicating strong representational adequacy and computational efficiency in the default configuration.

Implications and Future Directions

This research provides a practical and theoretically sound solution to both atmospheric downscaling and forecasting at arbitrary output resolutions, drastically reducing system complexity by unifying the generative mechanism. Practically, this enables high-resolution, multiscale NWP without retraining or escalating hardware requirements, immediately benefiting weather service and climate research pipelines. Theoretically, the generative 3DGS approach opens new opportunities for representing Earth system dynamics in a physically continuous, adaptable feature space, moving beyond the rigid grid-based tradition.

Potential future developments include integrating temporal consistency via sequence modeling (e.g., diffusion models), incorporating operational heterogeneous data streams (e.g., ungridded satellite and radar measurements), and deploying efficient sparse attention architectures to enable even finer global predictions. The framework’s modularity suggests it can generalize to other high-dimensional geoscience data and cross-modality prediction tasks.

Conclusion

The paper presents a unified, generative conditional 3DGS-based model with scale-aware transformers for high-fidelity, arbitrary-resolution atmospheric downscaling and forecasting (2604.07928). Both in quantitative and qualitative terms, GSSA-ViT achieves state-of-the-art results across CMIP6 and ERA5 benchmarks, at all tested spatial and temporal scales. The work sets a new concrete direction for scalable, resolution-agnostic, and physically grounded machine-learning weather models and unlocks significant promise for operational and research applications in climate and Earth system modeling.