Otter Weather: Skillful and Computationally Efficient Medium-Range Weather Forecasting

Abstract: State-of-the-art medium-range AI weather models can outperform traditional Numerical Weather Prediction (NWP) but require massive training budgets. This restricts usage for under-resourced groups and severely limits fast model iteration. Here we develop Otter Weather, a highly efficient spatiotemporal forecasting model designed to democratise high-performance weather prediction with AI. Evaluated on ERA5 reanalysis data at 1.5° resolution using standard WeatherBench protocols, the Otter family significantly advances the skill-compute Pareto frontier. The deterministic version outperforms the best NWP baseline by 9.6% at a 24-hour lead time while requiring fewer than 3.5 A100-days for training. It provides a 2x efficiency gain over lightweight AI models and a 100-fold reduction in compute compared to resource-intensive frontier architectures. We extend these efficiency gains into probabilistic forecasting by training via the Continuous Ranked Probability Score (CRPS). Scaling to a larger architecture, Otter-XL achieves a 9.7% CRPS improvement over the IFS ENS baseline. This yields an almost two-fold increase in predictive skill over comparable lightweight models at similar compute budgets. Otter-XL also outperforms frontier architectures like GenCast by over 2%, while using an order of magnitude less compute. Finally, Otter is applied out-of-the-box to a complex acoustic scattering PDE task where it outperforms a state-of-the-art foundation modelling approach, suggesting that the advances made here might apply across a range of scientific domains.

• The paper demonstrates that a domain-agnostic, hardware-optimized U-Net achieves competitive forecasting skill with over sixfold compute efficiency compared to state-of-the-art models.
• The architecture integrates Swin Transformer, RoPE, and SwiGLU modules, validated by ablation studies showing a 9.6% RMSE improvement over IFS HRES.
• The approach democratizes forecasting by reducing training costs and proving transferability to diverse spatiotemporal PDE tasks.

Otter Weather: Efficient and High-Skill Medium-Range Weather Forecasting

Introduction

"Otter Weather: Skillful and Computationally Efficient Medium-Range Weather Forecasting" (2606.26421) presents a spatiotemporal forecasting architecture designed to substantially reduce training costs while attaining skill competitive with the state-of-the-art in medium-range AI weather prediction. The paper addresses the pressing limitations on democratizing global forecasting—namely, the massive compute demands and rigid architectural priors seen in leading numerical and ML models—which hamper rapid iteration, retraining, and customization by resource-limited groups. By deliberately stripping away bespoke, domain-specific complexity in favor of hardware-optimized, general-purpose ML advances from vision and LLMs (specifically Swin Transformer, RoPE, SwiGLU, Muon), Otter Weather sets new benchmarks for the skill–compute Pareto frontier in both deterministic and probabilistic forecasting, and further demonstrates strong architectural transferability to non-atmospheric PDE tasks.

Model Architecture and Experimental Setup

Otter Weather adopts a symmetric U-Net configuration based on the 2D Swin Transformer, with minimal Earth-specific modifications. All input variables—including multi-level atmospheric, surface, and static features—are concatenated channel-wise and paired with Fourier-based time embeddings. Tokenization is performed via strided convolution, yielding patch-based latent representations, with a constant channel width maintained across encoder, bottleneck, and decoder stages. Key architectural innovations include SwiGLU feed-forward networks, 2D RoPE for relative position encoding, no explicit attention masking (torus topology), and direct utilization of commodity hardware optimizations.

Optimization is performed using the Muon optimizer, supported by strong regularization (dropout, weight decay, stochastic depth) and gradient accumulation to amortize Muon's step cost. The model is trained at ERA5 reanalysis data resolution (1.5°) over decades of historical data, using WeatherBench-2 protocols for both deterministic and probabilistic evaluation.

Figure 1: Otter Weather architecture—2D Swin Transformer with channel-wise concatenation and hardware-friendly attention.

Advancing the Skill–Compute Pareto Frontier

Deterministic Forecasting

Through systematic ablations, Otter Weather identifies balanced U-Net depth ($[2,8,4]$ blocks, $D=1536$), SwiGLU, RoPE, and the Muon optimizer as primary drivers of skill and efficiency. Components (Hyperconnections, NATTEN, MAE pretraining, MoE) offering marginal gains or requiring custom kernels are excluded for practical reasons. Attention masking across poles is proven unnecessary given positional embeddings and yields negligible skill drops.

Quantitative evaluation at 24h lead shows Otter surpassing IFS HRES (RMSE skill improvement 9.6%) and outperforming ensembles such as ArchesWeather with over sixfold compute efficiency (<3.5 A100-days). Scaling to Otter-XL (patch size 1) boosts skill to 14.9%, matching GraphCast’s performance at ~130x less compute.

Figure 2: Deterministic models—Otter Weather establishes a new Pareto frontier in skill versus compute.

Figure 3: Architectural ablations—only balanced backbones, Muon, SwiGLU, and RoPE justifiably increase skill without disproportionate compute cost.

Figure 4: RMSE skill over IFS HRES—Otter variants demonstrate skill persistence at lead times up to 10 days, outperforming most baselines at high efficiency.

Figure 5: RMSE comparison—Otter and Otter-XL show favorable scaling versus GraphCast and NWP baselines.

Probabilistic Forecasting

Otter’s deterministic backbone is retrofitted for probabilistic forecasting using the CRPS scoring rule, via (1) fine-tuning from deterministic checkpoints and (2) end-to-end CRPS training. Monte Carlo Dropout is found superior to AdaLN for stochasticity, offering better CRPS in rollout and eliminating parameter overhead.

Ensembling is performed efficiently by aggregating RFT checkpoints across learning rates, producing high-quality Deep Ensembles without expensive retraining. Under 8 A100-days, Otter CRPS achieves a 6.1% CRPS improvement over IFS ENS; Otter-XL Deep Ens (29 A100-days) attains a 9.7% improvement and a Spread-Skill Ratio (SSR) of 0.95, surpassing GenCast by 2.2% CRPS at one-tenth the compute.

Figure 6: CRPS—performance across training strategies, demonstrating efficiency of MC Dropout and deep ensembling.

Figure 7: Scaling behavior—Otter-XL models deliver over 55% relative forecasting skill increases with moderate compute scaling.

Figure 8: CRPS skill—Otter variants outperform flow-based baselines and scale successfully to longer lead times.

Figure 9: SSR—Otter variants maintain tight calibration relative to IFS ENS across lead times.

Figure 10: CRPS skill—Otter-XL matches or exceeds skill of GenCast and ArchesWeatherGen at 10 days.

Figure 11: SSR—Otter-XL achieves robust calibration across all major ensemble baselines.

Architectural Transfer to Scientific ML

Otter’s domain-agnostic design principles were validated on the Well benchmark’s acoustic scattering PDE task, a substantially different physical system. Despite restricted training budgets, Otter outperformed Walrus (PDE foundation model), achieving 0.0030 one-step VRMSE versus 0.0089 and consistently lower rollout VRMSE across extended horizons.

Ablation studies confirm that Muon, SwiGLU, RoPE, and balanced backbones—all motifs from weather forecasting—retain their importance in PDE modeling. The generality observed suggests hardware-optimized Transformers can template efficient spatiotemporal prediction across domains.

Figure 12: Ablation study on the acoustic scattering PDE task—core Otter design principles consistently transfer and improve skill.

Forecast Visualization

Visualizations for headline variables (Z500, Q700, T850, U850, V850, T2m, SP, U10m, V10m) under deterministic and probabilistic settings illustrate Otter’s capacity to preserve forecast detail and dispersion over varying lead times, supporting robust uncertainty quantification.

Figure 13: Visualization of Q700 at 12h lead time—Otter maintains high skill and spatial resolution.

Figure 14: Visualization of SP at 3 days—ensemble spread and mean retain physical plausibility.

Figure 15: Visualization of T2M at 10 days—Otter-XL ensemble mean remains competitive with baselines at extended horizons.

Implications and Future Directions

The democratization and high skill efficiency encouraged by Otter Weather have immediate practical effects: groups with modest hardware can now train competitive models, audit architectures, iterate rapidly, and generate ensembles without reliance on foundation models or large-scale distributed resources. Operational agencies and domain researchers benefit from greater independence, auditability, and adaptability to regional or scientific idiosyncrasies.

Theoretical implications include a reevaluation of the necessity for domain-specific inductive biases—general-purpose ML components suffice for many tasks, and compounding benefits may arise when paired with explicit physics-based priors in future work. Efficient probabilistic ensembles (via MC Dropout, learning rate branching) enable uncertainty quantification without iterative inference overhead typical in diffusion models. Deep ensemble construction methodology remains open; analysis of branching points and independent RFT rounds may optimize diversity and calibration. Future scaling studies (higher spatial resolutions, downstream tasks, automated hyperparameter search) are now feasible given Otter's low compute demands.

Broader societal impacts include the possibility for rapid, decentralized model development but raise caution regarding unphysical outputs or miscalibrated uncertainty, especially in safety-critical contexts.

Conclusion

Otter Weather establishes a new standard for efficient, skillful medium-range forecasting, demonstrating that high-performance modeling does not require massive training budgets or rigid architectural complexity. The simplicity and accessibility of its hardware-friendly, domain-agnostic design enable both practical and theoretical advances in spatiotemporal ML, with preliminary evidence indicating effective transferability to broader scientific domains. The methodology encourages a diversified ecosystem of forecasting models and ensembles, fostering both scientific sovereignty and operational robustness.