XiChen: An observation-scalable fully AI-driven global weather forecasting system with 4D variational knowledge (2507.09202v1)

Abstract: Recent advancements in AI demonstrate significant potential to revolutionize weather forecasting. However, most AI-driven models rely on Numerical Weather Prediction (NWP) systems for initial condition preparation, which often consumes hours on supercomputers. Here we introduce XiChen, the first observation-scalable fully AI-driven global weather forecasting system, whose entire pipeline, from Data Assimilation (DA) to medium-range forecasting, can be accomplished within only 17 seconds. XiChen is built upon a foundation model that is pre-trained for weather forecasting. Meanwhile, this model is subsequently fine-tuned to serve as both observation operators and DA models, thereby scalably assimilating conventional and raw satellite observations. Furthermore, the integration of four-dimensional variational knowledge ensures that XiChen's DA and medium-range forecasting accuracy rivals that of operational NWP systems, amazingly achieving a skillful forecasting lead time exceeding 8.25 days. These findings demonstrate that XiChen holds strong potential toward fully AI-driven weather forecasting independent of NWP systems.

• The paper introduces XiChen as a fully AI-driven global weather forecasting system that integrates 4D-Var knowledge for rapid data assimilation and extended forecast lead times.
• The paper describes how XiChen combines a pre-trained foundation model with a cascade DA framework, outperforming traditional NWP benchmarks like GFS and IFS HRES.
• The paper demonstrates XiChen's ability to accurately track tropical cyclones and enhance forecast accuracy through efficient satellite observation assimilation.

XiChen: An observation-scalable fully AI-driven global weather forecasting system with 4D variational knowledge

Introduction

XiChen is a fully AI-driven global weather forecasting system devised to supersede traditional Numerical Weather Prediction (NWP) dependencies, capable of performing Data Assimilation (DA) and medium-range forecasting with impressive efficiency. By utilizing a foundation model pre-trained for weather forecasting, XiChen incorporates four-dimensional variational (4D-Var) knowledge, allowing efficient assimilation of conventional and satellite observations within mere seconds on an A100 GPU. XiChen achieves a forecasting lead time of over 8.25 days, comparable to current operational systems, indicating a significant milestone towards fully AI-driven weather forecasting.

Methodology and Architecture

XiChen's core architecture is built upon the Conditional Hybrid Neural Operator (CHNO), integrating structural design to enable adaptability to diverse weather-related tasks via pre-training and fine-tuning mechanisms. First, XiChen is pre-trained as a medium-range weather forecasting model using lead time as conditional input. Subsequently, it is fine-tuned to function as both observation operators and DA models. XiChen employs a cascade DA framework for chronological assimilation of GDAS prepbufr, Advanced Microwave Sounding Unit-A (AMSU-A), and Microwave Humidity Sensor (MHS) observations. The integration of 4DVar constraints enhances the model’s ability to maintain accuracy over time, asserting its robustness in comparison to standard systems.

Figure 1: Overview of XiChen, illustrating its AI-driven approach to weather forecasting and data assimilation integrating 4D-Var knowledge.

Experimental Evaluation

One-Year DA Cycle

The evaluation of the one-year DA cycle demonstrates XiChen's stability in atmospheric state estimation accuracy, showing lower RMSE for key variables such as Z500, T850, and MSLP compared to GFS, and approaching IFS HRES results. The systematic inclusion of temporal observations through extended DAWs corroborates the enhancement in analysis accuracy.

Figure 2: Comparison of RMSE and Bias for a one-year DA cycle using XiChen, GFS, and IFS HRES, evaluated against ERA5 data.

Medium-Range Weather Forecasting

XiChen's performance in medium-range weather forecasting demonstrates significant skill, with ACC maintaining above 0.6 for over 8.25 days, surpassing notable systems like Aardvark and GraphDOP. This remarkable forecasting ability, despite a relatively lower resolution and observational data count, substantiates XiChen’s potential in operational meteorological applications.

Figure 3: RMSE and ACC of 10-day medium-range weather forecasts using XiChen, GFS, and IFS HRES.

Contribution of Satellite Observations

The assimilation of AMSU-A directly impacts upper-air temperature fields, whereas MHS primarily affects atmospheric humidity. XiChen's configurational flexibility permits assessment of individual satellite contributions, revealing overall enhancements across forecasting errors when AMSU-A and MHS are collectively assimilated.

Figure 4: RMSE scorecards reflecting assimilation impacts of different observations on DA cycle and medium-range forecasts.

Impact of 4D-Var Knowledge

Introducing 4D-Var knowledge into DA rejuvenates analysis and forecasting performance by chronologically harnessing more observational information. Variable DAWs underline improvements in XiChen's predictive capacity, underscoring the effectiveness of integrating variational knowledge into AI models.

Figure 5: RMSE and Bias showing the impact of different DAW lengths on one-year DA cycle performance.

Tropical Cyclone Assimilation and Forecasting

XiChen’s performance in tropical cyclone trajectory forecasting exemplifies its efficacy in accurately mapping cyclone paths, enhanced by assimilating satellite data. The TC trajectory tracks demonstrate XiChen's ability to match and sometimes exceed ERA5 reanalyses.

Figure 6: TC trajectory tracks from IBTrACS and various assimilations within XiChen’s framework, showing performance comparisons.

Conclusion

XiChen represents a pioneering advancement in fully AI-driven global weather forecasting, demonstrating efficient performance across various meteorological tasks. The scalability facilitated by foundational models and cascade DA frameworks positions XiChen as a crucial development towards operational AI meteorology. Despite certain limitations, such as smoothing effects and limited resolution, XiChen’s adaptability to new observational data heralds a future where AI-driven systems seamlessly integrate into weather forecasting infrastructures. Continued research is necessary to address current challenges and elevate AI models to the predictive accuracy of traditional NWP systems, enhancing global meteorological understanding and decision-making efficacy.