HKO-7 Nowcasting Dataset

• HKO-7 is a benchmark dataset comprising radar reflectivity images captured every 6 minutes over a seven-year period to support realistic precipitation nowcasting.
• It employs rigorous preprocessing, temporal continuity, and standardized training, validation, and testing splits to ensure comparability across deep learning models.
• The dataset supports evaluation using metrics such as CSI, HSS, and MSE, fostering innovations in high-resolution rainfall prediction.

The HKO-7 dataset is a benchmark dataset extensively used in spatiotemporal forecasting, particularly for precipitation nowcasting using radar echo data. It was originally introduced in the Hong Kong Observatory weather radar-based studies, providing a standardized and realistic testbed for the evaluation of deep learning models targeting short-term, high-resolution rainfall prediction tasks. HKO-7 consists of consecutive radar reflectivity images spanning multiple years, with rigorous protocols for training, validation, and testing splits, tailored loss metrics, and fine-grained evaluation criteria to foster principled comparison across research in deep learning-based nowcasting.

1. Origin and Motivation

HKO-7 was constructed to address the practical challenges of precipitation nowcasting, a task grounded in predicting rainfall intensity maps at high spatiotemporal resolution over a short time horizon. Originating from the Hong Kong Observatory’s Doppler radar, the dataset captures diverse meteorological patterns over the subtropical climate of Hong Kong. Its primary motivation is to facilitate the development and benchmarking of data-driven spatiotemporal prediction models, which must capture fine-scale motion, intensity variations, and abrupt pattern transformations characteristic of convective precipitation events.

HKO-7’s design reflects operational considerations in weather forecasting, including real-time data availability, the need to handle missing or corrupted sensor frames, and the requirement for models to generalize across years and synoptic regimes.

2. Dataset Composition and Statistics

HKO-7 comprises radar echo frames recorded every 6 minutes, covering a continuous seven-year period. Each sample in the dataset is a spatiotemporal sequence: an input context (historical sequence) followed by a forecast target sequence (future frames). Frames are spatially aligned and quantized into regular lattices, typically at resolutions such as 480×480 pixels, encoding reflectivity in dBZ.

Key features include:

• Temporal resolution: 6-minute intervals per frame
• Spatial resolution: Usually 480×480 grids, with reflectivity values quantized
• Sequence structure: Standard evaluation uses 5 input frames (30 minutes) to predict the next 20 frames (2 hours)
• Statistics: Multiple years of continuous operation, yielding tens of thousands of sequences for robust data-driven modeling
• Meteorological diversity: Includes typhoons, frontal events, and localized convection

Data quality control, temporal continuity across sequences, and careful curation of input-target splits are embedded in the dataset protocol to align with meteorological best practices.

3. Preprocessing and Data Protocols

HKO-7 enforces strict preprocessing to maximize model comparability. Each radar reflectivity frame is preprocessed to handle sensor-specific artefacts, missing values, and dynamic range clipping appropriate for radar meteorology. Preprocessing pipelines remove noise, correct known anomalies, and ensure geospatial and temporal consistency across the dataset.

Canonical splits are employed for training, validation, and testing, often with entire years reserved for testing to measure generalization out-of-sample. This temporal split strategy prevents information leakage and supports rigorous assessment of overfitting and generalization in operational scenarios.

4. Evaluation Metrics and Benchmarking

Benchmarking on HKO-7 standardizes evaluation across models with well-defined metrics reflecting both meteorological significance and ML community adoption. These include:

• Critical Success Index (CSI): Measures skill in predicting correct threshold exceedances within spatial maps
• Heidke Skill Score (HSS): Assesses model performance relative to a random classifier baseline
• Mean Squared Error (MSE): Quantifies pixel-wise regression accuracy
• Rainfall-specific metrics: Include hit rate, false alarm rate, and others at various dBZ thresholds

Benchmarks typically require reporting metrics at multiple lead times and over different rainfall intensities, aligning with meteorological impact priorities.

5. Impact and Research Applications

HKO-7 has become a pivotal benchmark for the field of nowcasting and spatiotemporal prediction, supporting extensive evaluation of deep learning architectures such as ConvLSTM, 3D CNNs, GAN-based models, and transformers. Its realistic noise conditions, long-range variability, and meteorological complexity pose substantial challenges, driving algorithmic innovation in addressing motion forecasting, uncertainty quantification, and robustness to missing data.

The dataset’s wide adoption underpins a growing corpus of comparative empirical studies, catalyzing advancements in learning-based geoscientific forecasting. It also motivates the development of models capable of end-to-end uncertainty propagation, domain adaptation, and transfer learning.

6. Limitations and Considerations

While HKO-7 provides a comprehensive and challenging testbed, several limitations are widely discussed in the literature. Its regional focus constrains its climatic diversity, potentially limiting generalization to different meteorological regimes. Refined spatial and temporal coverage is still governed by the observing radar’s specifications and local topography. Observational biases due to partial beam blockage, ground clutter, and attenuation must be considered in model evaluation and deployment.

A plausible implication is that while HKO-7 benchmarks serve as strong indicators of solution viability, operational deployment beyond the Hong Kong region or application to different sensor infrastructures will require further domain-specific adaptation and validation.