Urban Heat MiniCubes: An AI-Ready dataset for urban heat research

Abstract: Urban heat is amplified by impermeable surfaces and heterogeneous built environments, yet street-level variability remains difficult to quantify because multi-sensor observations are rarely available in consistent, analysis-ready form at the necessary spatiotemporal scales. We present "Urban Heat MiniCubes," a publicly available, FAIR-oriented dataset designed for machine learning applications in urban heat research. The dataset provides harmonized 90 x 90 km gridded data cubes for 48 cities in the Western Hemisphere spanning 2022-2023, with variables reprojected and collocated to a common grid to reduce preprocessing (e.g., reprojection, resampling, and spatiotemporal alignment). Urban Heat MiniCubes includes two complementary modalities: (i) higher-spatial-resolution, lower-frequency observations from Landsat 8/9 (e.g., surface reflectances) and Sentinel-1 (e.g., synthetic aperture radar backscatter), and (ii) higher-temporal-frequency, coarser observations from GOES-R (e.g., longwave infrared brightness temperatures) and a microwave land surface temperature product. We document variables and metadata and provide technical assessment using inter-variable analyses and autoencoder-based reconstruction-error summaries across pixel classes (e.g., water and cloud). Potential use cases and limitations are also discussed.

• The paper presents a harmonized, AI-ready multi-modal dataset covering 48 major Western Hemisphere cities to enable detailed urban heat analysis.
• The methodology fuses Landsat, Sentinel-1, GOES-R, and microwave data using systematic spatial resampling and temporal matching to overcome sensor disparities.
• The dataset supports advanced ML applications such as super-resolution, inpainting, and urban equity analysis while addressing challenges like cloud contamination.

Urban Heat MiniCubes: An AI-Ready Multi-Modal Dataset for Urban Heat Research

Introduction and Problem Motivation

Quantifying urban heat at neighborhood-relevant spatiotemporal scales remains a critical challenge, driven by the complex interplay of built-environment heterogeneity, rapid land use changes, and disparate data streams from remotely sensed and in situ observations. Existing satellite remote sensing platforms provide essential data on urban surface temperature, but incompatibilities in their spatiotemporal resolutions, retrieval definitions, and file formats present a high barrier to analysis, particularly for methods in ML and AI. The "Urban Heat MiniCubes" dataset ("Urban Heat MiniCubes: An AI-Ready dataset for urban heat research" (2606.11534)) directly addresses these challenges by providing harmonized, FAIR-compliant, and AI-ready data cubes covering 48 major Western Hemisphere cities during 2022–2023.

Dataset Construction: Sensor Fusion and Harmonization

Urban Heat MiniCubes aggregates and harmonizes multi-platform satellite datasets at a fixed 90 km × 90 km grid per city, addressing both spatial and temporal matching between sensors:

• Landsat 8/9: High spatial resolution (30 m), low revisit frequency (8 days combined), including a full suite of L2 surface reflectances, LST (retrieved from TIR imagery), and cloud masks.
• Sentinel-1: Active SAR acquisitions (VV/VH/HH/HV and local incidence angles) resampled to Landsat’s 30 m grid, exploiting all-weather sensing to complement TIR limitations.
• GOES-R (GOES-16/17/18): Geostationary full disk, providing longwave infrared brightness temperature (BT) every 10 minutes at 2 km native resolution, permitting tracking of diurnal thermal cycles.
• Microwave LST: Derived from Ka-band radiometers pooled from multiple LEO satellites, providing coarser (0.25°) but cloud-invariant surface temperature, resampled via nearest neighbor to GOES resolution for alignment and completeness.

Spatial harmonization employs systematic resampling, strict adherence to native pixel value preservation where possible (nearest-neighbor), and UTM projection to a fixed grid for each city. Temporal matching procedures provide for joint analyses, including mosaicking strategies for partial Landsat coverages and ±6 day matching of Sentinel-1 to Landsat for cross-modality feature sets.

Figure 1: Various satellite sensor measurements over urban domains, illustrating trade-offs between spatial coverage and temporal frequency.

Figure 2: Geographical distribution of the 48 city MiniCubes with concurrent LST brightness temperature overlays.

Data Structure, Metadata, and FAIR/AI-Ready Principles

Dataset files are provided in netCDF format, adhering to CF metadata conventions, and structured for programmatic access and georeferencing. Two primary file types are constructed per city:

• Landsat/Sentinel-1 files (13 bands, 3000×3000 px) at up to 8-day intervals,
• GOES/Microwave LST files (5 bands, 45×45 px, time-concatenated across the two-year span at 10-minute steps).

  Figure 3: Example file structure, showing joint storage of multiband imagery and supporting coordinates.

Supporting metadata encompass sensor type, variable descriptions (standard_name, long_name), projection information, physical units, valid ranges, wavelength specificity, and quality flags. A binary-encoded cloud mask, with a detailed bitmask key, supports pixel-level quality and cloud filtering within ML pipelines.

Cross-Sensor Comparisons and Statistical Assessment

Cross-modal statistics reveal systematic differences and complementarities:

• Landsat LST retrievals are systematically warmer than corresponding GOES-ABI BTs—an expected result as the former are surface-corrected, whereas GOES measures top-of-atmosphere radiances, and both are susceptible to cloud contamination unless explicitly masked.

  Figure 4: PDFs of Landsat LST and GOES Band 14 BT for all cities, with statistical tests delineating significant differences.

• Spatial variability (within-image standard deviation) is consistently greater for Landsat LST, even after low-pass filtering/downscaling to GOES resolution, indicating the persistence of fine-scale urban thermal heterogeneity not captured by coarser geostationary sensors.

  Figure 5: Comparison of within-image spatial variability (std) between GOES Band 14 BT and filtered/resampled Landsat LST.

• Cloud climatology exhibits substantial intra-city variation, with higher cloud fractions and increased masking challenges in tropical/locales versus arid centers.

  Figure 6: Distribution of Landsat-derived cloud cover fractions, ordered by mean cloudiness per city.

• Correlations among surface reflectance bands, SAR polarizations, and LST are provided city-wise as heatmaps, with SWIR bands typically exhibiting the strongest association with urban LST, supporting their use as features for urban LST modeling.

  Figure 7: Heatmaps of inter-variable Pearson correlations for Landsat/Sentinel-1 features per city.

Assessment of Reconstruction Difficulty: Autoencoder Analysis

To characterize which surface features or atmospheric artifacts (e.g., clouds, water) present difficulties for spatial ML models, the authors trained convolutional autoencoders on city-specific Landsat LST image patches. The reconstruction error, grouped by cloud mask categories, is systematically analyzed:

• Non-cloudy, non-water (“other”) pixels show the lowest mean error, indicating that cloud and water contamination—and to a lesser extent, cloud shadow—drive high reconstruction loss.
• In some coastal cities with abundant water coverage, water-pixel error distributions become more peaked due to greater representation and spatial homogeneity.

  Figure 8: Reconstruction error PDFs for Landsat LST autoencoders stratified by cloud-mask pixel category.

Use Cases and Limitations

Urban Heat MiniCubes is explicitly designed for ML/AI downstream use, targeting:

• Super-resolution: Using high-resolution but intermittent Landsat or Sentinel-1 data as ground truth to downscale temporally dense geostationary imagery via supervised or transfer learning frameworks [nguyen2022convolutional, lee2025guided].
• Inpainting: Filling cloud-contaminated pixels in thermal imagery by exploiting both multi-modal (e.g., SAR, reflectance) and temporal context, as well as leveraging all-weather microwave LST as physically informed priors [huber2024deep].
• Urban inequity and EJ analyses: Supporting public health, urban planning, and environmental justice research by quantifying fine-scale, intra-urban thermal variation linked to land cover, microclimate, and historical investment patterns.

Critical limitations include:

• LST is not always a linear proxy for near-surface air temperature, especially in heterogeneous or non-homogenous land cover and in non-clear-sky cases.
• Cloud contamination remains an important confounder; the fidelity of cloud-mask flagging and high spatiotemporal variability in cloud climatology constrain usable data fractions.
• MW LST provides all-weather coverage but is at too-coarse a resolution for direct micro-urban application—primarily supporting temporal completeness and as a feature for fusion-based reconstruction techniques.

Implications and Future Directions

The harmonized structure and systematic metadata of Urban Heat MiniCubes lower the barrier for state-of-the-art AI methods to operate on large-scale remote sensing archives, especially for transfer learning, representation learning, and foundation model development for the urban climate science domain. The dataset supports methodological advances in spatial downscaling, multi-modal domain adaptation, and urban process inference, while providing robust metadata for reproducibility and data transparency.

Ongoing work may extend the dataset temporally and to additional global cities, incorporate further sensor modalities (e.g., HLS-2/MSI for higher-frequency reflectance), or integrate ground-truth in situ observations for enhanced air temperature mapping.

Conclusion

Urban Heat MiniCubes establishes a new standard for urban-scale, multi-modal, AI-ready satellite data products. Through rigorous data harmonization, rich metadata, and structured access, it provides a reproducible foundation for advanced ML applications in urban heat analysis, city-scale process understanding, and climate resilience research. The resource is positioned to support both basic science—by enabling more nuanced multi-sensor fusion analyses—and applied policy work on urban heat mitigation and environmental equity.

---

Reference:

Urban Heat MiniCubes: An AI-Ready dataset for urban heat research (2606.11534)