---
title: 'ChronoEarth-Benchmark: Evaluation Suite'
url: https://www.emergentmind.com/topics/chronoearth-benchmark
type: topic
---

# ChronoEarth-Benchmark: Evaluation Suite

ChronoEarth-Benchmark is a unified evaluation suite for spatiotemporal hyperspectral representation learning built on top of ChronoEarth-492K, a temporally calibrated archive derived from NASA’s EO-1 Hyperion mission. It was introduced to address the absence of a standardized, large-scale, temporally grounded benchmark for hyperspectral self-supervised learning and temporal modeling, and it organizes evaluation into static, short-horizon temporal, and long-horizon temporal regimes using six open-source geospatial products covering land cover, crop type, forest dynamics, and soil properties [2605.15666].

## 1. Data foundation and temporal calibration

ChronoEarth-Benchmark is inseparable from ChronoEarth-492K, the underlying hyperspectral corpus from which its temporal structure is derived. ChronoEarth-492K contains **492,354 patches** from **185,398 unique global locations** over **2001 to 2017**, with **56,491** locations having at least two observations and **28,786** locations having three or more timestamps. The source sensor is **NASA EO-1 Hyperion Level-1T terrain-corrected scenes**, described as the world’s longest continuous spaceborne hyperspectral archive to date, with **more than 240 narrow spectral bands**, spectral range roughly **400–2500 nm**, and **30 m** spatial resolution [2605.15666].

Patch construction follows a deterministic geospatial indexing pipeline. Each Level-1T scene is kept in its native **UTM projection**, resampled to a **uniform 30 m pixel size**, divided into **non-overlapping \(128 \times 128\)** patches, and aligned to a **fixed global grid** within its UTM zone. Each patch receives a deterministic UID of the form `[\text{UTM\_zone} : \text{column} : \text{row}]`, and patches sharing the same UID across dates are treated as repeated observations of the same location. This UID-based gridding is the benchmark’s core mechanism for temporal sequence formation [2605.15666].

The benchmark inherits the corpus preprocessing decisions of ChronoEarth-492K. Only Level-1T acquisitions with reported cloud coverage below **10%** were retained. Hyperion’s original **242 bands** were filtered to **155 spectrally consistent bands** following Datt et al. (2003), with unstable and low-signal bands removed. The retained samples are released as **\(128 \times 128 \times 155\)** GeoTIFF cubes, and the archive is organized hierarchically as `dataset/<Region>/<UID>/<UID>_<YYYYDDD>.TIF`, accompanied by `dataset/metadata.parquet` [2605.15666].

A notable design choice is that spatial consistency is achieved through fixed-grid patching rather than learned co-registration. The paper states that Level-1T products are already orthorectified and radiometrically calibrated by USGS, so **no additional atmospheric corrections** and **no additional geometric corrections** were applied. This suggests that temporal alignment is benchmarked under realistic archive irregularities rather than under a heavily normalized remote-sensing preprocessing regime [2605.15666].

## 2. Temporal regimes and benchmark logic

ChronoEarth-Benchmark is explicitly partitioned into three temporal regimes: **Static**, **Short-Horizon temporal (SH)**, and **Long-Horizon temporal (LH)**. This partition is the benchmark’s central methodological contribution, because it distinguishes spatial-spectral representation quality from temporal aggregation and from explicit future prediction [2605.15666].

In the **static** regime, a sample pairs a label map or label vector with **one spatially aligned hyperspectral observation from the same year**. This evaluates representation learning without temporal aggregation. In the **short-horizon** regime, the model receives **multiple temporally adjacent observations** corresponding to the **same label**; SH is therefore defined by temporal aggregation under fully observed supervision for the target year, not by a fixed interval in days or months. In the **long-horizon** regime, the task becomes explicit future prediction: observations from **2001 to year \(n\)** are used to predict labels at **year \(n+1\)** [2605.15666].

Sequence formation is UID-centric. All patches with the same UID across timestamps are grouped into temporal sequences, then filtered with a **minimum observation threshold of 2** for benchmark temporal evaluation. Sequences are normalized to a fixed length \(T\) by **sub-sampling or zero-padding**, and evaluation uses deterministic sequence fixing for fair comparison. The paper also notes that the archive is **sparse, non-uniform, irregularly sampled**: most acquisition intervals are **under one year**, while **over one-third of multi-temporal locations** span **more than two years**. The benchmark is therefore designed for irregular temporal archives rather than dense revisit schedules [2605.15666].

The temporal decomposition is reflected in the evaluation settings. SH experiments compare **\(T=1\)** against **\(T \le 4\)**. LH experiments use historical context lengths **\(T \le 2\)**, **\(T \le 4\)**, and **\(T \le 8\)**. This structure makes the benchmark useful for separating three failure modes: insufficient static spatial-spectral encoding, inability to exploit nearby temporal context, and inability to convert historical observations into future-label prediction [2605.15666].

## 3. Label products, task inventory, and alignment rules

ChronoEarth-Benchmark integrates six enumerated geospatial products, although one sentence in the paper refers to “seven publicly available geospatial products,” a reporting inconsistency explicitly noted in the source synthesis. The actual datasets listed in the tables and appendices are **GFC**, **ISDASoil**, **CDL**, **CORINE**, **NLCD-S**, and **CLCD** [2605.15666].

The benchmark first constructs a dictionary of ChronoEarth UIDs independent of time, then reconstructs the corresponding UTM-zone grid for each external label layer and matches UIDs to extract overlapping labels. Temporal alignment rules vary by product: **same-year alignment** for annual products in static tasks, **same-label-year multiple observations** for SH, **history through year \(n\) to predict year \(n+1\)** for LH, **nearest-epoch mapping** for CORINE, **geographic-overlap-only** for ISDASoil, and **between-time change labels** for GFC [2605.15666].

| Product | Task type | Benchmark role |
|---|---|---|
| **GFC** | Temporal change detection | **17,214** paired samples from **319** locations; retains patches with **>10% changed pixels** |
| **ISDASoil** | Static and SH multi-label classification | **17,808** pairs from **5,502** locations; uses intersecting soil texture classes |
| **CDL** | Static segmentation, SH, LH | **10,162** pairs from **2,864** locations; final taxonomy **15 agricultural classes + 1 background** |
| **CORINE** | Static multi-label classification; spatial/temporal generalization | **19,774** pairs from **9,706** locations; **44** classes mapped to **19** categories |
| **NLCD-S** | Static segmentation, SH, LH | **55,476** pairs from **24,662** locations; **16 thematic classes** |
| **CLCD** | Static segmentation, SH, LH | **17,614** pairs from **9,752** locations; **nine major land-cover types** |

For segmentation products, the benchmark applies entropy filtering to remove effectively homogeneous patches. The paper gives the normalized entropy as

$$
H_{\text{norm}}(Y)= - \frac{1}{\log C} \sum_{c=1}^{C} p_c \log p_c,
\qquad
p_c = \frac{1}{HW} \sum_{i,j} \mathbf{1}(Y_{i,j}=c),
\qquad \sum_c p_c = 1,
$$

with threshold \(\tau = 0.1\), and samples below this threshold are removed [2605.15666].

This task construction yields a heterogeneous downstream suite: **static segmentation**, **static multi-label classification**, **short-horizon temporal aggregation**, **long-horizon future prediction**, **spatial-temporal generalization**, **continental generalization**, and **forest change detection**. A plausible implication is that ChronoEarth-Benchmark is not merely a temporal benchmark, but a staged test of whether hyperspectral representations remain useful under increasingly realistic temporal and distribution-shift constraints [2605.15666].

## 4. Evaluation protocol and model families

ChronoEarth-Benchmark uses a **distance-aware grouping strategy** based on **EO-1 orbital swaths** to prevent spatial leakage. Patches from the same observation are grouped into spatial units, overlapping units are merged into connected components, and train/validation/test are assigned at the **component level**. The stated aim is to ensure **no overlapping spatial regions across splits**, **no patch-level leakage**, and geographically distinct test partitions. The benchmark therefore defines an explicitly **spatial OOD** setting rather than a random split [2605.15666].

Evaluation metrics are task-specific: **mIoU** for segmentation, **mAP** for multi-label classification, and **F1** in addition to mIoU for **GFC**. The paper does not provide formal mathematical definitions of these metrics, but it standardizes downstream evaluation by using a **linear classification head** for multi-label classification and **UPerNet** for segmentation. Comparability is further enforced by using all models at **ViT-Base** scale, deterministic temporal sequence fixing at evaluation time, common optimizer settings, and validation-based hyperparameter tuning [2605.15666].

The static baselines are **SpectralViT**, **HyperSigma**, **DOFA**, **LESSViT**, **SatMAE** (adapted), and **DINOv3** (adapted). For temporal evaluation, the paper compares three temporal adaptation strategies: **Max Pooling**, **AttentionPool**, and **Temporal SSL**. Beyond max pooling, the temporal module is evaluated only with **SpectralViT** [2605.15666].

The second-stage temporal pretraining protocol uses the **28,786** locations with at least three timestamps. Given a sequence \(\{x_1,\ldots,x_T\}\), the model predicts \(x_T\) from \(\{x_1,\ldots,x_{T-1}\}\) using causal masking and an **\(\ell_2\)** reconstruction loss in pixel space. The encoder is frozen, and only the temporal module plus reconstruction decoder are trained for **50 epochs**. The token tensor is defined as \(Z \in \mathbb{R}^{B \times T \times N \times D}\), then reshaped to \(\mathbb{R}^{(B \cdot N) \times T \times D}\) for temporal attention over the \(T\) dimension [2605.15666].

Downstream optimization is standardized: **AdamW**, learning-rate sweep \(\{1\times10^{-5}, 3\times10^{-5}, 5\times10^{-5}, 8\times10^{-5}, 1\times10^{-4}\}\), batch size **32**, **5%** warmup, cosine annealing, and weight decay **\(5 \times 10^{-2}\)**. This standardization is central to the benchmark’s function as a comparative evaluation suite rather than a collection of loosely comparable datasets [2605.15666].

## 5. Empirical behavior and benchmark discriminativeness

The benchmark produces distinct rankings across static, short-horizon, and long-horizon regimes. On static tasks, **LESSViT** is the strongest model on the major segmentation datasets: **54.84** mIoU on **CLCD**, **23.91** on **CDL**, and **33.59** on **NLCD-S**, compared with **46.04**, **19.38**, and **32.47** for **SpectralViT**. On **ISDASoil**, **SpectralViT** reaches **57.70** mAP, while the supervised SpectralViT baseline is slightly higher at **57.79**, one of the few cases where SSL does not clearly dominate [2605.15666].

The benchmark’s generalization settings expose a more nuanced picture. On **CORINE**, performance degrades from **ID** to **T-OOD** to **S-OOD** to **ST-OOD**; the paper highlights that **spatial shift causes the largest drop**, making geographic variability the main generalization bottleneck. On **GFC**, **SpectralViT** outperforms the alternatives with **19.29** mIoU / **32.34** F1 in-domain and **16.13** / **27.78** OOD. The authors attribute this to sparse change supervision favoring stronger spatial aggregation [2605.15666].

In the short-horizon setting, adding adjacent temporal observations usually helps. Representative results include **LESSViTmax** on **CLCD**, improving from **51.19** at \(T=1\) to **54.69** at \(T \le 4\), and on **CDL**, from **24.80** to **26.15**. Among controlled SpectralViT variants, **Temporal SSL > AttentionPool > Max Pooling** on most segmentation tasks, while **ISDASoil** benefits less from temporal SSL. The paper links this to the reconstruction objective being less aligned with classification via the global token [2605.15666].

The long-horizon regime is the benchmark’s most distinctive evaluative layer. On **CLCD**, **SpectralViTtemporal** is best at all tested context lengths, reaching **43.61 / 51.26 / 55.61** for \(T \le 2\), \(T \le 4\), and \(T \le 8\). On **NLCD-S**, the same model reaches **37.34 / 39.69 / 40.76**, substantially ahead of the alternatives. The main exception is **CDL**, where crop-type forecasting is unstable: some models degrade with longer context, and the paper suggests that older observations can become stale or harmful for rapidly changing agricultural targets [2605.15666].

These results make ChronoEarth-Benchmark discriminative in a specific technical sense. Strong static hyperspectral encoders are not automatically strong temporal predictors; nearby temporal aggregation is easier than future prediction; and temporal self-supervision is most beneficial when the downstream target evolves slowly enough that historical context remains informative. This suggests that the benchmark succeeds in isolating long-horizon temporal modeling as a distinct capability rather than as a trivial extension of static spectral-spatial representation learning [2605.15666].

## 6. Position within Earth benchmark research and known limitations

ChronoEarth-Benchmark occupies a distinct position within the recent Earth-science benchmark landscape. Unlike **MSEarth**, which targets graduate-level multimodal scientific comprehension from figures and refined captions, ChronoEarth-Benchmark is a hyperspectral representation-learning benchmark with explicit static/SH/LH temporal protocols rather than a figure-centered MLLM reasoning benchmark [2505.20740]. Unlike **OmniEarth-Bench**, which evaluates multimodal reasoning across six Earth spheres and cross-sphere interactions, ChronoEarth-Benchmark does not define perception, general reasoning, scientific-knowledge reasoning, or CoT tiers; its core abstraction is temporally calibrated remote-sensing sequences linked to downstream geospatial products [2505.23522]. Unlike **EarthSE**, which assesses LLM-based Earth scientific exploration through QA and multi-turn dialogue, ChronoEarth-Benchmark is not a literature-grounded language benchmark and does not evaluate open-ended methodology induction or limitation analysis [2505.17139]. It is closer in spirit to **ChaosBench** in its emphasis on long-horizon temporal difficulty, but the two benchmarks operate on different substrates: ChaosBench evaluates global subseasonal-to-seasonal forecasting on gridded geophysical fields, whereas ChronoEarth-Benchmark evaluates irregularly sampled hyperspectral patch sequences under fine-tuning protocols [2402.00712]. A further distinction can be made from throughput-oriented Earth-system benchmarking such as global fully coupled kilometer-scale ICON runs, where the benchmark objective is exascale simulation throughput rather than spatiotemporal hyperspectral representation quality [2511.02021].

The benchmark’s limitations are explicitly discussed. Temporal modeling is implemented as an **extension on top of pretrained spatial backbones**, not as a native end-to-end spatiotemporal architecture. The observations are **sparse and irregular**, Hyperion’s narrow swath produces partial coverage and nodata handling requirements, and regional sample counts are uneven. Label construction also introduces approximation and noise: **CORINE** uses nearest-epoch mapping, **ISDASoil** assumes soil texture is stable during **2001–2017**, class imbalance is substantial in several products, and crop forecasting is sensitive to stale temporal context [2605.15666].

Two further caveats shape its interpretation. First, the benchmark paper contains a small but consequential reporting inconsistency between **six** and **seven** source products; the enumerated products are six. Second, while reproducibility is emphasized through release of the processing pipeline, deterministic UID indexing, temporal sequence construction, and split generation, some exact split counts and implementation specifics remain in the appendices. The assembled ChronoEarth dataset and benchmark are to be released under an **open academic license permitting non-commercial research use**, which situates the benchmark as a research infrastructure artifact as much as a leaderboard instrument [2605.15666].

Source: https://www.emergentmind.com/topics/chronoearth-benchmark