TESSERA v2 Earth-Observation Model

• TESSERA v2 is a pixel-wise Earth-observation model that fuses Sentinel-1 and Sentinel-2 data to produce analysis-ready global annual embeddings.
• It applies a downstream-driven scaling study that optimizes encoder capacity and data allocation, moving beyond traditional pretraining loss metrics.
• The architecture employs a teacher–student distillation pipeline with a Matryoshka prefix structure to deliver flexible, storage-adaptive representations for diverse remote sensing tasks.

TESSERA v2 is the second-generation TESSERA Earth-observation foundation model family: an open, pixel-wise, Sentinel-1/Sentinel-2 representation model designed to produce global annual embeddings as data rather than only releasing a backbone. Its defining contribution is a downstream-driven scaling study that treats downstream utility, rather than self-supervised pretraining loss, as the central optimization target for pixel-wise Earth observation models. Within a fixed pixel-wise Barlow Twins family, the study derives a practical compute-allocation rule—scale the encoder and the data together, while keeping the projector fixed—and couples that rule to a teacher–student distillation pipeline that yields compact, storage-adaptive embedding products with Matryoshka-style nested prefixes (Feng et al., 4 Jul 2026).

1. Genealogy and problem formulation

The original TESSERA system introduced a remote sensing foundation model that learns global, 10-meter-resolution pixel representations from unlabeled Sentinel-1 SAR and Sentinel-2 MSI time series. It used two parallel Transformer-based encoders, one for Sentinel-1 and one for Sentinel-2, fused the modality-specific annual embeddings with an MLP, and produced a 128-dimensional embedding per 10 m pixel per year as a precomputed global representation map covering 2017–2024 (Feng et al., 25 Jun 2025).

TESSERA v2 preserves that pixel-wise, model-as-data orientation, but re-centers the research problem. The paper does not present v2 primarily as a larger successor model; rather, it asks how a fixed pretraining budget should be allocated across encoder size, projector size, and training data size to maximize downstream performance. This focus is specific to Earth observation. Sentinel-1 and Sentinel-2 observations are irregularly sampled, optical data are often cloud-obscured, many downstream tasks have scarce labels, and end users typically face substantial preprocessing burdens in harmonization, masking, temporal compositing, and inference. TESSERA v2 therefore frames the target artifact as an analysis-ready global embedding layer that fuses multi-source observations into reusable pixel representations (Feng et al., 4 Jul 2026).

2. Why downstream-driven scaling is central in Earth observation

The central methodological claim of TESSERA v2 is that scaling laws for pixel-wise Earth observation models should be fit to downstream performance rather than to pretraining loss. The motivating argument is that, in Earth observation, the self-supervised objective can be dominated by cloud patterns and orbital sampling effects, so optimization of the training loss need not track the practical usefulness of the learned representation. This directly challenges a common assumption imported from other foundation-model settings: that lower pretraining loss is an adequate proxy for better representations.

The paper situates this issue within the recent rise of pixel-wise Earth-observation foundation models based on generated spatial embeddings. Those models have achieved strong downstream results, but the scaling behavior of the underlying representation learners—and the corresponding compute-optimal tradeoffs among encoder capacity, projector capacity, and data volume—had remained poorly understood. TESSERA v2 addresses that gap with a controlled scaling study intended to yield an empirically grounded recipe for budget allocation and deployment (Feng et al., 4 Jul 2026).

3. Controlled scaling study and evaluation protocol

The study comprises 395 pretraining runs on 1,024 GH200 superchips within a fixed pixel-wise Barlow Twins family, with each run evaluated on 15 downstream tasks spanning classification, segmentation, regression, and change detection. Compute is measured with an iso-FLOP-style budget. The compute axis is defined as

$$C \;=\; 12\, D \, \bigl(N_{\mathrm{enc}}L_{\mathrm{ref}} + N_{\mathrm{proj}}\bigr), \qquad L_{\mathrm{ref}} = 240$$

so the budget scales with the number of training examples or temporal windows $D$, encoder size $N_{\mathrm{enc}}$, projector size $N_{\mathrm{proj}}$, and the fixed reference sequence length $L_{\mathrm{ref}}$.

Downstream evaluation uses chance-adjusted metrics. For classification, the paper uses

$\frac{\text{balanced accuracy}-1/K}{1-1/K},$

and for regression it uses

$\max(0, R^2).$

The task scores are averaged into a single composite downstream score. This composite is the quantity used to compare runs and fit scaling behavior, making downstream utility the explicit dependent variable of the scaling study rather than an after-the-fact validation criterion (Feng et al., 4 Jul 2026).

4. Empirical scaling laws and the rejection of loss-based model selection

The study’s strongest negative result is that pretraining loss is a poor proxy for downstream performance. The reported correlation values are Pearson $r=-0.18$ and Spearman $\rho=-0.16$, summarized in the abstract as $|\mathrm{Pearson}\ r| < 0.2$. The practical consequence is that choosing models by lower pretraining loss wastes compute; the paper reports that loss-based selection needs about 254 more compute to reach the same downstream score as downstream-based selection.

To derive compute-optimal allocations, the paper groups runs into compute buckets and fits parabolic curves in log-size. The resulting power laws are

$D$0

with confidence intervals $D$1 for the encoder exponent, $D$2 for the data exponent, and $D$3 for the projector exponent. The interpretation is direct: as compute grows, the encoder and the data should grow together, while the projector should remain roughly constant. The paper explicitly notes that this recovers a Chinchilla-like balance, since $D$4.

This result provides the paper’s operational prescription for pretraining pixel-wise Earth-observation models: spend additional budget on encoder capacity and matched data, not on a larger projector. It also clarifies a second methodological point. In this setting, downstream evaluation is not merely a benchmarking appendage; it is the model-selection criterion required to use compute efficiently (Feng et al., 4 Jul 2026).

5. Teacher architecture, distillation, and Matryoshka representations

TESSERA v2 separates the model best suited for training from the model best suited for global serving. The reported teacher is a 1B-parameter pixel-wise model with a dual-branch Sentinel-1/Sentinel-2 encoder, per-modality Transformers, a fusion Transformer, and a teacher embedding dimension of 768. It is trained on about 4.2 billion pixels for one epoch with global batch size 131,072 on 512 GPUs using bf16. The broader family includes 0.5B and 1B models, with a 2B model in training.

During representation learning, TESSERA v2 remains within the Barlow Twins redundancy-reduction family. The paper characterizes the objective with the standard form

$D$5

where $D$6 is the cross-correlation matrix between the two view embeddings. The diagonal terms enforce agreement between corresponding dimensions, while the off-diagonal terms discourage redundancy.

A key deployment problem is that this self-supervised objective does not impose an ordering on representation coordinates. The paper states that nested embeddings are not learned by adding prefix losses during self-supervised pretraining, because the representation space is only defined up to rotation. TESSERA v2 instead freezes the teacher and distills it into compact students using prefix heads. The distillation loss is

$D$7

with $D$8, where $D$9 is the student prefix of length $N_{\mathrm{enc}}$0, $N_{\mathrm{enc}}$1 is a linear head mapping that prefix to the teacher space, and $N_{\mathrm{enc}}$2 is the frozen teacher embedding. Each prefix is trained to reconstruct the full teacher embedding, so the embedding acts as a rate–distortion code: shorter prefixes are cheaper and less accurate, while longer prefixes retain more of the teacher signal.

The distilled student family is as follows.

Variant	Parameters	Role
Teacher	1B	Representation-quality model
N	1M	Distilled student
S	7M	Distilled student
M	21M	Distilled student
L	44M	Distilled student

This architecture–distillation split is fundamental to the v2 system design. Large teachers absorb the pretraining budget implied by the scaling laws, while compact students make global embedding serving practical (Feng et al., 4 Jul 2026).

6. Benchmark performance, storage tradeoffs, and deployment trajectory

On the 15-task AlphaEarth suite, the strongest model is the 1B-L student, with composite score 0.584 and mean rank 2.4, ahead of AlphaEarth at 0.560 and Tessera v1 at 0.541. The other student scores are 0.581 for M, 0.570 for S, and 0.558 for N. On the full 29-task suite, the standout model is TESSERA v2-1B-M, with composite score 0.611, outperforming AlphaEarth at 0.576, Tessera v1 at 0.574, and OlmoEarth-L at 0.562. The abstract emphasizes that the 21-million-parameter distilled TESSERA v2-1B-M in aggregate outperforms all open and proprietary models tested, including some that are orders of magnitude larger. The compared baselines include AlphaEarth, Tessera v1, OlmoEarth, MOSAIKS, Presto, and several open RSFMs and generic backbones (Feng et al., 4 Jul 2026).

The deployment mechanism is the Matryoshka prefix structure. Students expose 16-, 32-, 64-, and 128-dimensional prefixes. The key practical result is that the 16-dimensional prefix retains about 92% of the 128-dimensional score at 1/8 of the storage; the 32-dimensional and 64-dimensional prefixes retain about 97% and about 99%, respectively. The paper notes that the students lie on the Pareto frontier of score versus storage. This makes embedding dimensionality a serving-time control variable rather than a fixed design choice.

The release plan follows the same embeddings-as-data philosophy. The reproducibility statement says the project will release training code, the controlled scaling-study sweeps, distilled pixel-wise student checkpoints, the AlphaEarth-suite evaluation harness, and a frozen embedding-as-data product, with code at https://github.com/ucam-eo/tessera. The abstract states that, upon completion of training, the authors plan to release v2 global embeddings covering 2017–2025 (Feng et al., 4 Jul 2026).

A subsequent application study in fine-scale Local Climate Zone mapping provides an early indication of downstream utility in a concrete segmentation workflow. In that study, yearly, gap-free TESSERA embeddings at 10 m resolution with 128 bands and 8-bit quantization are used as a drop-in predictor dataset for an attention-based U-Net. Across five Swiss cities, TESSERA achieves the best same-year performance among the compared inputs, with test IoU 0.69 in the multi-city transfer setting and 0.82 in the Bern setting with a 78 m reference map, outperforming both Sentinel-1/2 composites and AlphaEarth. The same paper also reports that year-to-year transfer remains an open challenge and that Sentinel-1/2 seasonal composites are the most temporally stable baseline (Ko et al., 18 Jun 2026).

Taken together, these results position TESSERA v2 as both a scaling study and a deployable Earth-observation representation system. Its central prescription is explicit: do not choose models by pretraining loss; scale encoder and data together while keeping the projector fixed; train large teachers; distill them into compact students; and expose nested prefixes so that storage and performance can be traded off at serving time.