TESSERA v2-1B-M: Distilled EO Foundation Model

• The paper introduces TESSERA v2-1B-M as a 21M-parameter student distilled from a 1B teacher, achieving a composite score of 0.611 on a 29-task benchmark.
• It details a Matryoshka-inspired design whereby each prefix of the 128-D embedding is trained to recapitulate the full teacher output, enabling effective truncation.
• The work highlights significant inference cost reductions through distillation, positioning v2-1B-M as a balanced, efficient deployment option.

TESSERA v2-1B-M is the medium-capacity distilled deployment model in the TESSERA v2 family of pixel-wise Earth-observation foundation models. It is distilled from a frozen 1B teacher trained on annual Sentinel-1 and Sentinel-2 time series, emits a 128-dimensional embedding with Matryoshka-compatible prefixes at $d\in\{16,32,64,128\}$, and is positioned as “a balanced default” for embeddings-as-data deployment. On the reported 29-task benchmark suite, it achieves the highest composite score among the systems compared, at $0.611$, exceeding AlphaEarth, TESSERA v1, and OlmoEarth-L (Feng et al., 4 Jul 2026).

1. Family position and nomenclature

The exact identifier TESSERA v2-1B-M is defined by the TESSERA v2 scaling paper. In that paper, the model is described as the 21M-parameter student distilled from a 1B-parameter pixel-wise teacher, with the suffix “M” denoting the medium-capacity student in the deployment ladder (Feng et al., 4 Jul 2026). By contrast, the original TESSERA paper established the broader TESSERA system as a dual-encoder remote sensing foundation model producing annual 10 m, 128-dimensional pixel embeddings, but it did not define a taxonomy involving v2, 1B, or M (Feng et al., 25 Jun 2025).

Variant	Parameters	Role
N	1M	edge/on-device
S	7M	low-resource
M	21M	balanced default
L	44M	provider-side global inference

The family organization is central to the meaning of the name. The “1B” component refers to the teacher from which the student is distilled, not to the student itself. The “M” component denotes the medium student tier. The paper does not assign a more elaborate semantic interpretation beyond this capacity and deployment role (Feng et al., 4 Jul 2026).

This naming also marks a conceptual shift from TESSERA v1. The 2025 system was already framed as a model-as-data product based on annual global 10 m embeddings; TESSERA v2 preserves that orientation but reorganizes the system around a large teacher plus compact students designed for recurring global inference and serving (Feng et al., 25 Jun 2025).

2. Teacher and student architecture

TESSERA v2 is a pixel-wise Earth observation foundation model operating on annual Sentinel-1/2 time series at 10 m resolution. The teacher is a 1B dual-branch pixel-wise encoder. Each modality branch linearly embeds valid observations, adds sinusoidal day-of-year positional encoding, runs a 4-layer Transformer, and aggregates the variable-length sequence by learned attention pooling. A 2-layer fusion Transformer then combines the two modality tokens into a single teacher embedding

$t \in \mathbb{R}^{T}, \qquad T=768,$

followed by a final affine-free LayerNorm. A batch-normalized projector is used only during pretraining (Feng et al., 4 Jul 2026).

The student family, including v2-1B-M, is specified less granularly. The paper states that the four students share the same architectural form and differ only in backbone width and depth, and that each emits a 128D embedding. Exact layer counts and widths for the 21M M student are not provided. A scaled-down version of the same pixel-wise temporal architecture is therefore a plausible implication, but the manuscript does not enumerate a full per-layer specification for v2-1B-M (Feng et al., 4 Jul 2026).

Architecturally, TESSERA v2 remains continuous with the first-generation system. The original TESSERA model also used modality-specific temporal encoders over Sentinel-1 VV/VH and Sentinel-2 10-band MSI annual sequences, then fused them into a 128-dimensional representation for downstream use (Feng et al., 25 Jun 2025). What changes in v2 is not the pixel-wise annual EO premise, but the scaling strategy and deployment path.

3. Pretraining recipe and downstream-driven scaling laws

The TESSERA v2 study fixes a pixel-wise Barlow Twins family and varies encoder size, projector size, and training data under matched compute. Teacher pretraining uses two views per annual pixel, with view length sampled as

$L \sim \mathrm{Uniform}\{8,16\},$

so each view is a sparse random temporal subsample of the year. Sentinel-1 and Sentinel-2 are processed in separate modality branches and then fused. The training recipe includes Barlow Twins, a mix-up consistency regularizer, occasional whole-modality dropout, and global shuffling of pixels across tiles. The reported teacher setup uses one epoch over about 4.2 billion annual pixels, global batch size $131{,}072$, 512 GPUs, FSDP, and bf16 (Feng et al., 4 Jul 2026).

The paper’s principal claim is that model selection by pretraining loss is ineffective for this family. Across 395 runs evaluated on 15 downstream tasks, converged Barlow Twins loss correlates only weakly with the composite downstream score:

$$\text{Pearson } r = -0.18, \qquad \text{Spearman } \rho = -0.16.$$

The reported consequence is operational rather than merely diagnostic: selecting models by loss instead of downstream score would require roughly 254\% more compute to reach the same downstream score (Feng et al., 4 Jul 2026).

The resulting scaling laws are asymmetric. As pretraining compute grows, the encoder and the data should grow together, while the projector should remain approximately fixed:

$$N_{\mathrm{enc}}^{\star} \propto C^{0.36}, \qquad D^{\star} \propto C^{0.63}, \qquad N_{\mathrm{proj}}^{\star} \propto C^{0.00}.$$

The reported confidence intervals are $[0.29,0.45]$ for the encoder exponent, $[0.53,0.70]$ for the data exponent, and $[-0.03,0.08]$ for the projector exponent (Feng et al., 4 Jul 2026).

These results explain why v2-1B-M exists in its documented form. Rather than deploying a large self-supervised encoder directly, TESSERA v2 spends pretraining compute on a large teacher and matched data, keeps the projector disposable, and then distills a compact student for serving.

4. Distillation and Matryoshka representation structure

TESSERA v2-1B-M is not trained on task labels. It is trained by task-agnostic representation distillation from the frozen 1B teacher using a distillation corpus of about 200 million annual pixels. If $0.611$0 is the student embedding and $0.611$1 is the frozen teacher embedding with $0.611$2, then for prefix set

$0.611$3

the student objective is

$0.611$4

where each temporary linear head

$0.611$5

maps a student prefix into the teacher space (Feng et al., 4 Jul 2026).

This construction defines the model’s Matryoshka property. The first $0.611$6 coordinates of the 128D student vector are not trained merely to approximate the teacher’s first $0.611$7 coordinates; instead, each prefix is trained to reconstruct the full 768D teacher embedding. At inference time, the embedding can therefore be truncated directly to 16, 32, or 64 coordinates as a storage-accuracy tradeoff, without retraining or auxiliary decoding (Feng et al., 4 Jul 2026).

The empirical retention rates reported for the student family are central to the model’s practical significance. A 16-dimensional prefix retains about 92\% of the full 128D composite score at 1/8 of the storage; 32D retains about 97\%; and 64D retains about 99\% (Feng et al., 4 Jul 2026). The paper further notes that truncation hurts fine-class-count and regression tasks more than land-cover and change-detection tasks, making 64D the suggested default when task difficulty is high and storage is not the dominant constraint.

The paper also reports a negative result that motivates this design: naive Matryoshka-style prefix losses during self-supervised pretraining fail, because redundancy-reduction objectives identify only a subspace up to rotation. Distillation from a frozen teacher is therefore used to make prefix truncation semantically stable (Feng et al., 4 Jul 2026).

5. Benchmark performance and serving economics

The evaluation framework has two layers: a 15-task AlphaEarth suite spanning classification, segmentation, regression, and change detection, and a held-out 14-dataset evaluation not used in development. Together these form a 29-task benchmark. Downstream models are intentionally light: a 2-layer MLP for pixel-wise tasks and a simple CNN with fewer than 2M parameters for patch-level tasks (Feng et al., 4 Jul 2026).

On the full 29-task suite, TESSERA v2-1B-M achieves the best reported composite score:

$0.611$8

compared with

$0.611$9

The paper explicitly notes that v2-1B-M is the smallest of these four compared systems (Feng et al., 4 Jul 2026).

On the 15-task AlphaEarth suite alone, the ordering is slightly different. The L student leads at

$t \in \mathbb{R}^{T}, \qquad T=768,$0

while

$t \in \mathbb{R}^{T}, \qquad T=768,$1

All four exceed the directly comparable embedding products

$t \in \mathbb{R}^{T}, \qquad T=768,$2

This means that v2-1B-M is the strongest aggregate model on the full benchmark suite, but not the strongest member of its own family on every sub-benchmark (Feng et al., 4 Jul 2026).

The deployment rationale is reinforced by the paper’s global inference economics. A single global annual 10 m pass with the 1B teacher would cost about 50 H100-years. Inference cost follows an empirical near-linear power law in encoder parameters,

$t \in \mathbb{R}^{T}, \qquad T=768,$3

with $t \in \mathbb{R}^{T}, \qquad T=768,$4 in millions of parameters. The distilled students, by contrast, require only 0.04 to 2 H100-years across the family (Feng et al., 4 Jul 2026). The paper does not isolate a separate exact H100-year figure for the M student, but it places v2-1B-M inside that family-level range.

6. Embeddings-as-data deployment, continuity with TESSERA v1, and external use

TESSERA v2-1B-M is designed for embeddings-as-data deployment rather than end-user backbone inference. The paper states that annual global pixel embeddings are to be precomputed and served through GeoTESSERA, and that, upon completion of training, the authors plan to release v2 global embeddings covering 2017–2025 (Feng et al., 4 Jul 2026). This is a direct continuation of the original TESSERA program, whose principal deliverable was a set of precomputed annual global representation maps at 10 m and 128 channels for 2017–2024, intended for frozen-encoder downstream usage and public release (Feng et al., 25 Jun 2025).

The v1 system already established the main EO design commitments later preserved in v2: annual pixel-centric Sentinel-1/Sentinel-2 representations, a 128-dimensional embedding per pixel, and an open-source, privacy-preserving workflow in which users can consume precomputed embeddings rather than upload raw EO time series to a centralized service (Feng et al., 25 Jun 2025). TESSERA v2-1B-M should therefore be understood not as a departure from that model, but as a deployment-optimized refinement of it.

External downstream evidence also points to the utility of the broader TESSERA embedding family. A later LCZ mapping study across five Swiss cities used annual 10 m, 128-band, precomputed TESSERA embeddings and reported that TESSERA outperformed both S1S2 composites and AlphaEarth in the two same-year experiments, reaching test IoU 0.69 in the five-city setting and test IoU 0.82 in the Bern setting (Ko et al., 18 Jun 2026). That study, however, did not identify the checkpoint as TESSERA v2-1B-M; it therefore supports the TESSERA family’s downstream relevance, but not an exact per-variant attribution.

A final point of terminology follows from the publication history. The original 2025 TESSERA paper documented the model family, annual 10 m embeddings, and a large-scale implementation, but not a formal v2-1B-M naming scheme (Feng et al., 25 Jun 2025). The exact variant identity is introduced only by the 2026 scaling paper, where v2-1B-M is defined as the 21M-parameter medium distilled student and recommended implicitly as the practical default because it is both a “balanced default” and the best aggregate performer on the 29-task suite (Feng et al., 4 Jul 2026).