rs-embed: Unified Remote Sensing Embedding

• rs-embed is a Python library that provides a unified, ROI-centric interface for obtaining remote sensing foundation model embeddings.
• It standardizes heterogeneous model release formats, sensor specifications, and output structures to enable reproducible cross-model comparisons.
• The library supports both on-the-fly and precomputed models, optimizing batch workflows through rigorous validation and export capabilities.

Searching arXiv for the rs-embed paper and closely related remote-sensing embedding context. rs-embed is a Python library for obtaining remote sensing foundation model embeddings through a unified, region of interest (ROI)-centric interface. It was introduced to reduce the practical and benchmarking costs created by heterogeneous model release formats, platforms, interfaces, input specifications, output structures, and temporal semantics in the remote sensing foundation model ecosystem. In operational terms, rs-embed lets users specify a region of interest, a time specification, and a model identifier, then retrieves a standardized embedding object through a single interface described by the authors as “any model, any place, any time” (Ye et al., 27 Feb 2026).

1. Scope, motivation, and problem addressed

The library is positioned as an interoperability and benchmarking layer for remote sensing foundation models. Its motivation is the rapid growth of such models alongside substantial heterogeneity in how they are distributed and invoked. The paper identifies several concrete sources of fragmentation: some models release only precomputed embeddings, others release only model weights; some are distributed through relatively standardized channels such as Hugging Face, while others rely on custom repositories and runtimes; models differ in expected sensors, spectral bands, resolutions, preprocessing pipelines, and temporal support; and outputs may be fixed-length pooled vectors or spatial feature grids (Ye et al., 27 Feb 2026).

This fragmentation creates a systems-level problem rather than a purely modeling problem. Practical use becomes costly and error-prone because researchers must manually reconcile data backends, imagery sources, band subsets, compositing policies, resizing or cropping behavior, and output normalization for each model separately. rs-embed addresses this by making the ROI and temporal query the primary abstraction, rather than the idiosyncrasies of an individual model. The intended benefits are rapid experimentation across many models, reproducible model comparison, large-scale embedding generation, and more controlled downstream evaluation (Ye et al., 27 Feb 2026).

The library therefore standardizes embedding retrieval rather than attempting to standardize model semantics. This distinction is important: rs-embed normalizes invocation, data provisioning, validation, and output representation, but it does not eliminate fundamental differences in training data, sensor assumptions, or representational objectives across remote sensing foundation models (Ye et al., 27 Feb 2026).

2. ROI-centric abstraction and query specification

The user-facing design is centered on four specifications: spatial, temporal, output, and sensor. These define the query independently of the underlying model implementation.

Specification	Main forms	Purpose
Spatial Spec	BBox, PointBuffer	Defines ROI and CRS
Temporal Spec	TemporalSpec.year(...), TemporalSpec.range(start, end)	Defines yearly or interval-based time
Output Spec	pooled, grid	Chooses vector or spatial tensor output
Sensor Spec	source, bands, scale_m, cloudy_pct, fill_value, median or [mosaic](https://www.emergentmind.com/topics/mosaic-b2e93c3d-34d6-4f4a-9633-8f0432aa236b)	Defines imagery requirements and preprocessing policy

The spatial extent can be expressed either as a bounding box or as a point with buffer radius. The specification includes a coordinate reference system such as EPSG:4326. Validation includes latitude/longitude range checks, coordinate ordering checks, positive buffer radius, and limits on area and scale. This makes the spatial object the canonical query geometry regardless of which model is ultimately used (Ye et al., 27 Feb 2026).

The temporal specification supports either a year or a date range. The date-range form is defined as a left-closed, right-open interval,

$[start, end)$

and invalid dates or reversed intervals are rejected. Temporal specifications interact with observation synthesis policies such as median compositing or mosaic selection after time-window filtering, so the temporal abstraction is also a reproducibility mechanism rather than merely a convenience wrapper (Ye et al., 27 Feb 2026).

The output specification determines whether the embedding is returned as a pooled vector or a grid. In pooled mode, the output is a fixed-length vector

$z \in \mathbb{R}^d,$

typically after mean pooling. In grid mode, the output is a spatial tensor

$z \in \mathbb{R}^{h \times w \times d}.$

The standardized data field is represented in implementation/export conventions as either ($D,$) for pooled output or ($D,h,w$) for a spatial feature grid. The paper emphasizes that, except for AlphaEarth and Tessera, the embedding grid size $h \times w$ is generally not aligned with the input image resolution $H \times W$ (Ye et al., 27 Feb 2026).

The sensor specification makes raw imagery requirements explicit. It includes data source or collection, requested bands, spatial resolution via scale_m, cloud threshold via cloudy_pct, fill value, compositing method (median or mosaic), and optional input quality checking through check_input. This captures data-selection and preprocessing policy in a form that can be applied consistently across models (Ye et al., 27 Feb 2026).

3. Software architecture and model abstraction

The architecture is organized around a Provider Layer, an Embedder Layer, and orchestration/export logic. This decomposition is central to how rs-embed hides backend and model heterogeneity behind a common interface (Ye et al., 27 Feb 2026).

The Provider Layer abstracts remote sensing data access. It wraps cloud APIs such as Google Earth Engine and converts their outputs into standardized numeric tensors. Its responsibilities include spatiotemporal filtering according to SpatialSpec, TemporalSpec, and SensorSpec, projection and resampling, compositing via median or mosaic, and conversion into a consistent tensor format,

$(C, H, W),$

using NumPy/Xarray. The paper also notes that this abstraction is designed to support extension to platforms such as Microsoft Planetary Computer (Ye et al., 27 Feb 2026).

The Embedder Layer standardizes heterogeneous remote sensing foundation models behind a unified object-oriented interface. The paper defines an Embedder base class with methods such as get_embedding, get_embeddings_batch, and describe. Each model adapter encapsulates model-specific details, including feature extraction, scale alignment, band mapping, preprocessing, and checkpoint/runtime handling. This is effectively an adapter pattern in which model heterogeneity is absorbed by wrapper implementations while the user-facing contract remains stable (Ye et al., 27 Feb 2026).

A central architectural distinction is between on-the-fly models and precomputed models. On-the-fly models run forward inference on raw imagery fetched by the Provider Layer and apply model-appropriate preprocessing such as normalization and augmentation. Precomputed models instead retrieve embeddings already stored in the cloud; Alpha Earth is given as an example. rs-embed supports both acquisition modes through the same interface, which is one of its main interoperability claims (Ye et al., 27 Feb 2026).

The library explicitly discusses a broad set of supported or referenced models and model families. These include AlphaEarth, Tessera, Thor, AnySat, AgriFM, TerraFM, TerraMind, WildSat, Prithvi, Galileo, FOMO, NeuralSat, SatMAE, RemoteCLIP, SatVision, and Scale-MAE. It also highlights the diversity of input regimes across the ecosystem, including RGB models, 6-band Sentinel-2 models, 12-band Sentinel-2 models, and MODIS-based models (Ye et al., 27 Feb 2026).

4. Validation, capability matching, and standardized outputs

Before execution, rs-embed applies a two-stage control mechanism consisting of policy enforcement and capability matching. Policy enforcement validates request semantics such as geometry validity, year/range formats, allowed output modes, and resolution/pooling consistency. This is intended to fail early, before a request reaches backend-specific or model-specific code paths (Ye et al., 27 Feb 2026).

Capability matching then consults model capabilities via describe() and checks compatibility with the request. The paper lists backend support (gee or local), output-mode support, and temporal semantics as examples of constraints checked at this stage. If a mismatch occurs, the request is rejected with a human-readable error. This mechanism is central to making the common interface precise rather than permissive (Ye et al., 27 Feb 2026).

Outputs are standardized as Embedding(data, meta). The data field stores the numeric embedding in standardized shape: (D,) for pooled vectors and (D,h,w) for grid outputs. The meta field stores reproducibility-critical metadata, including model identity and type, backend, sensor settings, temporal settings, input size, pooling settings, bands, checkpoint used, token/grid layout, and other model-specific inference parameters. This separation of numeric payload from metadata is one of the library’s main reproducibility features (Ye et al., 27 Feb 2026).

The end-to-end data path is correspondingly explicit. A user specifies SpatialSpec, TemporalSpec, OutputSpec, and a model identifier; policy enforcement validates the request; capability matching verifies that the model can satisfy it; the provider fetches and preprocesses data into (C,H,W) tensors or locates precomputed cloud embeddings; the embedder produces the representation; and the result is normalized into Embedding(data, meta). Optional batch export then writes NPZ or NetCDF outputs together with manifests (Ye et al., 27 Feb 2026).

A recurrent interpretive boundary is that output standardization does not imply semantic equivalence. Pooled vectors and grid tensors can be retrieved uniformly, but the paper explicitly notes that models differ in temporal support, training data, input modalities, and output geometry. A plausible implication is that rs-embed is primarily a benchmarking and systems substrate rather than a method for cross-model representational alignment (Ye et al., 27 Feb 2026).

5. Large-scale execution, export, and supported workflows

For large-scale workloads, rs-embed implements a four-stage optimized pipeline: orchestration, prefetch, inference, and export. The orchestration stage ingests batched spatial queries, one or more models, temporal settings, and output settings; validates them; and splits work into sub-batches using chunk_size for memory stability. For each model, it resolves the effective sensor configuration globally or through per-model overrides (Ye et al., 27 Feb 2026).

The prefetch stage deduplicates raw input requests keyed by (point, sensor), fetches unique requests in parallel via a thread pool using num_workers, and caches fetched data for reuse across persistence and inference. This reduces duplicate downloads and total I/O cost. The inference stage caches embedder instances so that model weights are not repeatedly loaded, prefers get_embeddings_batch when supported, and otherwise falls back to per-sample inference. The export stage writes NPZ or NetCDF outputs and supports asynchronous writing via async_write and writer_workers so that disk I/O can overlap with computation (Ye et al., 27 Feb 2026).

The paper emphasizes batch processing, failure isolation, and recoverability. Batch workflows support multiple ROIs for one model, multiple models over many ROIs, and chunked execution for memory stability. Failures are isolated at both the point level and the model level. With continue_on_error=True, one failed item does not abort the full batch; results are marked as ok, partial, or failed. The system also supports bounded retries with exponential backoff using max_retries and retry_backoff_s for provider initialization, provider fetch, inference, and export (Ye et al., 27 Feb 2026).

Each export unit emits a structured manifest containing failure stage, error details, and summary statistics. This enables partial output auditing and efficient reruns, which is particularly important for long-running geospatial embedding jobs. The library therefore functions არა only as an in-memory retrieval wrapper but also as a production/export system for large embedding pipelines (Ye et al., 27 Feb 2026).

The workflows described in the paper include single embedding retrieval, batch embedding retrieval, multi-model batch export, benchmarking and comparative analysis under identical ROI/time conditions, visualization of grid embeddings from many models, and downstream feature extraction such as pooled embeddings for crop-yield regression. Representative API patterns include get_embedding(...), get_embeddings_batch(...), and export_batch(...), always organized around the same ROI-centric specification scheme (Ye et al., 27 Feb 2026).

6. Empirical demonstrations, significance, and interpretive boundaries

The paper includes two empirical demonstrations. The first is a maize-yield regression case study in Illinois. Labels come from SPAM2020V2, and samples are selected from regions with Maize Area greater than 2500 ha, yielding 991 sampling points. Embeddings are extracted for 2019-06-01 to 2019-08-01, pooled via OutputSpec.pool(), and used as features for a Random Forest regressor. The paper states that AgriFM achieves the highest $R^2$ among the compared models, while also noting that AgriFM still struggles to fit extreme high-yield and low-yield samples and that residual maps show spatial over- and under-estimation patterns (Ye et al., 27 Feb 2026).

The second demonstration is a cross-model visualization over 16 remote sensing foundation models. The setup uses TemporalSpec.range("2022-06-01","2022-09-01"), or year 2022 for year-only models, together with PointBuffer(lon=121.5, lat=31.2, buffer_m=2048) and OutputSpec.grid(). Embeddings are visualized by applying PCA to embedding channels and using the top three principal components as pseudo-RGB. The reported conclusion is qualitative: different models, due to their training datasets and objectives, emphasize different spatial aspects, while many capture recognizable land-cover structure such as rivers (Ye et al., 27 Feb 2026).

These demonstrations support the paper’s claims about usability, reproducibility, fairness of comparison, and scalability. The same ROI/time/query abstraction can be applied across many RSFMs, with explicit metadata and validation, enabling more controlled downstream evaluation. At the same time, the paper does not provide a formal throughput benchmark table or direct quantitative systems benchmark for runtime or memory scaling in the provided text, so the main empirical evidence is centered on workflow viability and comparative utility rather than low-level performance characterization (Ye et al., 27 Feb 2026).

A common misconception would be to treat rs-embed as a model unification method in the representational sense. The paper’s own framing is narrower and more concrete: rs-embed is an embedding middleware layer that unifies geospatial query specification, data provisioning, model invocation, and export for remote sensing foundation models. Its significance lies in transforming RSFM usage from a fragmented, model-specific engineering exercise into a more consistent embedding retrieval problem, while preserving the metadata and validation machinery needed for reproducible, cross-model comparison (Ye et al., 27 Feb 2026).