Beyond Visual Fidelity: Benchmarking Super-Resolution Models for Large-Scale Remote Sensing Imagery via Downstream Task Integration

Abstract: Super-resolution (SR) techniques have made major advances in reconstructing high-resolution images from low-resolution inputs. The increased resolution provides visual enhancement and utility for monitoring tasks. In particular, SR has been increasingly developed for satellite-based Earth observation, with applications in urban planning, agriculture, ecology, and disaster response. However, existing SR studies and benchmarks typically use fidelity metrics such as PSNR or SSIM, whereas the true utility of super-resolved images lies in supporting downstream tasks such as land cover classification, biomass estimation, and change detection. To bridge this gap, we introduce GeoSR-Bench, a downstream task-integrated SR benchmark dataset to evaluate SR models beyond fidelity metrics. GeoSR-Bench comprises spatially co-located, temporally aligned, and quality-controlled image pairs from about 36,000 locations across diverse land covers, spanning resolutions from 500m to 0.6m. To the best of our knowledge, GeoSR-Bench is the first SR benchmark that directly connects improved image resolution from SR models with downstream Earth monitoring tasks, including land cover segmentation, infrastructure mapping, and biophysical variable estimation. Using GeoSR-Bench, we benchmark GAN, transformer, neural operator, and diffusion-based SR models on perceptual quality and downstream task performance. We conduct experiments with 270 settings, covering 2 cross-platform SR tasks, 9 SR models, 3 downstream task models, and 5 downstream tasks for each SR task. The results show that improvements in traditional SR metrics often do not correlate with gains in task performance, and the correlations can be negative, indicating that these metrics provide limited guidance for selecting superior models for downstream tasks. This reveals the need to integrate downstream tasks into SR model development and evaluation.

• The paper introduces GeoSR-Bench, a large-scale benchmark that integrates super-resolution outputs with downstream geospatial tasks.
• It evaluates diverse SR models over 270 experimental settings, revealing weak correlations between visual fidelity metrics and practical task performance.
• The study emphasizes the need for task-coupled SR model design to better support applications such as land cover segmentation and infrastructure mapping.

Benchmarking Remote Sensing Super-Resolution through Downstream Task Integration

Introduction

This paper presents GeoSR-Bench, a large-scale benchmark for evaluating single image super-resolution (SR) models in the context of remote sensing, with a central focus on the impact of SR on practical downstream tasks rather than traditional visual fidelity metrics alone. The motivation is to challenge the prevailing focus of remote sensing SR research on PSNR/SSIM and instead explicitly assess how SR outputs support high-level geospatial applications such as land cover segmentation, infrastructure mapping, and biophysical variable estimation.

GeoSR-Bench is the first framework to directly connect remote sensing SR with a suite of downstream Earth observation tasks, providing high-quality, globally-distributed, temporally and spatially aligned imagery. Strong experimental evidence is collected by benchmarking 9 SR methods across GAN, transformer, neural operator, and diffusion model families over 270 settings, quantifying the mismatch between traditional visual fidelity metrics and actual task-driven utility. The dataset, trained models, and evaluation protocols are made available for open research.

GeoSR-Bench: Dataset Construction and Scope

GeoSR-Bench covers two major cross-platform SR tasks: MODIS (500m) to Landsat-8 (30m) and Sentinel-2 (10m) to NAIP (0.6m). This design ensures that the resolution range represents the majority of operational remote sensing platforms used in actual monitoring applications.

Figure 1: GeoSR-Bench targets two cross-platform SR tasks addressing diverse spatial scales used in Earth observation.

Coincident, spatially and temporally matched image pairs are sampled from more than 36,000 locations. Stratified sampling based on land cover ensures both urban and non-urban regions are well represented, addressing the historical imbalance of datasets toward easy-to-reconstruct homogeneous regions.

Figure 2: GeoSR-Bench dataset construction couples spatially-temporally aligned queries and rigorous QA.

Figure 3: Geographic distribution of MODIS-to-Landsat-8 and Sentinel-2-to-NAIP image pairs, confirming coverage across diverse ecoregions worldwide.

For evaluation, five pixel-level downstream tasks are assigned to each resolution setting, focused on applications where increased spatial granularity is expected to yield substantial advantages. Tasks span semantic segmentation (e.g., river, building, road) to biophysical regression (e.g., GPP, canopy height).

The construction pipeline strictly ensures radiometric consistency, masking cloud/snow, harmonizing spectral properties, and manually validating triplet (low-res, high-res, label) alignments.

Experimental Protocol and Model Suite

The benchmark adopts a two-stage experimental protocol:

1. SR model training: Each SR method is first pre-trained for each cross-platform task on domain-general data.
2. Downstream integration: Each SR model is then fine-tuned using the paired data specific to each downstream application, with downstream task models (U-Net, SegFormer, Swin Transformer) further fine-tuned on the SR model’s output.

This results in 270 model paths per experiment. SR methods are categorized by architectural family (transformer, GAN, neural operator, diffusion). The downstream task suite is dominated by dense pixel prediction architectures.

Visual Fidelity Evaluation and Its Limits

SR performance is first cross-compared using canonical metrics (PSNR/SSIM) across all downstream datasets. On the MODIS-to-Landsat-8 task, transformer and neural operator SR models achieve highest PSNR/SSIM scores, while GANs and diffusion approaches underperform under these metrics—notably due to their design goals focusing on perceptual realism rather than pixelwise reconstruction.

Yet, examples reveal PSNR and SSIM do not consistently reflect perceived quality or practical utility. Models producing sharper outputs that align better with human judgment can score lower than those generating over-smoothed images favored by pixelwise metrics.

Figure 4: Visual fidelity metrics (PSNR/SSIM) often diverge from visual perception in SR output on USBuildings data.

Dataset-wise, reconstruction is easier for spectrally homogeneous (e.g., forest) areas, harder for spatially complex (urban, cropland) regions, directly exposing the impact of land cover variation.

Downstream Task Evaluation

Downstream task performance is measured relative to both the original low-resolution and high-resolution imagery, establishing expected lower and upper bounds, respectively.

• For MODIS-to-Landsat-8, only a minority of SR models achieve consistent gains on downstream F1 or regression (MAE) compared to the lower-resolution images. On pixel-wise regression tasks (e.g., GPP), improvements are somewhat more common. Transformer and neural-operator models are most robust, while GAN and diffusion models frequently fail to exceed baseline.
• For Sentinel-2-to-NAIP, some tasks (building, road, multiclass land cover segmentation) see moderate improvements from SR, especially with GAN-based models recovering critical fine structure. However, SR models still leave a large performance gap versus true high-res imagery, especially for localized, binary tasks.

  Figure 5: Relative performance of SR models for MODIS-to-Landsat-8 downstream tasks (SegFormer), showing persistent performance gaps between SR output and true high-res imagery.

  

  Figure 6: Relative performance for Sentinel-2-to-NAIP downstream tasks (SegFormer), highlighting variance across model families and tasks.

The main pattern is that, even when SR improves on the low-res baseline, it rarely gets close to the utility of original high-resolution data, and some models—especially GANs/diffusions—sometimes degrade performance due to hallucinated, misaligned details.

Correlation Analysis: Visual Fidelity vs. Downstream Utility

A principal claim supported by the experiments is that PSNR and SSIM show weak, non-monotonic, or even negative correlation with downstream task performance among competitive models. This divergence is especially pronounced for Sentinel-2-to-NAIP and for subsets of models near the Pareto frontier.

Top-k analysis demonstrates that correlation values are only reliably positive when including large performance gaps (e.g., poor outlier models). Within competitive groups (small top-k), correlations are highly unstable and can be negative, making PSNR/SSIM a poor proxy for task-driven model selection.

Figure 7: Pearson correlation between visual fidelity metrics and downstream performance—in many competitive settings, no strong positive relationship emerges.

Figure 8: Spearman’s rank correlation between fidelity and downstream metrics, reinforcing the challenge of using PSNR/SSIM for reliable SR model ranking.

A further illustration is given on a sub-pixel delineation task (TreeFinder) where, despite reasonable PSNR, SR output makes little impact on practical detection performance—reflecting the ill-posedness of inferring true sub-grid features from coarse-resolution signals.

Figure 9: Downstream segmentation performance on TreeFinder; SR offers minimal gains over Sentinel-2, with both far below NAIP.

Implications and Future Directions

The results confirm that canonical SR benchmarking protocols using only visual fidelity are insufficient for guiding model development when practical geospatial monitoring tasks are the true target. The persistent qualitative and quantitative gap between high visual fidelity and high downstream utility, especially in challenging downstream cases, signals theoretical and practical limitations of task-agnostic SR design.

Key implications:

• Optimizing only for PSNR/SSIM can actively harm downstream utility, especially for applications where spatial fidelity of critical objects (e.g., roads, buildings) is more relevant than signal averaging.
• Integrating downstream tasks into SR model training and selection is non-optional for operational remote sensing deployments.
• Current models are far from saturating the utility achievable from actual high-resolution observations, implying a need for new loss formulations, architectures, or training regimes explicitly coupled to application-level supervision.

The GeoSR-Bench provides an open, extensible foundation for research in this direction.

Conclusion

GeoSR-Bench represents a decisive step toward application-driven evaluation of remote sensing SR, aligning model development and assessment with the requirements of actual downstream geospatial tasks. The evidence rigorously demonstrates that visual fidelity metrics, while an easy fallback, are frequently misleading indicators for end-use utility. Progress in remote sensing SR will require more direct, task-coupled approaches, both in future datasets and modeling paradigms.

GeoSR-Bench, with its large-scale, diverse, and rigorously curated SR/downstream datasets, will be instrumental in driving such developments and clarifying the genuine capabilities and limitations of SR in real-world Earth observation applications.