Remote Sensing Scene Classification

• Remote sensing scene classification is the automated process that assigns semantic labels, such as airport or crop field, to overhead imagery, underpinning applications like urban mapping and disaster management.
• Advances include deep CNNs, transformer models, multi-scale feature fusion, and pairwise/multi-modal techniques to tackle high intra-class variance and subtle inter-class differences.
• Practical applications span disaster response, land-use planning, and efficient model design while ongoing research addresses label noise, class imbalance, and multi-sensor data integration.

Remote sensing scene classification is the automated assignment of semantic scene labels (e.g., airport, residential area, crop field) to overhead imagery acquired from airborne or satellite platforms. It is a foundational task in Earth observation, supporting applications such as urban mapping, ecological monitoring, disaster response, and land-use planning. The singular challenges of this domain—large intra-class variance, high inter-class similarity, scale and viewpoint diversity, class imbalance, and the prevalence of label noise—have driven the development of a suite of specialized models, datasets, and learning techniques. The field has rapidly progressed from shallow feature engineering to advanced deep learning paradigms, including multi-granular feature fusion, transformer-based models, self-supervised and few-shot learning, and multi-modal information integration.

1. Historical Evolution and Benchmarks

Early work in remote sensing scene classification utilized handcrafted features (texture, color histograms, structural descriptors), paired with SVMs or random forests. The introduction of deep convolutional neural networks (CNNs) and large annotated datasets such as UC-Merced, AID, and NWPU-RESISC45 catalyzed significant improvements in accuracy (Liu et al., 2016, Zhao et al., 2020).

More recently, datasets have expanded in both scale and semantic granularity, exemplified by the MEET benchmark, which comprises over one million fixed-resolution, globally distributed, context-rich samples annotated into 80 fine-grained categories. The MEET corpus introduces the "scene-in-scene" layout, in which each sample includes a central region, a larger surrounding context, and a global field-of-view, supporting spatial-context–aware modeling for practical, zoom-free applications (Li et al., 14 Mar 2025).

2. Modeling Paradigms

2.1 Deep CNN Backbones and Multi-Scale/Multi-Level Fusion

CNN backbones such as ResNet, VGG, GoogleNet, and DenseNet have become standard for remote sensing, with transfer learning from ImageNet commonly used for initialization (Pham et al., 2022). Advanced paradigms augment these architectures via:

• Multi-layer and multi-scale feature fusion: Adaptive Deep Pyramid Matching (ADPM) combines convolutional-layer histograms across scales, assigning learned weights to different layers and input resolutions. ADPM demonstrates that exploiting both shallow (local/texture) and deep (semantic/contextual) features, as well as multiple spatial scales, is critical for robust classification (Liu et al., 2016).
• Multi-Granularity Feature Ensembles: Networks such as MGML-FENet extract features at multiple depths and spatial granularities, fusing them through ensemble voting of independent classification heads to reduce intra-class variance and mitigate confusing context (Zhao et al., 2020).

2.2 Transformer Architectures and Context Integration

Transformer-based models, both pure and hybrid, have been increasingly adopted to capture long-range spatial dependencies and global context. The Context-Aware Transformer (CAT) in the MEET benchmark fuses center, surrounding, and global fields-of-view via parameter-efficient AdaptFormer modules and attention-based cross-branch fusion, enabling precise discrimination in "scene-in-scene" scenarios (Li et al., 14 Mar 2025).

Self-attention transformers trained with self-supervised paradigms (e.g., Masked Image Modeling, MIM) rival or surpass domain-specific transformer designs while providing strong robustness to spatial and class imbalance (Wang et al., 2023).

2.3 Pairwise, Metric, and Siamese Approaches

Discriminative metric learning and pairwise comparison architectures address the subtleties of inter-class confusion:

• Siamese embeddings: Simultaneously optimize per-image classification and contrastive pairwise losses, clustering intra-class samples and maximizing margin to inter-class samples (Wang et al., 2019).
• Pairwise Comparison Networks (PCNet): Deliberately sample intra- and inter-class neighbor pairs in feature space, extract both self- and mutual-attention representations using channelwise attention, and jointly optimize with a margin ranking constraint (Yue et al., 2022). These approaches yield state-of-the-art performance, especially under low-label regimes and for classes with subtle visual differences.

2.4 Automated and Efficient Network Design

Neural Architecture Search (NAS)-based techniques optimize CNN macro- and micro-architecture via gradient-based exploration in a continuous relaxation of discrete operator choices (convolution type, pooling, skip connections). This enables dataset-specific and resource-constrained model adaptation (Chen et al., 2020).

Lightweight convolutional mixing architectures, as in SceneMixer, alternate multiscale depthwise spatial mixing with channelwise pointwise mixing, delivering strong accuracy–efficiency trade-offs with minimal parameter count and computational load (Alkhatib et al., 7 Dec 2025).

3. Learning Paradigms: Transfer, Self-Supervision, and Label-Efficiency

3.1 Transfer Learning and Attention Pooling

Transfer learning from natural images remains essential, with fine-tuning after initialization universally outperforming training from scratch. Attention-based pooling (multi-head, spatial, or channelwise) replaces crude global pooling, focusing the model on discriminative localities and improving robustness to spatial heterogeneity (Pham et al., 2022).

3.2 Self-Supervised and Masked Modeling

Self-supervised learning mechanisms are crucial when labeled data is scarce or cross-domain generalization is required. Pretext tasks including contrastive instance discrimination, inpainting, and relative-position (jigsaw) prediction produce robust feature encoders using only unlabeled remote-sensing imagery. Such encoders consistently outperform both from-scratch and ImageNet-pretrained baselines under limited supervision (Tao et al., 2020).

Masked Image Modeling with Vision Transformers further improves representation quality. MAE and CAE variants trained on random patch masking yield gains up to 5 pp over supervised pre-training, and up to 18 pp over CNNs on various benchmarks, especially in coarse or multispectral imagery (Wang et al., 2023).

3.3 Few-Shot, Zero-Shot, and Meta-Learning

Few-shot scene classification remains challenging due to class imbalance and domain shift. Prompt learning techniques enable CLIP and other vision-LLMs to adapt efficiently with minimal samples by learning or conditioning input prompts, with variants such as PromptSRC incorporating self-regulating constraints to preserve multimodal alignment and prevent catastrophic forgetting (Dimitrovski et al., 28 Oct 2025).

Meta-metric learning frameworks (RS-MetaNet) optimize embeddings over episodic tasks and employ custom loss functions ("Balance Loss") to promote generalization by maximizing the separation between classes while maintaining within-class compactness. Ablation studies demonstrate that episodic meta-training and learnable metric modules significantly improve low-shot classification accuracy (Li et al., 2020).

Zero-shot and generalized zero-shot classification leverage automatically extracted attribute vocabularies (e.g., RSMM-Attr) and transformer-based cross-modal semantic-visual alignment to transfer knowledge from seen to unseen categories, achieving leading performance on large-scale remote-sensing benchmarks (Xu et al., 3 Feb 2024).

4. Multimodal, Multi-View, and Robust Training

With the rise of diverse sensor modalities and perspectives (multispectral, SAR, dual-view aerial/ground), the fusion of complementary information is a key research focus:

• Dual- and Multi-View Fusion: Evidential deep learning quantifies per-view credibility via Dirichlet-based uncertainty, and fusion rules weight decisions by uncertainty estimates, yielding state-of-the-art results on dual-view datasets (Zhao et al., 2023).
• Multimodal Fusion: Dual cross-attention networks integrate CLIP-generated text captions with visual transformer features, providing semantic context to disambiguate high intra-class and inter-class similarity. Multimodal fusion consistently outperforms image-only models, both in overall accuracy and zero-shot generalization (Cai et al., 3 Dec 2024).
• Robustness to Label Noise: Multi-view voting and entropy ranking iteratively refine noisy annotations, enabling training under up to 50% mislabeling with minimal performance degradation (Wang et al., 2023).

5. Practical Considerations, Limitations, and Future Directions

Recent research emphasizes balancing accuracy, efficiency, interpretability, and operational feasibility. Lightweight models (e.g., SceneMixer), compressed-domain inference pipelines, and NAS-designed architectures broaden the deployment spectrum, from real-time to edge devices (Alkhatib et al., 7 Dec 2025, Byju et al., 2020, Chen et al., 2020).

Interpretable visualizations—feature saliency maps, class-wise t-SNE—have become standard tools for diagnosing model confusions, optimizing attention mechanisms, and guiding module design (e.g., local patch attention vs. global context) (Zhao et al., 2020, Li et al., 14 Mar 2025, Cai et al., 3 Dec 2024).

Open challenges persist:

• Building larger, cleaner, explicitly multi-level annotated datasets to reduce label ambiguity and support hierarchical models (Sen et al., 2021).
• Advancing zero-shot and few-shot methods to handle open-set recognition and class emergence, especially by better grounding attribute spaces and extending to truly open attributes (Xu et al., 3 Feb 2024).
• Incorporating temporal and multimodal remote-sensing data (SAR, LiDAR, multi-temporal series) in unified, self-supervised pretraining schemas.
• Improving resource-adaptive model design and developing robust model selection guidance for variable sensor, geographic, and application constraints.

6. Summary Table of Representative Methods and Datasets

Below, representative state-of-the-art methods and key datasets are tabulated for reference.

Method	Architecture/Paradigm	Benchmark Dataset(s)	Peak OA (%)
ADPM (Liu et al., 2016)	Multi-scale/multi-layer fusion (CNN)	UC-Merced, 19-Class Satellite	94.9
MGML-FENet (Zhao et al., 2020)	Multi-granularity ensemble (CNN)	AID, NWPU-RESISC45, UC-Merced	98.6 (AID/50%)
CAT+MEET (Li et al., 14 Mar 2025)	Context-aware Transformer	MEET (1M+, 80 classes)	97.7 (OA)
PCNet (Yue et al., 2022)	Pairwise attention-based ResNet	AID, NWPU-RESISC45	96.8 (AID/50%)
MIM-ViT (Wang et al., 2023)	Masked Image Modeling (ViT)	UC-Merced, AID, NWPU-RESISC45	100 (UCM), 98.1
PromptSRC (Dimitrovski et al., 28 Oct 2025)	Prompt learning (CLIP, few-shot)	EuroSAT, AID, SIRI-WHU, etc.	87.0 (avg 16-shot)
RS-MetaNet (Li et al., 2020)	Meta-metric episodic learning	UCMerced, NWPU-RESISC45, AID	76.1 (UCM/5-shot)

7. Implications and Outlook

Remote sensing scene classification has evolved into a highly technical, interdisciplinary field at the confluence of computer vision, geospatial science, and machine learning. Essential advances have been achieved through novel architectures capable of extracting multi-scale, multi-level, and multi-source features, robust and label-efficient learning paradigms, and integrative data fusion methodologies. With the continued expansion of massive annotated and unlabeled datasets, and the adoption of multimodal foundation models aligned to remote sensing semantics, the state of the art will continue to advance in both accuracy and practical applicability (Li et al., 14 Mar 2025, Cai et al., 3 Dec 2024, Wang et al., 2023, Dimitrovski et al., 28 Oct 2025).