PolSAR Image Classification using a Hybrid Complex-Valued Network (HybridCVNet)

Abstract: Recently, convolutional neural networks (CNNs) have become popular for image classification due to their effectiveness in computer vision tasks. Now, researchers are exploring the potential of vision transformers (ViTs) in remote sensing and Earth observation. However, traditional Real-Valued networks often overlook important phase information in Complex-Valued (CV) data like polarimetric synthetic aperture radar (PolSAR) data. To address this, new CV deep architectures have emerged. HybridCVNet, a novel hybrid network, blends CV-CNN and CV vision transformer (CV-ViT) techniques. It efficiently combines CV 3D and 2D CNNs as feature extractors, enhancing PolSAR image classification by extracting complementary information and effectively leveraging interdependencies within the data. Experimental results from widely-used PolSAR datasets show HybridCVNet outperforms other methods, achieving an overall accuracy of 97.39% on the Flevoland dataset and showing promise even with just a 1% sampling ratio, with a Kappa value of 0.972 on the San Francisco dataset. Source code is accessible through https://github.com/mqalkhatib/HybridCVNet

• The paper introduces a novel hybrid complex-valued CNN-ViT architecture that effectively preserves both amplitude and phase information for PolSAR classification.
• The proposed HybridCVNet outperforms state-of-the-art models, achieving up to 97.39% overall accuracy on benchmark datasets with just 1% labeled data.
• The model combines local feature extraction and global self-attention to enhance class boundary discrimination and noise suppression in complex PolSAR imagery.

HybridCVNet: Complex-Valued Hybrid CNN-ViT Architecture for PolSAR Image Classification

Introduction

Polarimetric Synthetic Aperture Radar (PolSAR) imaging is pivotal for Earth observation due to its capability to capture polarization properties, thus offering rich information in diverse environmental conditions. The inherent complexity of PolSAR data—especially the presence of phase information in the coherency matrix—necessitates models capable of processing and exploiting complex-valued signals. Previous approaches, predominantly based on real-valued convolutional neural networks (CNNs) and more recently Vision Transformers (ViTs), have shown advancements but consistently under-leverage the full potential of complex-valued representations and global-local feature joint modeling.

HybridCVNet addresses these limitations by integrating complex-valued CNNs (CV-CNNs) for hierarchical feature extraction with complex-valued Vision Transformers (CV-ViT) for global context modeling. The network architecture explicitly processes the six-dimensional complex-valued pixel vectors derived from the upper triangle of the PolSAR coherency matrix, maintaining both amplitude and phase information throughout the pipeline.

Figure 1: Overall architecture of HybridCVNet, illustrating the sequential interplay between CV-CNN local feature extractors and the CV-ViT global context module.

Methodology

HybridCVNet leverages a two-stage feature extractor–classifier paradigm. The initial stage employs cascaded 3D and 2D CV-CNNs to hierarchically extract polarimetric-spatial and refined spatial descriptors. Unlike real-valued models that disregard phase relationships, these CV convolutional layers perform the convolution in the complex domain, thereby preserving polarimetric coherence features essential to PolSAR discrimination.

Post CV-CNN encoding, the feature tensor is partitioned into $3\times3\times6$ complex patches, each flattened for input into the CV-ViT. The transformer layers utilize self-attention in the complex space, capturing long-range dependencies and inter-class boundaries more robustly than CNNs alone.

This structural synergy allows HybridCVNet to combine localized semantic context inherent to CNNs with transformer-based aggregation of non-local spatial and polarimetric correlations, substantially improving classification boundaries in ambiguous or spectrally mixed pixels.

Experimental Evaluation on Benchmark PolSAR Datasets

Classification Performance on Flevoland Dataset

HybridCVNet is rigorously evaluated on the AIRSAR Flevoland and San Francisco datasets, each posing unique challenges due to their class heterogeneity. Utilizing only 1% of labeled data for training, HybridCVNet surpasses a robust suite of baselines, including familiar architectures such as 3D-CNN, WaveletCNN, ViT, Swin Transformer, PolSARFormer, and its own real-valued variant (HybridRVNet).

For Flevoland, HybridCVNet achieves an overall accuracy (OA) of 97.39% (±1.35), with a Kappa of 97.15—outperforming PolSARFormer and Swin Transformer by 1.65% and 3.81% OA, respectively. Improvements are evident particularly in spatially complex classes (e.g., Built-up, Peas, Wheat2), demonstrating the hybrid model's superior discrimination capacity.

Figure 2: Comparative classification maps for the Flevoland dataset, with HybridCVNet (h) exhibiting clear reduction in speckle noise and improved class boundary fidelity.

Detailed ablation demonstrates that the CV hybrid (CNN plus ViT) is consistently superior to stand-alone CV-CNN (96.93% OA) or CV-ViT (96.11% OA), validating the hypothesis that joint local-global modeling is indispensable for extracting the full spectrum of polarimetric features from multi-channel complex data.

Generalization and Data Efficiency Analysis

San Francisco experiments echo these findings, with HybridCVNet attaining 98.21% OA and 93.32% AA, compared to 94.92% (ViT) and 95.44% (PolSARFormer). The model demonstrates significant resilience under sparse supervision (1% labeled data), with Kappa indices as high as 97.20—highlighting the efficiency and expressiveness of complex-valued modeling.

Figure 3: Classification results for the San Francisco region; HybridCVNet (h) delivers the most spatially accurate and semantically consistent classifications.

A systematic analysis with increasing training sample sizes (1–5%) reveals that HybridCVNet remains robust across varying supervision levels, with OA and AA steadily increasing, while transformer-only models lag when annotated data is scarce.

Figure 4: Relationship between training data ratio and accuracy metrics (OA, AA, Kappa) for the Flevoland dataset, confirming the high sample efficiency of HybridCVNet.

Theoretical and Practical Implications

This work demonstrates that complex-valued neural architectures—particularly when hybridizing CV-CNNs and CV-ViT layers—effectively exploit both the magnitude and phase information in PolSAR data. The explicit use of CV operations avoids the representational collapse inherent in real-valued simplifications and confirms, empirically and structurally, that global and local polarimetric context are both essential for maximizing classification performance.

From a practical viewpoint, the open-source release of HybridCVNet facilitates adoption in the remote sensing community, with immediate relevance for applications demanding high-accuracy mapping in resource-constrained annotation scenarios.

The computational overhead introduced by complex-valued arithmetic is nontrivial; training times are longer, but complexity can be mitigated in future work using model compression (e.g., pruning, distillation) without sacrificing accuracy.

Figure 5: High-resolution classification results in challenging regions of the Flevoland data demonstrate HybridCVNet's effective noise suppression and class discrimination.

Conclusion

HybridCVNet sets a new performance standard for PolSAR classification, driven by its complex-valued architectural design and joint modeling of local and global features. Its results across benchmark datasets underscore the value of complex-domain attention mechanisms in fully leveraging polarimetric SAR data. Future research directions include optimization for inference speed and hardware deployment, and extension to other modalities with inherent complex-valued structure.