ANROT-HELANet: Robust Hellinger Feature Aggregation

• The paper introduces a Hellinger distance-based aggregation method that replaces KL divergence, ensuring symmetry and numerical stability in few-shot classification tasks.
• It employs adversarial training with FGSM and additive Gaussian noise to enhance robustness against both worst-case perturbations and natural variations.
• Empirical evaluations across miniImageNet, tieredImageNet, and CIFAR-FS demonstrate improved classification accuracy and lower FID scores compared to previous methods.

ANROT-HELANet, formally designated as the "Adversarially and Naturally Robust Hellinger Aggregation Network," embodies a methodological advance in few-shot classification by integrating attention mechanisms with Hellinger distance-based probabilistic feature aggregation, and by jointly optimizing for adversarial and natural robustness. Its architecture and design address persistent instabilities found with prior approaches—especially those relying on Kullback-Leibler (KL) divergence—yielding significant empirical gains in both classification accuracy and generative fidelity under challenging conditions (Lee et al., 14 Sep 2025).

1. Motivation and Theoretical Foundations

Few-shot learning (FSL) tasks demand generalization from limited samples, a setting in which classical deep networks, and even Bayesian meta-learning models, are susceptible to adversarial attacks (targeted perturbations undermining classifier confidence) and to the effects of naturally occurring noise (sensor variation, illumination changes). Traditional probabilistic aggregation, predominantly using KL divergence, is inherently asymmetric and, under sample scarcity, can be numerically unstable or easily perturbed.

ANROT-HELANet reformulates feature aggregation and prototype learning in the embedding space by adopting the symmetric, bounded Hellinger distance: $D_H^2(p, q) = 1 - \int \sqrt{p(z) \cdot q(z)}\, dz$ where $p(z)$ and %%%%1%%%% are the densities of query and support-set induced latent variables, respectively.

This choice confers three principal advantages:

• Symmetry: The measure treats source and target distributions equivalently, unlike KL.
• Numerical Stability: Boundedness within [0,1] constrains gradients across minibatches.
• Geometric Interpretability: The Hellinger distance equates to the Euclidean distance in the space of square-root densities, naturally supporting clustering and feature aggregation among high-dimensional prototypes.

2. Hellinger Distance-Based Class Aggregation

Within ANROT-HELANet’s variational framework, support and query sets are represented as probabilistic embeddings. Rather than inferring task-specific class prototypes via direct averaging or KL aggregation, prototypes are constructed by minimizing the expected Hellinger distance between the pooled support-set embedding and the query distribution.

For latent variable models (e.g., VAEs), this strategy is incorporated into the ELBO, replacing the KL term with the Hellinger functional, which produces class clusters more resistant to perturbation and less prone to degenerate solutions when data is sparse.

This approach is reflected during optimization by loss terms that penalize the Hellinger separation between class prototypes and sample queries. The network thus learns probabilistic codes whose mutual proximity reflects true class membership, but whose separation is robust to both adversarial and stochastic noise.

3. Adversarial and Natural Robustness Mechanisms

ANROT-HELANet enhances robustness through two explicit mechanisms:

(a) Adversarial Training via FGSM

Sample feature maps ($\psi''$) undergo adversarial perturbation during training: $$\tilde{\psi}'' = \psi'' + \epsilon \cdot \mathrm{sign}\left( \nabla_{\psi''(x)} \ell(\psi'', y) \right)$$ where $\epsilon$ is the perturbation magnitude and $\ell$ is the loss. This process compels the network to learn representations stable under worst-case perturbations up to $\epsilon=0.30$.

(b) Natural Robustness via Additive Gaussian Noise

Simultaneously, natural input variation is simulated by injecting zero-mean Gaussian noise ($\mathcal{N}(0,\sigma)$), with $\sigma$ sampled up to 0.30. This mechanism ensures that the discriminative features—emphasized via the attention module—remain salient under practical data corruption.

Together, these procedures improve empirical stability and support transfer to real-world contexts in medical imaging or sensor-driven applications.

4. Hellinger Similarity Contrastive Loss

A central innovation is the Hellinger Similarity Loss ($\mathcal{L}_{\text{hesim}}$), which generalizes contrastive objectives beyond cosine similarity to scenarios where feature vectors are probability distributions.

For query embeddings $v_{Q,i}$ and class prototypes $c_{Q,j}$, the probability of class membership is measured by a softmax over negative Hellinger distances: $$p(y=j | v_{Q,i}) = \frac{\exp(-\{v_{Q,i}, c_{Q,j}\})}{\sum_{k=1}^{N} \exp(-\{v_{Q,i}, c_{Q,k}\})}$$ with the loss

$$\mathcal{L}_{\text{hesim}} = -\sum_{i=1}^{N_t} y_i \log p(y=j | v_{Q,i})$$

where $\{v_{Q,i}, c_{Q,j}\}$ encapsulates the Hellinger similarity. This formulation enables variational inference over distributional prototypes, rather than point estimates, promoting robust and discriminative embeddings under few-shot constraints.

5. Evaluation and Empirical Performance

Experiments span miniImageNet, tieredImageNet, CIFAR-FS, and FC-100 datasets, in both 5-way 1-shot and 5-shot regimes.

Key metrics:

• Classification Accuracy: ANROT-HELANet attains improvements of approximately 1.20% (1-shot) and 1.40% (5-shot) over previous robust baselines (e.g., HELA-VFA).
• Image Generation Quality: When equipped with a variational autoencoder backbone, it achieves a Fréchet Inception Distance (FID) of 2.75, outperforming VAE (FID=3.43) and WAE (FID=3.38) configurations.
• Robustness Under Perturbation: With adversarial perturbations ($\epsilon\leq0.30$) and natural noise ($\sigma\leq0.30$), controlled degradation in accuracy is observed, with significant preservation for models trained with both robustness regimes.

6. Comparison with Prior Art

Relative to KL-based approaches and established prototype-based FSL methods:

• Distributional Aggregation: The use of Hellinger distance and attention modules improves numerical stability and prototype clustering, especially under data scarcity.
• Robustness: The joint adversarial and noise training surpasses vanilla variational models in maintaining accuracy under perturbation.
• Generative Performance: Using the Hellinger distance within ELBO leads to lower FID scores in image reconstruction tasks compared to both VAE and WAE paradigms.

A plausible implication is that the symmetric and bounded nature of the Hellinger distance can be further generalized to other robust meta-learning contexts where symmetric divergence is sought.

7. Implementation and Accessibility

The ANROT-HELANet source code is publicly available at https://github.com/GreedYLearner1146/ANROT-HELANet/tree/main, enabling reproducibility and facilitating adaptation to alternative domains such as medical imaging or satellite-based remote sensing. This openness supports continued progress on few-shot robustness and probabilistic meta-learning research.

In summary, ANROT-HELANet represents an integration of attention-based feature extraction and robust probabilistic aggregation using Hellinger distance, yielding empirical improvements in both classification and generative quality for few-shot learning tasks. Its design addresses critical limitations in robustness exposed by KL-divergence-centered methods and establishes new benchmarks for adversarial and natural resistance in deep meta-learning models (Lee et al., 14 Sep 2025).