Influential Prototypical Networks (IPNet)

• The paper introduces a novel influence weighting mechanism based on Maximum Mean Discrepancy (MMD) to assess the statistical impact of each support sample.
• The approach integrates a sample-weighting stage into the standard episodic meta-learning pipeline, enhancing robustness by suppressing outlier effects.
• Empirical results demonstrate IPNet improves accuracy and AUC by up to 5% compared to traditional prototypical networks on challenging few-shot tasks.

Influential Prototypical Networks (IPNet) are a class of metric-based meta-learning algorithms for few-shot classification that extend the prototypical network (PN) paradigm by explicitly weighting each support sample according to its statistical influence within its class. By leveraging influence quantification based on Maximum Mean Discrepancy (MMD)—either in an embedding space using a RBF kernel or via Euclidean means—IPNet suppresses outlier and noisy samples and improves generalization in high-variance, small-sample regimes. The approach is designed as a drop-in replacement for classical PN, introducing only a sample-weighting stage within the standard episodic meta-learning pipeline (Chowdhury et al., 2022, Chowdhury et al., 2021).

1. Foundations: Prototypical Networks and Their Limitations

Standard Prototypical Networks construct a prototype for each class by embedding support samples using a parameterized encoder $f_\theta$, typically a convolutional network. For an episode with $N$ classes and $K$ support samples per class, each class prototype is the unweighted arithmetic mean: $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$ This prototype definition assumes all support elements are equally reliable, irrespective of potential outliers, mislabeled or atypical instances. Classification is accomplished by computing (softmaxed) negative Euclidean distances between each query vector and class prototypes. Conventional PNs, therefore, lack an explicit robustness mechanism in the prototype formation step, leading to prototype drift and diminished few-shot generalization, especially under class imbalance or corrupted support sets (Chowdhury et al., 2022, Chowdhury et al., 2021).

2. MMD-derived Influence Weighting Mechanism

IPNet introduces a mechanism for data-driven estimation of the relative importance of each support sample via Maximum Mean Discrepancy (MMD). For a support set $S_c$ of class $c$, consider the empirical distributions with and without a particular sample $x_i$: $$P_c = \frac{1}{m}\sum_{j=1}^m \delta_{f_\theta(x_j)}, \quad P_c^{-i} = \frac{1}{m-1}\sum_{j \neq i} \delta_{f_\theta(x_j)}$$ MMD in an RKHS $\mathcal H$ quantifies the change in mean feature embedding arising from leaving out $x_i$: $N$0 Here, $N$1 is a positive-definite kernel, most commonly the Gaussian (RBF) kernel $N$2. The unbiased estimator for MMD is evaluated using pairwise kernel evaluations between support elements. The raw influence is then: $N$3 Samples with negligible distributional influence (typical support points) yield $N$4, while outliers garner low or negative values (Chowdhury et al., 2022).

Practical implementations may utilize Euclidean embedding distances as a kernel-free proxy, especially when computational efficiency is paramount (Chowdhury et al., 2021).

3. Weighted Prototype Construction and Query Classification

The influence weights are post-processed to ensure positivity and normalization within each class: $N$5 or equivalently via a shift-and-scale normalization ensuring all $N$6 and $N$7. Each class prototype is then the convex weighted sum: $N$8 Classification proceeds as in standard PN, with query embeddings compared to these weighted prototypes using the Euclidean metric: $N$9 Probabilities are assigned by softmax over (negated) distances: $K$0 (Chowdhury et al., 2022, Chowdhury et al., 2021).

4. Training Pipeline and Algorithmic Flow

The IPNet training and evaluation pipeline preserves the episode-based strategy of classic meta-learning, distinguished only by the added influence-weight computation step. A typical episode proceeds as follows:

1. Sample an $K$1-way, $K$2-shot support set $K$3 and query set $K$4.
2. Embed all support elements using $K$5.
3. For each class $K$6, compute pairwise kernel (or Euclidean) distances, derive influence weights $K$7, and form the weighted prototype $K$8.
4. Embed query samples, compute distances $K$9, apply softmax to obtain predicted probabilities, and accumulate episode-wise cross-entropy loss.
5. Backpropagate loss to update $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$0; since weights $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$1 are differentiable functions of support embeddings, gradient flow is preserved through all stages.

Pseudocode for one episode (from (Chowdhury et al., 2022)):

$$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$7 (Chowdhury et al., 2022)

5. Empirical Performance and Comparative Results

Empirical results on both medical image domains and diverse few-shot settings demonstrate the practical benefit of the influence-based weighting scheme. Evaluations against standard PN, MDDNet, and RRPNet across several dermatological datasets produce consistent gains:

Dataset	Task	PNet	RRPNet	IPNet (ours)
ISIC-2018	2-way 3-shot	65.52 (0.71)	76.32 (0.79)	79.00 (0.83)
Derm7pt	2-way 5-shot	67.21 (0.69)	77.49 (0.80)	80.21 (0.86)
SD-198	5-way 5-shot	71.69 (0.76)	77.90 (0.82)	81.44 (0.84)

Averaged across settings, IPNet achieves +3.5% accuracy and +5.1% AUC relative to the best non-influence baseline (Chowdhury et al., 2021). Cross-domain adaptation (training on one dataset, testing on another) yields typical gains of +2–5% accuracy and +3–6% AUC for IPNet.

Ablation studies replacing influence-based weights with uniform (vanilla PN) weights degrade performance by approximately 3.2% (Chowdhury et al., 2022). Kernel bandwidth selection affects performance marginally (≤0.8%) across reasonable ranges. Removing per-class normalization of weights destabilizes training and further reduces accuracy.

6. Implementation Aspects and Limitations

Influence computation for each support sample in a class of size $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$2 requires $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$3 kernel computations per class per episode. This cost remains manageable in classic few-shot regimes (e.g., $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$4 or $$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$5), but can become significant for higher-shot or large-class meta-learning contexts. The normalization step, enforcing weight positivity and sum-to-one constraints, is critical; omitting this step can destabilize optimization (Chowdhury et al., 2022, Chowdhury et al., 2021).

Approximate MMD (using either stochastic sampling or learned kernels) and kernel-free alternatives (Euclidean means) are suggested for further efficiency, with kernel bandwidth and normalization hyperparameters easily incorporated into differentiable pipelines.

7. Extensions, Robustness, and Future Research

IPNet’s weighting formalism is architecture-agnostic; no changes are imposed on the embedding network itself, and the method is compatible with Conv-6 and other standard backbone feature extractors. A plausible implication is that influence weighting could synergize with more advanced selection or attention mechanisms, or be integrated as a component in hierarchical, multi-stage meta-learners.

Potential extensions include:

• Parametric kernel learning per class for adaptive MMD estimation.
• Scalable stochastic or sampling-based approximations for influence weighting in large-$$p_c^{\rm PN} = \frac{1}{K}\sum_{i=1}^K f_\theta(x_i)$$6 settings.
• Combination with hard sample selection or attention-based weighting for further robustness (Chowdhury et al., 2021).

The influence quantification framework embodied in IPNet provides a statistical rationale for prototype reliability, improving resilience to outliers and domain drift, and consistently advancing both intra- and cross-domain few-shot performance within the PN paradigm (Chowdhury et al., 2022, Chowdhury et al., 2021).