kNN Proximity-Based Defense (KPB) Overview

• KPB is a thresholded k-nearest-neighbor defense designed to detect Beta Poisoning by identifying unusually dense clusters of poisoning samples.
• It computes the average distance to a sample's nearest neighbors and flags points with distances below a threshold τ to isolate anomalies.
• Empirical results on MNIST and CIFAR-10 achieve perfect detection under optimized τ, though performance slightly drops on more complex datasets like CIFAR-100.

kNN Proximity-Based Defense (KPB) is a thresholded $k$-nearest-neighbor pre-processing defense introduced as one of four defenses against Beta Poisoning, a heuristic data poisoning attack designed to avoid expensive bilevel optimization and to make training data linearly nonseparable. KPB is tailored to a specific geometric artifact of Beta Poisoning: poisoning samples tend to lie very close to one another, forming a dense local cluster, and they are centered near the mean of the target class while being labelled as the non-target class. For each sample in a suspicious training set, KPB computes the average distance to its nearest neighbors and flags the sample as poisoned when that average falls below a threshold $\tau$. In that sense, KPB is a high-density detector specialized to structured poisoning rather than a generic outlier detector (Gulciftci et al., 2 Aug 2025).

1. Attack geometry and motivation

Beta Poisoning operates on a target class $y_t$ by constructing poisoning samples $x_p$ that maximize the class-conditional likelihood $P(x_p \mid y_t)$ under a kernel density estimator, typically Gaussian KDE, while remaining within box constraints: $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$ The poisoning sample is parameterized as a linear combination of target-class prototypes,

$$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$

where $\mathcal{S} = \{x_1,\dots,x_k\}$ are prototypes from class $y_t$ and $\boldsymbol{\beta}$ is optimized by gradient ascent on the KDE likelihood. The stated objective is to make the training data linearly nonseparable, which is especially damaging to linear models (Gulciftci et al., 2 Aug 2025).

The defense rationale follows directly from two empirical properties of Beta-poisoned points. First, poisoning samples have close proximity: many lie very close to each other and form a dense local cluster. Second, they are centered near the target-class mean of $\tau$0, while being labelled as the non-target class $\tau$1. Generic outlier detection is described as not ideal because it looks for isolated points, whereas Beta Poisoning produces structured, high-density anomalies. KPB therefore targets the first property directly: anomalously short distances to nearest neighbors.

This geometry yields the central intuition of KPB. Clean data are spread across the support of the class distributions, so the average distance to their nearest neighbors is moderate. Beta-poisoned points, by contrast, are clustered in a tight region and are close to one another in feature space, so each poisoning point has many very close neighbors, largely other poisoning points. KPB exploits this contrast by treating unusually high local density as the defining anomaly.

2. Formal definition and decision rule

KPB is defined on a suspicious dataset

$\tau$2

where some $\tau$3 are clean and others are poisoning samples generated by Beta Poisoning. Its output is a flagged subset

$\tau$4

interpreted as the set of samples detected as poisoned. The defense uses a distance function $\tau$5; the paper does not lock in a specific metric, but in practice this is typically Euclidean,

$\tau$6

Its two parameters are the distance threshold $\tau$7 and the neighbor fraction $\tau$8, which defines the number of neighbors as

$\tau$9

In implementation, this quantity is rounded to an integer (Gulciftci et al., 2 Aug 2025).

For each sample $y_t$0, KPB first identifies its $y_t$1 nearest neighbors: $y_t$2 It then computes the average distance from $y_t$3 to these neighbors: $y_t$4 The flagging criterion is

$y_t$5

Equivalently,

$y_t$6

Algorithmically, the procedure is direct. Initialize $y_t$7, set $y_t$8, iterate through all samples, compute nearest neighbors, accumulate total distance, divide by $y_t$9, and add the sample to $x_p$0 whenever the resulting average is below $x_p$1. The cleaned dataset is then

$x_p$2

and the experimental protocol retrains the model $x_p$3 on $x_p$4.

Two characteristics distinguish KPB from the other defenses in the same study. It is conceptually simpler than Clustering-Based Defense (CBD), which requires k-means clustering and the elbow method, and unlike Mean Distance Threshold (MDT) it does not require computing class means. The paper also states that KPB does not require knowledge of which class is target or non-target; it uses only local proximity in feature space.

3. Hyperparameters, implementation, and complexity

The recommended implementation begins with feature normalization consistent with classifier training, such as per-pixel normalization for images. If dimensionality is very high, PCA or another embedding may optionally be applied for computational or numerical reasons, although KPB itself does not require dimensionality reduction. Neighbor search is performed over the entire suspicious dataset $x_p$5, commonly excluding the query point itself from its neighbor set. Exact search by brute force, k-d trees, or ball trees is suggested for moderate $x_p$6 and modest dimensionality, while approximate kNN libraries such as FAISS, Annoy, and FLANN are suggested for large datasets or high dimension (Gulciftci et al., 2 Aug 2025).

The paper’s default setting is

$x_p$7

so that

$x_p$8

The stated rationale is that this scales with dataset size and avoids manually tuning $x_p$9 in absolute terms. The paper does not provide a sensitivity analysis over $P(x_p \mid y_t)$0; it fixes $P(x_p \mid y_t)$1 and focuses on $P(x_p \mid y_t)$2. It also notes the trade-off that too small $P(x_p \mid y_t)$3 yields tiny neighborhoods and a noisy estimate, whereas too large $P(x_p \mid y_t)$4 washes out the local density signal.

Threshold selection is presented as the central tuning problem. On MNIST and CIFAR-10, the best-performing values are reported in the range

$P(x_p \mid y_t)$5

under the paper’s normalization and feature scale. For very small $P(x_p \mid y_t)$6, both precision and recall drop sharply because the threshold becomes too strict and true poisons are missed. For large $P(x_p \mid y_t)$7, recall remains high but precision drops because legitimate points in moderately dense regions begin to be flagged. The practical guideline is that $P(x_p \mid y_t)$8 “should be neither too large nor too small,” and the paper explicitly characterizes the optimal $P(x_p \mid y_t)$9 as dataset-dependent.

When labeled poisoned points are unavailable, the suggested procedure is to compute $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$0 for all points, inspect the empirical distribution, and choose $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$1 near the lower tail where an unusually dense cluster is expected, for example by using a small percentile cut-off such as $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$2 to $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$3, possibly adjusted with validation.

The computational bottleneck is neighbor search. A naïve implementation requires $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$4 distance computations and $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$5 neighbor selection overall, while space complexity is $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$6 for storing the dataset and $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$7 extra space if points are processed sequentially. This suggests that KPB is naturally suited to pre-processing settings, where the cost is paid once per dataset rather than per query.

4. Empirical performance and comparison within the Beta Poisoning study

The evaluation treats defense as a binary classifier at the sample level and reports accuracy, precision, recall, and F1. Experiments are reported on MNIST, CIFAR-10, and preliminarily on CIFAR-100, all at a 20% poison rate. On MNIST and CIFAR-10, KPB achieves perfect scores under optimized $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$8; on CIFAR-100 it remains near-perfect but not flawless, with precision preserved at $$\arg\max_{x_p} \; P(x_p \mid y_t) \quad \text{s.t.} \quad \mathbf{x}_{lb} \preceq x_p \preceq \mathbf{x}_{ub}.$$9 and recall reduced to $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$0. The tuning plots show a characteristic regime change: precision and recall are close to zero at very small $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$1, both reach $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$2 in the mid-range, and at large $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$3 precision falls while recall stays near $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$4, indicating increasing false positives (Gulciftci et al., 2 Aug 2025).

Dataset	Accuracy / F1 / Precision / Recall	Notes
MNIST	$$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$5	Optimized $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$6
CIFAR-10	$$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$7	Optimized $$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$8
CIFAR-100	$$x_p = \psi(\boldsymbol{\beta}, \mathcal{S}) = \sum_{x_i \in \mathcal{S}} \beta_i x_i,$$9	Preliminary experiment

These results imply different operating profiles across datasets. On MNIST and CIFAR-10, KPB fully recovers a clean training set in the reported setting: all poisoning samples are detected and no clean samples are falsely flagged. On CIFAR-100, the outcome is more conservative: all flagged points are actually poisoned, but a small fraction of poisoning samples remain undetected. This suggests that increasing data complexity makes the density-based separation harder to exploit perfectly, while leaving the basic mechanism intact.

Within the same study, KPB is compared to Neighborhood Class Comparison (NCC), Clustering-Based Defense (CBD), and Mean Distance Threshold (MDT). Under optimized parameters, MDT also achieves perfect $\mathcal{S} = \{x_1,\dots,x_k\}$0 scores on MNIST and CIFAR-10; CBD is perfect on MNIST and very high but slightly non-perfect on CIFAR-10, with accuracy $\mathcal{S} = \{x_1,\dots,x_k\}$1, F1 $\mathcal{S} = \{x_1,\dots,x_k\}$2, precision $\mathcal{S} = \{x_1,\dots,x_k\}$3, and recall $\mathcal{S} = \{x_1,\dots,x_k\}$4; NCC shows lower precision and overall accuracy because of many false positives. KPB therefore matches the best-performing method in the paper while using only local proximity and no class-mean information.

5. Position within the broader nearest-neighbor defense literature

KPB belongs to a broader family of nearest-neighbor defenses, but its role within that family is unusually specific. In the Beta Poisoning setting it is a training-set sanitization mechanism based on high local density. By contrast, other nearest-neighbor defenses use proximity in representation space at test time, certification by vote margins, or semantic retrieval over a trusted database. A representation-space defense classifies a test input by kNN over intermediate activations and, in a 1-NN plus Lipschitz-network variant, derives a certified lower bound

$\mathcal{S} = \{x_1,\dots,x_k\}$5

on the perturbation required to change the decision. A certified poisoning defense for kNN and rNN uses the vote margin

$\mathcal{S} = \{x_1,\dots,x_k\}$6

to guarantee prediction stability under bounded poisoning. In 3D point cloud classification, KNN-Defense performs nearest-neighbor search in feature space and aggregates neighbors’ softmax outputs instead of reconstructing geometry. For clean-label data poisoning on CIFAR-10, a Deep k-NN defense filters training examples whose labels disagree with the plurality label among their deep-feature neighbors, detecting over $\mathcal{S} = \{x_1,\dots,x_k\}$7 of poisoned examples in both feature collision and convex polytope attacks. ASK-Def goes further by training DkNN with a soft kNN loss that directly regularizes proximity relations under adversarial perturbations (Sitawarin et al., 2019, Jia et al., 2020, Jamali et al., 7 Jun 2025, Peri et al., 2019, Wang et al., 2021).

This comparison clarifies what is distinctive about KPB in the strict sense of the Beta Poisoning literature. It is not a classifier replacement, not a certification mechanism, and not an embedding-learning objective. It is a specialized pre-processing detector that assumes poisoning samples form an anomalously dense local cluster. A plausible implication is that KPB should be viewed less as a generic property of kNN and more as a geometry-matched countermeasure against one particular poisoning construction.

6. Limitations, controversy, and open directions

The principal limitation stated for KPB is dependence on attack structure. The defense leverages specific traits of Beta Poisoning, especially the dense cluster produced by linearly combined prototypes. If an attacker uses a poisoning method that does not create such clusters, KPB could be less effective. The paper does not examine KPB against other poisoning attacks and explicitly leaves that for future work. It also identifies dataset dependence of $\mathcal{S} = \{x_1,\dots,x_k\}$8, the difficulty of perfect separation on more complex data such as CIFAR-100, the $\mathcal{S} = \{x_1,\dots,x_k\}$9 cost of naïve kNN search, and the prospect of adaptive attacks specifically optimized to evade the proposed defenses, for example by spreading poisoning samples more diffusely (Gulciftci et al., 2 Aug 2025).

Broader nearest-neighbor research sharpens these caveats. Attack papers on deep kNN and other kNN-based models show that non-differentiability is not itself a defense: gradient-based attacks can produce stronger results than earlier methods and can make several kNN-based defenses appear less robust than standard adversarial training under proper evaluation. Theory further shows that nearest-neighbor robustness depends critically on the value of $y_t$0: constant $y_t$1 can be asymptotically non-robust wherever $y_t$2, whereas robustness approaches that of the Bayes Optimal classifier for fast-growing $y_t$3. Geometric work on $y_t$4-NN poisoning also indicates that strong poisoning attacks are highly local and act by inducing local majority changes in small regions. This suggests that KPB’s present form is best understood as one effective point in a larger design space whose general problem is still open: how to convert local proximity structure into defenses that remain robust under adaptive, geometry-aware poisoning strategies (Sitawarin et al., 2020, Sitawarin et al., 2019, Wang et al., 2017, Centurion et al., 2023).