---
title: Deep k-Nearest Neighbors (DkNN)
url: https://www.emergentmind.com/topics/deep-k-nearest-neighbors-dknn
type: topic
---

# Deep k-Nearest Neighbors (DkNN)

Searching arXiv for the cited DkNN-related papers to ground the article in current bibliographic records.
Deep k-Nearest Neighbors (DkNN) is a hybrid inference procedure that combines a trained deep neural network with $k$-nearest-neighbor retrieval in the network’s learned representation spaces. Instead of relying solely on the final parametric classifier, DkNN compares a test input to neighboring training points at one or multiple layers and uses the labels of those neighbors to support prediction, confidence estimation, and explanation [1803.04765]. In its original formulation, the method was introduced to address three recurrent deficiencies of standard deep models—unreliable confidence away from the training manifold, limited interpretability, and fragility to adversarial perturbations—by grounding decisions in layer-wise neighborhood structure and calibrating support with conformal prediction [1803.04765]. Subsequent work examined its robustness under adaptive white-box attack [1903.08333], extended the framework to adversarially enhanced neighbor aggregation [2108.06797], and explored parametric approximations of deep-space $k$NN behavior through a learned network, termed kNet, that operates on penultimate-layer features [2107.09735].

## 1. Origins and defining formulation

DkNN was introduced in "Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning" [1803.04765]. The central construction is to take a trained deep network $f$ and, for each layer $\ell$, use the representation $f_\ell(x)$ of an input $x$ as the basis for a nearest-neighbor query against stored training representations from the same layer [1803.04765].

In the original formulation, for a test input $x$ and layer $\ell$, one retrieves the $k$ nearest training representations under angular distance and denotes their labels by $\Omega_\ell(x)$ [1803.04765]. DkNN then defines a label-wise nonconformity score by counting disagreements between a candidate label $y$ and the labels of retrieved neighbors across layers:
$$
\alpha_\ell(x, y) = \left| \{ i \in N_\ell(x) : y_i \neq y \} \right|,
$$
and
$$
\alpha(x, y) = \sum_{\ell=1}^{L} \alpha_\ell(x, y).
$$
A labeled calibration set, disjoint from the training and evaluation sets, is used to form empirical $p$-values
$$
p_y(x) = \frac{\left| \{ \alpha \in A : \alpha \geq \alpha(x, y) \} \right|}{|A|},
$$
with prediction, confidence, and credibility defined by
$$
\hat{y}(x) = \arg\max_y p_y(x),
$$
$$
\mathrm{conf}(x) = 1 - \max_{y \neq \hat{y}(x)} p_y(x),
$$
and
$$
\mathrm{cred}(x) = \max_y p_y(x) = p_{\hat{y}(x)}(x)
$$
[1803.04765].

This formulation distinguishes DkNN from a conventional single-layer $k$NN classifier. The method is not merely majority voting on a final embedding; rather, its original version aggregates layer-wise evidence through conformal nonconformity and uses a calibration set to obtain empirical support scores [1803.04765]. A later paper, "Deep Adversarially-Enhanced k-Nearest Neighbors," operationalized DkNN differently, using per-layer class-support fractions
$$
p_l^c(x; D_{\text{tr}^b}) = \frac{K_c^l}{K}
$$
and summing these across selected layers without conformal calibration [2108.06797]. This establishes an important terminological distinction: the same label “DkNN” has been used both for the original conformal formulation and for simpler multi-layer neighbor-voting variants.

## 2. Representation-space retrieval and inference mechanics

The defining mechanism of DkNN is representation-space retrieval. The original paper stores, for each training sample $x_i$ with label $y_i$, the representation $f_\ell(x_i)$ at each analyzed layer $\ell$ together with $y_i$ [1803.04765]. At inference time, the test input is propagated through the trained network, and each selected layer issues a nearest-neighbor query in that layer’s representation space.

The original DkNN work uses cosine similarity, or equivalently angular distance, between representation vectors, with data-dependent locality-sensitive hashing for angular distance as the approximate nearest-neighbor backend [1803.04765]. The 2019 robustness analysis notes that, in its equations, distance is written as Euclidean,
$$
d(a,b) = \|a-b\|_2,
$$
but that the implementation follows Papernot et al. and uses cosine distance by normalizing all representations to unit norm; with normalization, cosine distance is equivalent to Euclidean distance [1903.08333]. By contrast, the DAEkNN paper uses Euclidean distance in the layer-wise feature space by default [2108.06797]. These descriptions are method-specific rather than contradictory: they reflect different instantiations of deep-space neighbor search in later work.

The original end-to-end DkNN pipeline consists of four stages: train a DNN normally; precompute and store layer-wise training representations and build one approximate nearest-neighbor index per layer; hold out a labeled calibration set and compute calibration nonconformities; then, for each test input, retrieve neighbors at each layer, compute nonconformity scores and empirical $p$-values, and output prediction, confidence, credibility, and optionally neighbor exemplars [1803.04765].

A concise comparison of the principal formulations described in the cited papers is given below.

| Variant | Layer usage | Decision mechanism |
|---|---|---|
| Original DkNN | All hidden layers were used | Conformal nonconformity and empirical $p$-values [1803.04765] |
| DkNN in DAEkNN paper | Selected layers $L$ | Sum of per-layer class supports $K_c^l/K$ [2108.06797] |
| kNet instantiation | Penultimate layer | Neural approximation of deep-space $k$NN distribution [2107.09735] |

The practical cost of this design is explicit in the source material. DkNN requires storage of layer-wise training representations and per-layer index structures, and query time adds multiple nearest-neighbor searches to the underlying forward pass [1803.04765][1903.08333]. This suggests that DkNN trades low-latency parametric inference for exemplar-based evidence and calibration.

## 3. Confidence, credibility, and exemplar-based interpretability

A defining contribution of the original DkNN paper is the use of conformal prediction to convert layer-wise neighbor agreement into calibrated notions of support [1803.04765]. In that framework, the distinction between confidence and credibility is explicit. Confidence is
$$
1 - \max_{y \neq \hat{y}(x)} p_y(x),
$$
whereas credibility is the largest label-wise $p$-value,
$$
\max_y p_y(x).
$$
Confidence reflects separation from alternative labels, while credibility reflects how well any label is supported by the training manifold as represented by the calibration nonconformities [1803.04765].

The paper reports that reliability diagrams show softmax confidence concentrated at $>0.8$ for most test inputs, while DkNN credibility spreads across the full range [1803.04765]. On out-of-distribution inputs, DkNN credibility drops substantially relative to softmax confidence. The reported mean softmax versus DkNN credibility values are: MNIST $\rightarrow$ NotMNIST, $33\%$ versus $6\%$; MNIST rotated $45^\circ$, $31\%$ versus $9\%$; SVHN $\rightarrow$ CIFAR-10, $52\%$ versus $15\%$; and SVHN rotated $45^\circ$, $33\%$ versus $18\%$ [1803.04765]. These results support the original claim that DkNN estimates the lack of support for inputs outside the model’s training manifold [1803.04765].

Interpretability in DkNN is exemplar-based rather than feature-attribution-based. The method returns, per layer, the actual training examples whose representations are closest to the test input [1803.04765]. The original paper presents an ImageNet/ResNet case in which an image of Barack Obama throwing a football is classified as basketball; the nearest neighbors at the last hidden layer are mostly basketball players, and cropping the ball changes the prediction to racket with tennis-player neighbors [1803.04765]. The same paper also reports that high-credibility DkNN misclassifications on MNIST and SVHN surfaced samples with incorrect or ambiguous ground-truth labels, suggesting a route to dataset cleaning [1803.04765].

A common misconception is to treat DkNN credibility as interchangeable with conventional softmax confidence. The original formulation rejects that equivalence: credibility is explicitly tied to empirical support from the training manifold through conformal calibration, whereas softmax confidence is not [1803.04765]. Another misconception is to regard DkNN explanations as post hoc saliency. In the cited work, the explanation mechanism is instead the retrieved neighbor set itself [1803.04765].

## 4. Robustness claims and adversarial evaluation

DkNN was proposed partly as a response to adversarial fragility, with the argument that adversarial examples often lose consistent layer-wise support from the training set [1803.04765]. The original evaluation reports accuracy improvements over the base DNN on adversarial examples generated by FGSM, BIM, and CW. For MNIST, DNN versus DkNN accuracy is reported as $27.1\%$ versus $54.9\%$ on FGSM, $0.7\%$ versus $16.8\%$ on BIM, and $0.7\%$ versus $94.4\%$ on CW; for SVHN, $9.3\%$ versus $28.6\%$, $4.7\%$ versus $17.9\%$, and $4.7\%$ versus $80.5\%$; for GTSRB, $12.3\%$ versus $22.3\%$, $6.5\%$ versus $13.6\%$, and $3.0\%$ versus $74.5\%$ [1803.04765]. The same paper also states that DkNN credibility is typically below $0.5$ for most adversarial inputs unless the prediction is correct [1803.04765].

However, the later paper "On the Robustness of Deep K-Nearest Neighbors" argues that evaluating DkNN requires adaptive attacks tailored to the non-differentiability of $k$NN voting [1903.08333]. It introduces a gradient-based surrogate in which the hard inclusion of a point among the $k$ nearest neighbors is approximated by a sigmoid threshold on distance. For DkNN, the multi-layer surrogate objective is
$$
\min_\delta \sum_{i=1}^{m} \sum_{\lambda=1}^{L} w_i \cdot \sigma\!\left( \| f_\lambda(x_i) - f_\lambda(z+\delta) \|_2 - \eta_\lambda \right)
$$
subject to norm and box constraints [1903.08333].

Under this adaptive white-box threat model on MNIST, the reported clean DkNN accuracy is $98.83\%$, but the proposed gradient attack reduces accuracy to $17.44\%$ under $\ell_\infty$ with $\epsilon = 0.2$, to $0.00\%$ under $\ell_2$, and to $0.59\%$ in a fixed-$\ell_2$ setting with $\epsilon = 3.476$ [1903.08333]. The paper further reports that, as $\epsilon$ increases in the $\ell_\infty$ setting, DkNN accuracy under the gradient attack reaches $0$ at $\epsilon = 0.3$ [1903.08333]. It also notes that thresholding credibility at $0.1$ reduces clean accuracy to $91.15\%$ while still allowing $28\%$ of adversarial examples at $\epsilon = 0.2$ and $43\%$ at $\epsilon = 0.3$ to pass [1903.08333].

These results establish a substantive controversy in the DkNN literature. The original work shows that DkNN can lower credibility on unsupported inputs and improve adversarial handling relative to the underlying classifier [1803.04765]. The later attack paper shows that these properties are insufficient against fully adaptive, white-box adversaries [1903.08333]. A plausible implication is that DkNN’s robustness claims are best interpreted as relative and threat-model-dependent rather than absolute.

## 5. Adversarially enhanced variants

"Deep Adversarially-Enhanced k-Nearest Neighbors" proposes DAEkNN as a variant intended to mitigate the robustness-accuracy trade-off observed when DkNN is applied across deeper layers [2108.06797]. The paper states that, on a standard-trained model, adversarial accuracy under PGD drops dramatically as the layer goes deeper while standard accuracy monotonically increases with depth, yielding a pronounced trade-off [2108.06797].

DAEkNN modifies DkNN in two ways. First, it uses an adversarially trained backbone network. Second, it predicts by combining, per layer, a benign $K$NN built from $D_{\text{tr}^b}$ and an adversarial $K$NN built from an adversarially hardened training set $D_{\text{tr}^a}$ generated by PGD:
$$
D_{\text{tr}^a} = \mathrm{PGD}(\theta, \epsilon_r, D_{\text{tr}^b}).
$$
Per-layer class supports are $p_l^c(x; D_{\text{tr}^b}) = K_c^l/K$ and $p_l^c(x; D_{\text{tr}^a}) = K_c^l/K$, and the two are aggregated with distance-based softmax weights $\omega_a^l$ and $\omega_b^l$ satisfying $\omega_a^l + \omega_b^l = 1$ [2108.06797]. The final decision rule is
$$
y = \arg\max_{c \in [C]} \sum_{l \in \mathcal{L}} \Big[ \omega_a^l \, p_l^c(x; D_{\text{tr}^a}) + \omega_b^l \, p_l^c(x; D_{\text{tr}^b}) \Big].
$$

The reported evaluations use MNIST with a three-convolutional-layer CNN backbone and CIFAR-10 with VGG16, Euclidean distance in feature space, MNIST layers $2$ and $3$, and CIFAR-10 layers $3$ and $4$ [2108.06797]. Under $10$-step PGD with $\ell_\infty$ norm, evaluation attack powers are $\epsilon = 80$ for MNIST and $\epsilon = 8$ for CIFAR-10, both on pixel scale $[0,255]$ [2108.06797]. Table 1 in the paper reports the following:

| Dataset / method | SA | AA |
|---|---:|---:|
| MNIST DkNN | 98.78% | 0.06% |
| MNIST DAEkNN | 97.90% | 68.26% |
| CIFAR-10 DkNN | 91.78% | 0.06% |
| CIFAR-10 DAEkNN | 78.44% | 50.48% |

The same table gives harmonic mean values of $0.12\%$ for DkNN on both MNIST and CIFAR-10, versus $80.44\%$ and $61.43\%$ for DAEkNN respectively [2108.06797]. The paper’s leave-one-out ablation on CIFAR-10 reports that DAEkNN-WAT achieves SA $85.96\%$, AA $13.10\%$, and HM $22.74\%$; DAEkNN-WAD achieves SA $79.90\%$, AA $46.66\%$, and HM $58.91\%$; and full DAEkNN achieves SA $78.44\%$, AA $50.48\%$, and HM $61.43\%$ [2108.06797].

This line of work reframes DkNN not as a complete defense, but as a component whose behavior depends strongly on the robustness of the underlying feature extractor and on how the reference distribution is constructed. The paper explicitly notes that broader threat models and mixtures of hardening strengths and attack types are future directions [2108.06797].

## 6. Efficiency, scalability, and parametric approximation

A recurrent practical limitation of DkNN is storage and retrieval overhead. The original formulation requires storing representations of all training points at each selected layer and building one approximate nearest-neighbor index per layer [1803.04765]. The 2019 robustness analysis restates that storage scales with $\sum_l n \cdot d_l$ for precomputed representations, and query time requires $L$ approximate nearest-neighbor searches plus aggregation [1903.08333]. The DAEkNN extension increases this burden further because it stores deep features for both $D_{\text{tr}^b}$ and $D_{\text{tr}^a}$, effectively doubling storage relative to DkNN [2108.06797].

The paper "kNet: A Deep kNN Network To Handle Label Noise" addresses precisely these deployment costs, although it does so in the context of label-noise robustness rather than the original conformal DkNN setting [2107.09735]. Its abstract states that exact $k$NN requires a huge memory footprint to store all training samples and an advanced data structure for retrieval, and proposes a neural network, termed kNet, that learns to perform $k$NN so that, once trained, one no longer needs to store the training data and processing a query sample is a simple matter of inference [2107.09735].

In the detailed material provided for that paper, kNet is described as a network $g_\theta$ trained on penultimate-layer embeddings $z = f(x)$ from a preliminary network $f$, with targets given by a soft $k$NN label distribution in deep feature space [2107.09735]. The target distribution is
$$
p_{kNN}(c \mid x) = \sum_{i \in N_k(z)} w_i(z)\,\mathbf{1}[y_i = c],
$$
and the predictor is
$$
\hat{p}_\theta(c \mid x) = \mathrm{softmax}_c(h_\theta(z)),
$$
trained by cross-entropy to the $k$NN soft targets [2107.09735]. The source states that kNet gives a smooth approximation of $k$NN and cannot handle the sharp label changes between samples that $k$NN can exhibit, indicating that currently kNet is best suited to approximate $k$NN with a fairly large $k$ [2107.09735]. The abstract reports that, in practice, kNet consistently improve the results of all preliminary networks, in all label noise regimes, by up to $3\%$ [2107.09735].

This relationship to DkNN is conceptually important. The original DkNN obtains interpretability and calibration by explicit retrieval from the training set at inference time [1803.04765]. kNet instead distills deep-space neighbor behavior into parametric weights, removing training-set storage and neighbor search at deployment [2107.09735]. A plausible implication is that such an approximation gains efficiency while trading away the exemplar lists and conformal support structure that make DkNN interpretable in the original sense.

## 7. Position within the broader deep-neighbor literature

Within the cited works, DkNN occupies a distinct position at the intersection of deep representation learning, conformal prediction, neighbor-based reasoning, and adversarial robustness. The original paper explicitly frames it as a hybrid classifier that combines the $k$-nearest-neighbors algorithm with representations learned by each layer of a DNN [1803.04765]. The method therefore differs from ordinary deep classifiers by replacing sole reliance on the final linear head with training-set support checks in learned representation space.

The later literature shows two main directions of reinterpretation. One direction emphasizes robustness engineering: DAEkNN replaces the original conformal machinery with a weighted combination of benign and adversarial neighbor supports built on an adversarially trained backbone [2108.06797]. The other emphasizes deployability: kNet learns a parametric approximation to deep-space neighbor voting so that inference requires only a forward pass [2107.09735].

At the same time, the 2019 attack analysis demonstrates that DkNN is not a provable defense and that adaptive attacks can manipulate layer-wise neighbor composition, particularly in later layers [1903.08333]. This limits any strong claim that neighbor-based deep inference is intrinsically robust. What the cited literature does support is narrower: DkNN provides a principled mechanism for tying predictions to retrieved training evidence and, in its original formulation, for expressing support through conformal confidence and credibility [1803.04765]; its adversarial behavior, however, depends on attack adaptivity, representation quality, and the exact inference variant under consideration [1903.08333][2108.06797].

Taken together, these papers define DkNN less as a single fixed algorithm than as a family of deep-space nearest-neighbor inference schemes. The original version is characterized by layer-wise retrieval, nonconformity, and conformal calibration [1803.04765]. Subsequent variants retain the central idea of neighbor support in learned representations while altering aggregation rules, robustness mechanisms, or computational realization [2108.06797][2107.09735].

Source: https://www.emergentmind.com/topics/deep-k-nearest-neighbors-dknn