KDE-KNN Augmentation Methods

• KDE-KNN Augmentation is a method that combines kernel density estimation with nearest-neighbor techniques to generate synthetic data and preserve local structure.
• It decouples the process by using KDE for spatial or density estimation and KNN for transferring or interpolating local features, ensuring both realism and efficiency.
• The approach is versatile, with successful applications in outdoor positioning, imbalanced classification, graph construction, and retrieval-augmented generation, yielding significant performance gains.

KDE-KNN augmentation denotes a class of methods in which kernel density estimation (KDE) and nearest-neighbor mechanisms are combined so that density estimates govern where augmentation should occur, while nearest-neighbor structure governs what should be transferred, interpolated, or weighted. In the most explicit formulation, the method is a two-stage, training-free pipeline for outdoor multi-cell fingerprinting-based positioning: KDE generates geographically coherent synthetic locations from the empirical distribution of MDT records, and a KNN-based block synthesizes per-cell radio fingerprints by transferring measurements from the nearest observed location and perturbing them with Gaussian shadowing (Chahoud et al., 23 Sep 2025). Closely related formulations appear in imbalanced classification, where KDE identifies safe minority regions before KNN interpolation (Miraj et al., 14 Sep 2025), in adaptive kNN graph construction, where KDE-informed density controls node-wise neighborhood size (Min et al., 2023), and in kNN-augmented sequence generation, where the neighbor posterior is interpreted as a KDE-style distribution over discrete outputs (Zhu et al., 2023, Deguchi et al., 2023).

1. Conceptual scope and recurring design pattern

Across the cited literature, KDE and KNN occupy complementary roles. KDE estimates empirical density, scores local support, or defines a smooth kernel over retrieved samples; KNN supplies local structure, whether by selecting a donor sample, building an interpolation neighborhood, adapting graph degree, or retrieving token-level examples from a datastore. This decomposition is explicit in outdoor positioning and GK-SMOTE, and implicit in kNN-MT frameworks that transform distance-weighted neighbor sets into predictive distributions (Chahoud et al., 23 Sep 2025, Miraj et al., 14 Sep 2025, Zhu et al., 2023).

Context	KDE role	KNN role
Outdoor MDT positioning	Model empirical 2D location density; sample synthetic coordinates	Transfer nearest measured fingerprint and add Gaussian shadowing
GK-SMOTE	Score minority density; separate safe, borderline, and noisy regions	Filter via neighbor composition; interpolate among minority points
DaNNG	Approximate sample density and global distribution	Adapt per-node $k$ and construct the graph
kNN-MT frameworks	Form a KDE-style posterior from distance-weighted neighbors	Retrieve top-$k$ datastore entries

This pattern should not be reduced to a single algorithm. In some settings, KDE is global and KNN is fixed, as in the outdoor positioning framework with Gaussian KDE and $K=1$ spatial neighbor transfer. In others, KDE is local and KNN is adaptive, as in DaNNG. In retrieval-augmented generation, KDE is not used for synthetic sample creation at all; it is used to turn retrieved neighbors into a smoothed posterior over tokens (Min et al., 2023, Deguchi et al., 2023).

2. Outdoor multi-cell fingerprinting formulation

In outdoor multi-cell fingerprinting-based positioning, the reference database consists of geo-tagged radio fingerprints, typically vectors of LTE RSRP values indexed by PCI. Position estimation proceeds by matching an observed radio vector to nearby fingerprints in radio-feature space. The operational problem is that MDT data, although cost-efficient relative to exhaustive drive tests, are sparse, heterogeneous, and expensive to densify through repeated surveying. The stated objective is therefore to augment operator-collected MDT data with large volumes of realistic synthetic samples that fill spatial gaps and preserve empirical radio statistics, without additional survey campaigns (Chahoud et al., 23 Sep 2025).

The architecture decouples spatial synthesis from radio-feature synthesis. For spatial augmentation, let $\mathbf{p}\in\mathbb{R}^2$ denote latitude-longitude and let $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$. The KDE module models the empirical density as

$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$

with Gaussian KDE motivated by the fact that MDT locations are obtained with UE-side GPS, where measurement noise is commonly modeled as Gaussian. The bandwidth $h$ is treated as a hyperparameter and tuned by empirical risk minimization through validation negative log-likelihood,

$$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$

Silverman’s or Scott’s rules are not used; boundary handling, region masks, and stratification are not specified (Chahoud et al., 23 Sep 2025).

For radio synthesis, the measured reference database is

$$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$

where $\hat{\mathbf r}_j \in \mathbb R^{N_c}$ is a per-PCI RSRP vector and absent PCIs are represented as missing values. Given a synthetic location $k$0, the radio block selects the nearest measured location in Euclidean latitude-longitude,

$k$1

copies the neighbor’s PCI-wise RSRP values, and perturbs them by additive Gaussian noise in dB,

$k$2

The framework uses $k$3, does not use haversine or projected CRS distances, leaves absent PCIs unassigned, and does not impose explicit cross-cell correlation constraints. Its interpretability follows from the fact that KDE bandwidth, Gaussian kernel choice, and the KNN transfer procedure are explicit, and its training-free character follows from the absence of iterative RF, GPR, or DNN training loops (Chahoud et al., 23 Sep 2025).

3. Validation, operating regimes, and end-to-end gains

The outdoor positioning study evaluates the spatial generator, the radio generator, and the downstream localization effect on real MDT data from an Italian MNO across Bologna in four regions: city center, stadium, airport, and highway. The regions differ markedly in area, user density, shadowing variance, and PCI count: city center has area $k$4, density $k$5 UEs/km², $k$6 dB, and $k$7 PCIs; stadium has area $k$8, density $k$9 UEs/km², $K=1$0 dB, and $K=1$1 PCIs; airport has area $K=1$2, density $K=1$3 UEs/km², $K=1$4 dB, and $K=1$5 PCIs; highway has area $K=1$6, density $K=1$7 UEs/km², $K=1$8 dB, and $K=1$9 PCIs. Propagation is predominantly NLOS, and shadow fading follows 3GPP TR 38.901 (Chahoud et al., 23 Sep 2025).

Spatial validation uses multivariate two-sample KS tests between real MDT locations and synthetic samples. KDE and GMM yield small KS statistics and non-significant p-values across regions, whereas GAN and normalizing flows often require additional regularization to match empirical MDT distributions. Representative KDE averages over 10 runs are $\mathbf{p}\in\mathbb{R}^2$0 with $\mathbf{p}\in\mathbb{R}^2$1 in the city center, $\mathbf{p}\in\mathbb{R}^2$2 with $\mathbf{p}\in\mathbb{R}^2$3 in the stadium, $\mathbf{p}\in\mathbb{R}^2$4 with $\mathbf{p}\in\mathbb{R}^2$5 at the airport, and $\mathbf{p}\in\mathbb{R}^2$6 with $\mathbf{p}\in\mathbb{R}^2$7 on the highway. For $\mathbf{p}\in\mathbb{R}^2$8, KDE and GMM are not rejected across regions (Chahoud et al., 23 Sep 2025).

Radio synthesis is evaluated through per-PCI RSRP prediction MAE. KNN consistently attains the lowest MAE: $\mathbf{p}\in\mathbb{R}^2$9 in the city center, $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$0 in the stadium, $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$1 at the airport, and $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$2 on the highway. RF and GPR variants lag behind, although GPR with the rational quadratic kernel generally outperforms squared exponential GPR (Chahoud et al., 23 Sep 2025).

The downstream positioning estimator is wKNN in radio space, using

$\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$3

and a weighted average of neighbor coordinates. Performance is reported as mean positioning error in meters, with augmentation factors $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$4, where $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$5 is the original database (Chahoud et al., 23 Sep 2025).

Region	Baseline $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$6	Best reported KDE-KNN
City center	$\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$7 m	$\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$8 m at $\mathcal{D}=\{\mathbf{p}_i\}_{i=1}^m$9
Stadium	$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$0 m	$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$1 m at $$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$2
Airport	$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$3 m	$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$4 m at $$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$5
Highway	$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$6 m	$$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$7 m at $$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$8

These results imply markedly different operating regimes. Dense regions such as city center and stadium show significant improvements up to approximately $$f(\mathbf{p}, \mathcal{D}, h) = \frac{1}{m}\sum_{i=1}^{m} K(\mathbf{p} - \mathbf{p}_i, h),$$9–$h$0, after which gains saturate. The airport shows marginal gains and early saturation. The highway shows persistent gains up to $h$1, which the paper attributes to sparse, anisotropic sampling, so that augmentation fills gaps more effectively. This suggests that KDE-KNN augmentation is most valuable when sparsity and structural complexity dominate the positioning error budget (Chahoud et al., 23 Sep 2025).

4. Density-aware oversampling and adaptive neighborhoods

In imbalanced classification, KDE-KNN augmentation appears in GK-SMOTE as a density-aware extension of SMOTE. The method begins by computing the imbalance ratio, filters minority samples whose $h$2 nearest neighbors are all majority points ($h$3), computes a local Gaussian KDE score

$h$4

clusters the remaining density scores by 2-means into higher-density and lower-density subsets, and then allocates synthetic samples proportionally before KNN-based interpolation in high-density minority regions. The paper characterizes minority samples as noisy, borderline, or safe, with KDE acting as the density-aware gate and KNN supplying local structure for both filtering and interpolation. The authors describe the method as hyperparameter-free and noise-resilient, but the paper does not specify the Gaussian kernel’s explicit functional form, bandwidth selection strategy, or the exact interpolation equation used in the Populate routine (Miraj et al., 14 Sep 2025).

Empirically, GK-SMOTE is evaluated on 27 UCI binary datasets with classifiers including Random Forest, LightGBM, Logistic Regression, KNN, and Decision Tree. Reported relative MCC gains over SMOTE under $h$5 include $h$6 for RF, $h$7 for LightGBM, $h$8 for LR, $h$9 for KNN, and $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$0 for DT; under $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$1, the gains remain substantial, including $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$2 for RF and $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$3 for KNN. Across 27 datasets, GK-SMOTE achieves top performance on 21 datasets under both $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$4 and $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$5 conditions (Miraj et al., 14 Sep 2025).

A different use of KDE-KNN coupling appears in DaNNG, where the goal is not sample generation but graph construction with adaptive neighborhood size. DaNNG estimates density by KDE,

$$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$6

learns a one-dimensional fitness kernel $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$7 by minimizing a sample-wise KL divergence between the data distribution and the kernel distribution, and maps the result into node-wise degree through

$$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$8

Borderline samples are identified by the bottom $$h^* = \arg\min_h - \sum_{i=1}^{m'} \log f(\mathbf{p}_i', \mathcal{D}, h).$$9 of density estimates and split by topology criteria into cases requiring smaller or larger $$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$0. Average accuracy is reported as approximately $$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$1 for DaNNG, versus best baselines around $$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$2–$$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$3, and average macro F1 as approximately $$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$4 versus a best baseline of approximately $$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$5. The maximum relative accuracy gain over plain kNNG reaches approximately $$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$6 on BCC (Min et al., 2023).

Taken together, these two lines of work show that KDE need not generate samples directly. It can instead determine where KNN interpolation is permissible, as in GK-SMOTE, or how large local neighborhoods should be, as in DaNNG. A plausible implication is that “augmentation” in KDE-KNN methods is as much about restructuring local support as about synthesizing new points.

5. Retrieval-time KDE interpretations and neighboring paradigms

In kNN-augmented sequence generation, the KDE-KNN relationship takes a probabilistic form. kNN-BOX decomposes the method into a datastore, a retriever, and a combiner. A token-level datastore

$$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$7

stores continuous decoder states as keys and reference tokens as values. At decoding time, the retriever finds top-$$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$8 neighbors and the combiner converts them into a KDE-style token posterior,

$$\hat{\mathbf R} = \bigl\{(\hat{\mathbf r}_j,\;\hat{\mathbf p}_j)\bigr\}_{j=1}^m,$$9

which is fused with the base model through

$\hat{\mathbf r}_j \in \mathbb R^{N_c}$0

In this view, the retrieved neighbor set is a kernel density estimate over discrete labels rather than a mechanism for producing synthetic features (Zhu et al., 2023).

kNN-BOX implements Vanilla, Adaptive, Smoothed, Robust, PCK, Efficient, and PLAC kNN-MT variants. On WMT’19 De-En OPUS domains, the Robust kNN-MT configuration reaches Law BLEU $\hat{\mathbf r}_j \in \mathbb R^{N_c}$1, Medical $\hat{\mathbf r}_j \in \mathbb R^{N_c}$2, IT $\hat{\mathbf r}_j \in \mathbb R^{N_c}$3, and Koran $\hat{\mathbf r}_j \in \mathbb R^{N_c}$4, versus base scores of Law $\hat{\mathbf r}_j \in \mathbb R^{N_c}$5, Medical $\hat{\mathbf r}_j \in \mathbb R^{N_c}$6, IT $\hat{\mathbf r}_j \in \mathbb R^{N_c}$7, and Koran $\hat{\mathbf r}_j \in \mathbb R^{N_c}$8. A mixed configuration, PLAC Datastore at $\hat{\mathbf r}_j \in \mathbb R^{N_c}$9 scale with Robust Combiner, reaches Law BLEU $k$00 (Zhu et al., 2023).

knn-seq presents the same KDE-like interpretation at billion scale. The datastore is

$k$01

the neighbor posterior is

$k$02

and it is interpolated with the NMT posterior as

$k$03

On WMT’19 De→En, base MT obtains BLEU $k$04, chrF $k$05, and COMET $k$06; kNN-MT reaches BLEU $k$07, chrF $k$08, and COMET $k$09; kNN-MT + OPQ reaches BLEU $k$10, chrF $k$11, and COMET $k$12. The billion-scale datastore construction over $k$13M target tokens from $k$14M sentence pairs takes $k$15 seconds, approximately $k$16 hours, on 8 NVIDIA V100 GPUs and 16-core CPUs (Deguchi et al., 2023).

A useful contrast is Minimax-kNN for knowledge distillation in NLP. It also uses nearest-neighbor augmentation, but the paper explicitly states that no KDE is used. Instead, it retrieves neighbors from an unlabeled repository, scores them by teacher-student KL divergence, and selects the top-$k$17 most informative neighbors. The comparison is important because it guards against a common conflation: not every kNN-based augmentation is KDE-KNN augmentation (Kamalloo et al., 2021).

6. Limitations, deployment constraints, and recurrent misconceptions

The literature identifies several recurrent failure modes. In outdoor positioning, overly large bandwidth $k$18 can smear spatial structure; no road masks, land-use masks, terrain features, or antenna metadata are used; the approach is context-agnostic by design; and there are no explicit cross-cell correlation constraints in the radio synthesis block. Region-dependent saturation is a central empirical limitation: adding synthetic samples beyond a threshold yields diminishing returns, and the airport scenario shows that additional synthetic data can harm performance if radio synthesis is mismatched (Chahoud et al., 23 Sep 2025).

In GK-SMOTE, the authors emphasize robustness and efficiency, but the paper does not specify bandwidth $k$19 selection, explicit density thresholds, or the interpolation formula. In DaNNG, high-dimensional KDE remains subject to the curse of dimensionality, and the paper recommends mitigations such as PCA, local kNN-KDE, class-wise KDE, and bandwidth refinement by cross-validation. In retrieval-augmented generation, latency and memory dominate the engineering burden: large datastores, high $k$20, and approximate search structures such as IVFPQ, PQ, OPQ, and ADC define the trade-off between KDE fidelity and throughput (Miraj et al., 14 Sep 2025, Min et al., 2023, Deguchi et al., 2023).

Several misconceptions recur across domains. First, KDE-KNN augmentation does not imply deep generative modeling. The outdoor MDT framework is explicitly training-free and interpretable, and KDE is favored over GANs and normalizing flows because it yields statistically consistent spatial samples with lower complexity in that setting (Chahoud et al., 23 Sep 2025). Second, KDE-KNN augmentation does not imply adaptive $k$21. The radio synthesis module uses $k$22, whereas DaNNG adapts $k$23 per node (Chahoud et al., 23 Sep 2025, Min et al., 2023). Third, KDE need not generate synthetic points; in kNN-MT it acts as a kernelized posterior over retrieved labels (Zhu et al., 2023, Deguchi et al., 2023). Fourth, positive augmentation results for KNN do not automatically establish corresponding gains for KDE-based density modeling. In unsupervised visual inspection under extremely low-data regimes, KNN, Mahalanobis, and PaDiM all benefit from original image-space augmentation, but KDE-based anomaly detection was not evaluated; the paper only states that analogous benefits for KDE-style density modeling are plausible and not experimentally validated (Gutierrez et al., 2021).

The broader significance of KDE-KNN augmentation is therefore methodological rather than tied to a single domain. It supplies a nonparametric mechanism for injecting empirical density information into neighbor-based reasoning. Depending on the application, that may mean sampling synthetic coordinates, restricting interpolation to safe minority regions, adapting graph degree near decision boundaries, or turning retrieved exemplars into a smooth posterior at inference time. The cited work consistently shows that the value of this combination is greatest when data are sparse, heterogeneous, noisy, or structurally ambiguous, and that its practical success depends on disciplined handling of bandwidth, neighborhood definition, and saturation effects (Chahoud et al., 23 Sep 2025, Miraj et al., 14 Sep 2025, Min et al., 2023).