PointNN Selector: Nearest-Neighbor Condensation

• PointNN Selector is an algorithm that selects a sparse set of labeled points ensuring perfect nearest-neighbor classification using a separation constraint.
• It modifies the classic FCNN heuristic by incorporating a user-defined separation parameter δ to control density and prevent over-representation in training sets.
• The method provides a constant-factor approximation to the minimum consistent subset, with theoretical guarantees based on doubling dimension and nearest-enemy complexity.

A PointNN Selector is a selection algorithm for identifying representative or relevant points from a dataset embedded in a metric space, with formal guarantees on both accuracy and subset size. It is most prominently defined in the context of nearest-neighbor condensation, where the goal is to minimize the subset of labeled points required for perfect classification under the nearest neighbor rule. The term encompasses a specific algorithmic modification of the classic Fast Condensed Nearest Neighbor (FCNN) heuristic, introducing separation constraints to prevent pathological clusterings and to obtain size and approximation guarantees (Flores-Velazco, 2020). The PointNN Selector framework is distinguished by its theoretical foundation, making it applicable as a robust solution for subset selection tasks in geometric and learning contexts.

1. Problem Definition and Formalism

The PointNN Selector algorithm addresses the minimum consistent subset (Min-CS) problem for nearest-neighbor classification in metric spaces $(X,d)$. Given a labeled dataset $P \subset X$ with class labels $\ell:P\to\{1,2,\dots,c\}$, and the nearest neighbor function $nn_P(q) = \arg\min_{p\in P} d(q,p)$, the aim is to find a subset $S\subseteq P$ such that every $p\in P$ is classified correctly, i.e., $\ell(nn_S(p)) = \ell(p)$. The size of $S$ should be as small as possible, ideally approaching the minimum required for perfect consistency.

Key notations and structural parameters central to the problem include:

• Margin $\gamma$: The smallest distance between a point and its nearest enemy,

$\gamma = \min_{p \in P} d(p, \mathrm{ne}_P(p))$

where $P \subset X$0.

• Nearest-enemy complexity $P \subset X$1: The number of distinct nearest-enemy points in $P \subset X$2.
• Doubling dimension $P \subset X$3: The smallest integer such that every ball of radius $P \subset X$4 can be covered by $P \subset X$5 balls of radius $P \subset X$6.
• Diameter $P \subset X$7, assumed normalized to 1.

2. Classic FCNN and Its Limitations

The original FCNN heuristic (Angiulli 2007) builds the subset $P \subset X$8 iteratively:

1. Initialize $P \subset X$9 with the centroid of each class.
2. For each $\ell:P\to\{1,2,\dots,c\}$0, find misclassified points in $\ell:P\to\{1,2,\dots,c\}$1 for which $\ell:P\to\{1,2,\dots,c\}$2 is the nearest representative.
3. Add the closest such misclassified point to $\ell:P\to\{1,2,\dots,c\}$3.
4. Repeat until no misclassifications remain.

While this approach preserves nearest-neighbor accuracy, its output size can become pathological (arbitrarily large in $\ell:P\to\{1,2,\dots,c\}$4), especially when points are densely packed near class boundaries (Flores-Velazco, 2020).

3. PointNN Selector: Algorithmic Description

The PointNN Selector is a modification of FCNN, introducing a user-specified separation parameter $\ell:P\to\{1,2,\dots,c\}$5, usually set to the empirical margin $\ell:P\to\{1,2,\dots,c\}$6. The algorithm is as follows:

1. $\ell:P\to\{1,2,\dots,c\}$7 centroids of all classes.
2. For each $\ell:P\to\{1,2,\dots,c\}$8, enqueue any $\ell:P\to\{1,2,\dots,c\}$9 with $nn_P(q) = \arg\min_{p\in P} d(q,p)$0 and $nn_P(q) = \arg\min_{p\in P} d(q,p)$1.
3. While the queue $nn_P(q) = \arg\min_{p\in P} d(q,p)$2 is not empty:
   • Dequeue $nn_P(q) = \arg\min_{p\in P} d(q,p)$3.
   • If $nn_P(q) = \arg\min_{p\in P} d(q,p)$4 for all $nn_P(q) = \arg\min_{p\in P} d(q,p)$5, add $nn_P(q) = \arg\min_{p\in P} d(q,p)$6 to $nn_P(q) = \arg\min_{p\in P} d(q,p)$7.
   • For the new $nn_P(q) = \arg\min_{p\in P} d(q,p)$8, enqueue any additional misclassified points for which $nn_P(q) = \arg\min_{p\in P} d(q,p)$9 is now the closest.

The algorithm ensures $S\subseteq P$0 is always $S\subseteq P$1-separated; no two selected points are closer than $S\subseteq P$2. This enforces a packing constraint, preventing arbitrarily high local density in $S\subseteq P$3 (Flores-Velazco, 2020).

4. Theoretical Guarantees

The PointNN Selector is the first variant in this family to provide provable worst-case size bounds and approximation guarantees for the Min-CS problem:

• Packing Bound: In a metric space of doubling dimension $S\subseteq P$4 and diameter 1, the size of $S\subseteq P$5 is

$S\subseteq P$6

• Approximation Guarantee: Compared to the minimum-size consistent subset OPT, the PointNN Selector produces a $S\subseteq P$7-approximation:

$S\subseteq P$8

• These results are obtained by partitioning $S\subseteq P$9 by nearest-enemy and distance scale, showing that within each, packing numbers in doubling spaces limit cardinality.

If $p\in P$0, the algorithm always achieves exact consistency (zero error) for the training set. If $p\in P$1, a small number of boundary misclassifications may occur.

5. Parameter Selection and Practical Considerations

• Separation Parameter $p\in P$2: In practice, set to the empirical margin $p\in P$3 to guarantee consistency and optimal separation.
• Algorithmic Complexity: Each insertion spends $p\in P$4 time checking the separation constraint; total runtime is $p\in P$5, matching FCNN asymptotically.
• Queue Mechanics: The FIFO structure ensures that additions are well-ordered, and that density control is maintained throughout progress.

A typical application involves running PointNN Selector on a dataset to produce a sparse, robust, and representative set of exemplars, with size and approximation guarantees, to accelerate nearest-neighbor queries or to serve as condensed training sets for resource-constrained deployments.

6. Comparison with FCNN and Related Approaches

	FCNN	PointNN Selector
Size bound	None (unbounded)	$p\in P$6
Approximation to Min-CS	Heuristic only	$p\in P$7 factor
Runtime	$p\in P$8	$p\in P$9
Consistency on $\ell(nn_S(p)) = \ell(p)$0	Always if full	Always if $\ell(nn_S(p)) = \ell(p)$1

FCNN demonstrates no non-trivial worst-case size bound, while PointNN Selector achieves a provable packing bound and constant-factor approximation for the NP-hard Min-CS problem. Both share similar asymptotic runtimes.

7. Interpretive Notes and Implications

The introduction of a separation constraint enables PointNN Selector to be robust to pathological input configurations and prevents over-representation of localized high-density regions. This suggests the method is suited to high-dimensional, potentially low-margin datasets where classic condensation algorithms fail by redundancy or overselection.

A plausible implication is that PointNN Selector is broadly applicable as a core method for prototype selection in metric learning, geometric data condensation, and for accelerating the inference speed of nearest-neighbor-based classifiers, while maintaining formal error and size guarantees. Its parameters expose an explicit trade-off between sparsity and fidelity, controlled via the separation $\ell(nn_S(p)) = \ell(p)$2. The algorithm’s performance is determined by the underlying geometry (doubling dimension) and the labeling complexity (through $\ell(nn_S(p)) = \ell(p)$3).

References: The main definition, results, and algorithm are presented in "Social Distancing is Good for Points too!" (Flores-Velazco, 2020).