Threshold-Based Cosine Search

• Threshold-Based Cosine Search is a similarity search paradigm that identifies all database vectors with cosine similarity above a given threshold, ensuring exact matching.
• It leverages geometric properties such as supermetricity and Hilbert exclusion to efficiently prune search spaces, reducing computation compared to exhaustive scans.
• Practical frameworks including binary hyperplane trees, convex hull–based indexing, and adaptive group testing enable scalable performance and are applied in document, image retrieval, and plagiarism detection.

Threshold-Based Cosine Search is an exact similarity search paradigm that identifies all database vectors with cosine similarity to a query vector exceeding a pre-specified threshold. This approach is fundamental in retrieval, classification, and large-scale matching tasks, especially under high dimensionality and sparsity common in modern applications such as document retrieval, image search, and plagiarism detection. The key technical challenge is avoiding exhaustive scan while preserving precision, requiring both theoretical insight into the geometric structure of cosine similarity and efficient algorithmic frameworks.

1. Mathematical Foundation and Cosine-Derived Metrics

Threshold-based cosine search begins by formalizing the query condition: given database vectors $S = \{s_1,\dots,s_N\}$, each in $\mathbb{R}^d$, and a query $q\in\mathbb{R}^d$ (typically $\|q\|_2=1$), one returns all $s$ satisfying $q\cdot s \geq \tau$ for some threshold $\tau\in(0,1]$. For nonzero vectors, cosine similarity is $\mathrm{CosSim}(u,v)={u\cdot v}/{\|u\|\|v\|}$. The transformation $d(u,v)=\sqrt{1-\mathrm{CosSim}(u,v)}$ induces a metric, specifically a scaled $\ell_2$ distance on the unit sphere, making the search space a supermetric with the four-point (Hilbert) property (Connor et al., 2017). This property is crucial: it allows geometric exclusion rules significantly stronger than those available under mere triangle inequality, supporting highly effective pruning and index structures.

2. Algorithmic Index Structures for Threshold Cosine Search

Multiple complementary frameworks enable efficient threshold queries under cosine similarity:

• Supermetric Binary Hyperplane Trees: Internal nodes store two far-apart pivots $\mathbb{R}^d$0; data is partitioned by comparing distances to pivots using the metric induced above. The key exclusion rule is the Hilbert exclusion: for query $\mathbb{R}^d$1 and distance threshold $\mathbb{R}^d$2,

$\mathbb{R}^d$3

if $\mathbb{R}^d$4, exclude all points in the side closer to $\mathbb{R}^d$5, and symmetrically for $\mathbb{R}^d$6 (Connor et al., 2017). This condition enables sharper pruning compared to classical hyperbolic (triangle-inequality) exclusion.

• Convex Hull–Based Inverted Index Traversal: For sparse or high-dimensional data, a TA-style inverted index is used, with dimensional lists sorted by component values. Rather than naïve lockstep traversal, a tight, unit-vector-constrained KKT-based stopping criterion is employed, and steps advance on the steepest monotone descent in dot-product score, guided by per-list convex hulls. If lists have bounded hull gaps $\mathbb{R}^d$7, the traversal guarantees at most $\mathbb{R}^d$8 access cost, where $\mathbb{R}^d$9 is the best possible (Li et al., 2018).
• Binary Splitting Adaptive Group Testing: Databases are recursively partitioned into pools, each characterized by the sum (or max) of member vectors. If the dot-product of a pool with $q\in\mathbb{R}^d$0 is below $q\in\mathbb{R}^d$1, the entire pool is pruned. Otherwise, the pool splits and the process recurses. Using precomputed cumulative sums, the per-query cost is typically an order of magnitude below exhaustive scan, given the empirical distributional skew of dot-products (e.g., truncated exponential) (Shah et al., 2023).

The following table summarizes core structural properties:

Method	Geometry Used	Exclusion Type
Binary hyperplane tree	Supermetric (Hilbert)	Four-point (Hilbert)
Inverted index/hull traversal	Unit sphere, convex hull	Tight stop via KKT, convex gap
Group testing	Additive pooling	Pool-level dot-product bound

3. Triangle Inequality and Exclusion Bounds for Cosine Similarity

Cosine similarity is not a true metric (it lacks global triangle inequality). However, recent work establishes tight analogues suitable for search pruning:

Given $q\in\mathbb{R}^d$2 (unit vectors), define $q\in\mathbb{R}^d$3, $q\in\mathbb{R}^d$4. Then

$q\in\mathbb{R}^d$5

and

$q\in\mathbb{R}^d$6

(Schubert, 2021). This enables tight subtree exclusion in vantage-point and metric-tree-like indexes: for a pivot $q\in\mathbb{R}^d$7, if the upper bound on $q\in\mathbb{R}^d$8 realized within a child subtree falls below $q\in\mathbb{R}^d$9, the subtree is pruned. Empirically, this arccos-based bound improves pruning compared to additive or chord-based heuristics by $\|q\|_2=1$027% on average, greatly reducing the number of examined nodes in practice.

4. Algorithmic Complexity, Scalability, and Optimality Guarantees

Empirical and theoretical analyses confirm significant speedups and scalability:

• Supermetric Trees: Index construction is $\|q\|_2=1$1. At query time, if little pruning occurs, cost is $\|q\|_2=1$2; otherwise, observed reductions in distance computations are $\|q\|_2=1$3–$\|q\|_2=1$4 compared to hyperbolic exclusion, and even higher on some datasets (Connor et al., 2017).
• Hull-Based TA Enhancement: Query time is reduced by $\|q\|_2=1$5–$\|q\|_2=1$6 over classical TA. The candidate set size can be pruned by $\|q\|_2=1$7–$\|q\|_2=1$8 more using unit-vector-specific tests. The hull-based pointer-advancement is provably near-optimal, with total list accesses at most $\|q\|_2=1$9, where $s$0 empirically falls between $s$1 and $s$2 (Li et al., 2018).
• Adaptive Group Testing: For databases with millions of items and strongly right-skewed softmax features (typical $s$3 for a truncated exponential fit), only a small fraction of pools survive early pruning. Per-query dot-product cost reduces $s$4–$s$5 compared to exhaustive scan. The algorithm guarantees 100% recall and precision for all thresholds, without parameter tuning (Shah et al., 2023).

5. Practical Applications and Domain Adaptation

Threshold-based cosine search finds applications in document and image retrieval, plagiarism detection, and mass spectrometry. For image-plagiarism detection using deep net softmax features, group testing reduces per-query times by an order of magnitude, preserving full recall. For spectral databases and word embeddings, hull-based index traversal supports high-throughput, exact semantic search. The frameworks extend to any monotonic, decomposable similarity with known per-vector constraints (e.g., Jensen-Shannon divergence, certain $s$6 kernels), providing versatility beyond cosine (Li et al., 2018).

6. Data Distribution, Streaming, and Memory Considerations

Empirical dot-product distributions for modern embeddings are sharply right-skewed (truncated exponential), resulting in reliable early pruning for high thresholds. Group testing requires only $s$7 storage and supports $s$8 amortized insertion per new vector for streaming updates, as cumulative sums can be extended incrementally. No extra hyperparameters, hashing, or bucket tuning is needed for group testing. For extremely high dimensions, random projection may be used to reduce $s$9, but the threshold must be re-tuned according to the new distribution.

7. Summary and Comparative Insights

Threshold-based cosine search integrates metric-space insights (supermetricity, four-point property), index-aware branching and exclusion (Hilbert, arccos bounds), and data distribution modeling to deliver exact, high-performance neighbor queries. Across paradigms, the core advances are: formulation of tighter geometric exclusion bounds, efficient index traversal algorithms leveraging data sparsity and convex-dominated decay, and practical adaptation for streaming and very large-scale deployment. These methods collectively represent the state-of-the-art for exact thresholded neighbor search under cosine similarity in high-dimensional regimes (Connor et al., 2017, Li et al., 2018, Shah et al., 2023, Schubert, 2021).