Hyperplane-Based Locality-Sensitive Hashing (LSH)

Hyperplane-Based Locality-Sensitive Hashing (LSH) is a foundational method for approximate similarity search in high-dimensional spaces. It operates by partitioning the space with hyperplanes, converting continuous similarity relations (such as distances or inner products) into discrete hash codes that preserve locality. Over the past decade, this approach has served as the starting point for numerous theoretical refinements, adaptions to data geometry, distributed system optimizations, learning-based improvements, and extensions to broader problem domains. The primary goal is to achieve sublinear-time retrieval with high recall, and practical schemes must balance statistical fidelity, computational efficiency, memory, and distributional support.

1. Mathematical Foundations and Standard Construction

Hyperplane-based LSH typically targets the Euclidean ($l_2$) or cosine similarity metrics. A data point $x \in \mathbb{R}^d$ is transformed by a hash function that encodes its position relative to a set of $m$ random hyperplanes. For each hyperplane with normalized normal vector $\vec{a}_i$, one records:

$$h_i(x) = \begin{cases} 1 & \vec{a}_i \cdot x + b_i > 0 \ 0 & \text{otherwise} \end{cases}$$

where $b_i$ is often either $0$ (for classic sign-LSH) or uniformly random in $[0,W]$ if quantization is desired. The hash code for $x$ is $(h_1(x), ..., h_m(x))$, interpreted as a binary string or a multi-dimensional integer for bucket assignment. The collision probability between vectors is monotonic in their angular distance (cosine similarity) [Charikar, 2002].

For Euclidean ($l_2$) LSH [Datar et al.], a common form is:

$$h_{\vec{a},b}(x) = \left\lfloor \frac{\vec{a} \cdot x + b}{w} \right\rfloor$$

where $\vec{a} \sim \mathcal{N}(0, I)$ and $b \sim \mathrm{Uniform}(0, w)$.

2. Network-Efficient and Distributed Schemes

Scaling LSH to distributed and large-scale clusters leads to nontrivial challenges. The Layered LSH approach (Bahmani et al., 2012 ) builds upon Entropy LSH, which reduces the number of hash tables (and associated space) by allowing multiple probe offsets per query. However, with naive distribution, these offsets dramatically increase network calls. Layered LSH addresses this by adding a second LSH layer on the hash codes:

$$G(H(x)) = \left\lfloor \frac{\boldsymbol{\alpha} \cdot H(x) + \beta}{D} \right\rfloor$$

This second layer clusters buckets likely to be hit together by a given query and its offsets onto the same server, exponentially reducing the expected number of network calls per query to $O(\sqrt{\log n})$. Load balance is mathematically analyzed and experimentally verified. The scheme maintains search quality while substantially decreasing network traffic, achieving order-of-magnitude reductions in wall-clock runtime for high-dimensional text and image datasets.

3. Advancements in Hyperplane Arrangements and Space Partitioning

Standard hyperplane LSH partitions the space using hyperplanes through the origin, limiting the number of representable regions. Introducing offset hyperplanes via a lift map (Konoshima et al., 2012 )—embedding each $x \in \mathbb{R}^N$ as $(x, 1) \in \mathbb{R}^{N+1}$—enables learning hash functions equivalent to general hyperplanes (not just those through the origin) without algorithmic change to learning. This substantially increases the partitioning granularity, improves the correlation between Hamming and Euclidean distances, and is especially beneficial when the number of hash bits far exceeds the intrinsic data dimension and the data is distributed spherically.

$$f(x) = (x_1, ..., x_N, 1) ~;~ (\vec{n}, w) \cdot (x, 1) = 0 ~\Leftrightarrow~ \vec{n} \cdot x + w = 0$$

Empirical analysis demonstrates the lift's benefit is data-dependent, requiring careful consideration of principal component scales and intended hash code length.

4. Supervised Hyperplane Arrangement and Learning

While original LSH schemes use random hyperplanes, supervised learning can improve hash arrangements—ensuring that Hamming space distances correlate with semantic or label similarity. One approach (Noma et al., 2013 ) learns each hyperplane's orientation via Markov Chain Monte Carlo (MCMC) sampling, maximizing an evaluation function defined on positive (same-label) and negative (different-label) data pairs. The process samples directions on the sphere, with temperature and multimodality controlling the diversity of learned hyperplanes. Key evaluation functions include counts of well-separated pairs and proportions of labels. Sampling strategies for positive and negative pairs (random, near, boundary, far) significantly affect the learned hash quality. Supervised MCMC-based LSH typically outperforms not only random LSH but also margin-based schemes, particularly when data labels require nontrivial boundary orientations.

5. Adaptations for Scalability, Efficiency, and Variant Metrics

Recent work emphasizes the need for both high-dimensional scalability and practical efficiency:

• Layered and Network-Efficient LSH: Results demonstrate significant reduction in distributed network shuffle and runtime, with theoretical and practical guarantees on load balancing (Bahmani et al., 2012 ).
• Hash Computation Acceleration: FastLSH (Tan et al., 2023 ) and Count Sketch-based schemes (Verma et al., 9 Mar 2025 ) reduce hash computation from $O(md)$ to $O(m)$ or $O(d)$, where $d$ is dimensionality and $m$ is code length, by random sampling or sketching. Higher-order sketch variants decrease space further.
• Multipurpose LSH (mp-LSH): Codes are shared across L2, cosine, and inner product metrics, and can be dynamically reweighted at query time without reindexing (Pronobis et al., 2016 ). The CAT variant efficiently encodes all information needed for multi-metric search, preserving accuracy and providing memory efficiency.
• Extension to Function Spaces: Hyperplane-based LSH is naturally extended to $L^p$ and Wasserstein spaces by projecting onto orthonormal bases or using Monte Carlo discretization, enabling similarity search among functions and distributions (Shand et al., 2020 ).

6. Domain-Specific and Advanced Applications

• Polygon and Shape Retrieval: Hyperplane-LSH principles are adapted to function-based "turning functions" of polygons for efficient shape similarity search (Kaplan et al., 2021 ), with careful treatment of invariances and tight approximation bounds.
• Mixture-of-Experts (MoE) Training: In distributed training of large MoE networks, communication bottlenecks are reduced by clustering similar tokens using cross-polytope LSH before all-to-all operations, transmitting only centroids and compensating with per-token residuals. This yields substantial speedups with negligible loss in model quality (Nie et al., 13 Nov 2024 ).

7. Performance Metrics, Empirical Results, and Limitations

Empirical studies consistently report:

• Substantial reductions in time and communication overhead, e.g., Layered LSH achieving 10–100x less network shuffle and up to 3x faster end-to-end runtime.
• In learned and supervised settings, improved recall and precision over random LSH for semantic similarity, and robustness across different data types.
• FastLSH and count-sketch methods yield up to 80x faster hash computations with comparable recall to classic E2LSH, enabling practical deployment at scale.
• Multipurpose LSH (mp-LSH-CAT) attains near-optimal accuracy for all metric mixes with a single efficient code base.
• For distributed frameworks and MoE training, LSH-based compression can reduce communication by 80–90% with negligible accuracy impact.

Known limitations include sensitivity to the statistical geometry of the data (especially for lifting and learned hyperplanes), increased space or memory for certain bucket or sketching strategies, and practical selection of parameters such as hash length, number of layers, and network thresholds. Theoretical guarantees for learned/neural LSH are typically inherited from the code family being imitated; further theoretical advances are areas of active research.

Key Mathematical Summary

Hyperplane-based LSH and its major extensions provide a principled, versatile framework for sublinear approximate similarity search, scalable distribution, supervised adaptation, and flexible metric support, with proven theoretical and empirical effectiveness across high-dimensional domains.