--- title: Sketch-Based LSH for Dynamic Sets url: https://www.emergentmind.com/topics/sketch-based-lsh-for-dynamic-sets type: topic --- # Sketch-Based LSH for Dynamic Sets Sketch-based Locality Sensitive Hashing (LSH) for dynamic sets addresses the problem of efficient similarity search and exact nearest-neighbor retrieval in large-scale datasets where both insertions and deletions of elements (items, features) are permitted. This paradigm synthesizes advances in online sketching algorithms, compressed sensing, and the design of dynamic LSH functions, enabling sublinear-memory data structures for the near neighbor problem and robust similarity search in streaming and fully dynamic models [1902.06687, 1605.03949]. ## 1. Problem Formulation and Dynamic Setting Similarity search in set-based or vector datasets frequently leverages LSH, particularly for measures such as Jaccard similarity. The canonical setting considers a dataset $\mathcal{D} = \{x_1,\dots,x_N\} \subset \mathbb{R}^d$ or subsets $A,B \subset U$ under a one-pass data stream or fully dynamic stream, supporting both insertions and deletions. The objective is to construct a compact sketch $\mathcal{S}$ of the dataset such that, given a query $q$, one can efficiently estimate set similarities or report the exact $v$ nearest neighbors with high probability, while using $o(N)$ memory [1902.06687, 1605.03949]. In the context of dynamic streams, updates arrive as tuples (i, j, δ), denoting an increment or decrement of the j-th coordinate for the i-th user or set. The fundamental challenge is to maintain sketch structures under arbitrary additions and deletions, a requirement not satisfied by classical min-hash or LSH schemes designed for insertion-only streams [1605.03949]. ## 2. Sketch Structures for Approximate Similarity in Dynamic Streams Rational set similarities, including Jaccard, admit linear sketch representations under the dynamic model. For each user $i$, two linear $\ell_0$ sketches are maintained: one for the union and one for the symmetric difference (i.e., $|A^i \cup A^j| = \ell_0(a^i + a^j)$ and $|A^i \Delta A^j| = \ell_0(a^i - a^j)$, where $a^i \in \{0,1\}^{|U|}$). These sketches are updated in $O(1)$ time per operation and achieve a $(1\pm\epsilon)$ multiplicative approximation to the rational set similarity or its associated metric distance [1605.03949]. For sublinear memory sketches with nearest-neighbor support, the dataset is compressed to size $O(N^b \log^3 N)$ bits for $b<1$, determined by query-specific stability parameters. The sketch consists of short integer arrays updated in a streaming fashion and allows for the recovery of nearest neighbors in queries satisfying stability conditions [1902.06687]. ## 3. Dynamic LSH Constructions The main technical contribution in dynamic LSH is the ability to process deletions without scanning the support of the underlying set. This is achieved via a bucketed bit-sampling approach: - Each coordinate $j$ is sampled independently at random granularity (level $k = \mathsf{lsb}(h(j))$ for a hash $h$), creating multi-level fingerprints. - For each set, a hash table $T^i_{k, \cdot}$ of size $c^2$ is maintained per level, counting how many sampled coordinates hit each bucket. Insertions and deletions update these counters accordingly. - The LSH fingerprint is defined as the index of the first nonzero bucket ("min-bucket") at each level. - Candidate generation exploits multi-level hashes and min-bucket fingerprints, analogous to Indyk–Motwani LSH for other metrics [1605.03949]. The collision probability between two sets at level $k$ reflects the (sampled) similarity $S(A^i_k,A^j_k)$, which closely tracks $S(A^i,A^j)$ for well-chosen parameters. Amplification across bands and levels reduces error and concentrates sensitivity. Table 1 summarizes the key aspects of the dynamic LSH construction. | Structure | Update Time | Space Complexity | |-----------------------|-------------|------------------------------------------| | Dynamic LSH (bucketed)| $O(1)$ | $O(H c^2 \log|U|)$ per user | | $\ell_0$-sketches | $O(1)$ | $O(\epsilon^{-2} \log|U| \log(1/\delta))$| ## 4. Sublinear-Memory Sketching via LSH and Compressed Sensing The RACE (Repeated ACE) estimator enables unbiased estimation of LSH kernel sums in the streaming model. With a family $\mathcal{H}$ of LSH functions and amplification parameter $K$, each input $x$ is mapped to $L(x) = (h_1(x),...,h_K(x))$ with collision probability $p(x,q)^K$. R independent arrays count occurrences, and at query time, $\widehat{\mathrm{KDE}}(q) = \frac{1}{R} \sum_{o=1}^R A_o[L(q)]$ estimates kernel densities robustly [1902.06687]. To recover the set of top $v$ nearest neighbors, the target vector $\mathbf{s}(q) = [p(x_1,q)^K,\ldots,p(x_N,q)^K]^\top$ is nearly $v$-sparse and can be sketched further via a Count-Min-Sketch measurement $\mathbf{y} = \Phi\,\mathbf{s}(q)$. Median-of-means estimators per row give pointwise estimates, with guarantees on ordering between the $v$-th and $(v+1)$-th kernels. These techniques allow sublinear-memory retrieval for queries exhibiting a sufficient collision gap $\Delta = p(x_{v+1},q)/p(x_v,q)$. The sketch update per point is $O(K)$; query time is $O(N^{b+1} \log^3 N)$ for $b<1$, and space requirements are $O(N^b \log^3 N)$. ## 5. Theoretical Guarantees and Performance Analysis For nearest-neighbor retrieval, let $p_v = p(x_v,q)$ denote the $v$-th largest LSH collision probability for query $q$, and $\Delta = p_{v+1}/p_v$ the collision gap. Choosing $K = \lceil 2 \ln N/\ln(1/\Delta) \rceil$, the memory exponent $$b = \frac{6|\ln p_v| + 2\ln r}{\ln(1/\Delta)}$$ ensures sublinear space when $b<1$ [1902.06687]. The algorithm returns the true nearest neighbors of $q$ with probability at least $1-\delta$ and provides additive error $|\hat s_i(q) - p(x_i,q)^K| \leq \epsilon/2$ (for $\epsilon = p_v^K - p_{v+1}^K$). Experiments on social network graphs (Google Plus, Twitter, Slashdot; each $\sim 10^5$ nodes) demonstrate that the RACE–CMS sketch reaches 80% recall for top-20 Jaccard neighbors at $\geq 0.9$ similarity with only $\approx 5\%$ of the original memory, outperforming random projections ($\sim 50\%$). Map-based sparse storage further reduces the RACE array footprint by up to $5\times$ [1902.06687]. ## 6. Distinctions from Insertion-only Min-Hash and Related Methodologies Classical min-hashing is not directly updatable in the dynamic setting because deletion of the minimum element requires a full rescan to determine the next smallest value. The bucketed bit-sampling approach provides dynamic support by maintaining per-level counters for all sampled coordinates and reconstructing min-buckets after arbitrary deletions in $O(c^2)$ (or amortized $O(1)$) time [1605.03949]. Sketch-based LSH for dynamic sets further differs from traditional LSH in its integration of linear sketching (e.g., $\ell_0$ sketches) for rational set similarities and in its explicit use of compressed-sensing recovery for sparse LSH kernel signals, allowing for improved space–accuracy tradeoffs in streaming and near neighbor search. ## 7. Applications, Limitations, and Implications Sketch-based LSH for dynamic sets is applicable in large-scale recommendation systems, social network friend suggestion, and streaming analytics where both additions and deletions must be handled without explicit data reconstruction. The approach supports exact $v$-nearest-neighbor recovery under “stable” queries for which the collision gap $\Delta$ is sufficiently separated from 1; when most data lie within a $(1+\epsilon)$-distance shell, sublinear memory is not attainable. A plausible implication is that, for real-world graphs and set systems exhibiting natural stability, these methods provide orders-of-magnitude improvement in compression and update/query efficiency, while the integration of dynamic LSH supports robust operation in continuously updating environments. References: [1902.06687], [1605.03949] Source: https://www.emergentmind.com/topics/sketch-based-lsh-for-dynamic-sets