Bottom-k Hashing Techniques

• Bottom-k hashing is a probabilistic data summarization technique that retains the k smallest hash values from a set to enable unbiased estimation of similarity and frequency.
• It produces mergeable sketches with strong theoretical guarantees, efficiently handling set unions, overlaps, and frequency estimation even under limited hash-function independence.
• Widely applied in scalable similarity search, linear learning, and pattern mining, bottom-k hashing offers significant speedups and computational benefits over traditional minwise methods.

Bottom-k hashing (also referred to as bottom-k sampling) is a probabilistic data summarization technique that produces compact, mergeable sketches of sets, enabling scalable approximation of set cardinalities, overlaps, and similarities. By maintaining the k elements with the lowest hash values (or permuted indices) from a set under a suitably chosen hash function or permutation, bottom-k hashing provides unbiased or nearly-unbiased estimators for Jaccard similarity, subset frequencies, and set unions. Its composability, strong theoretical guarantees (even for limited hash-function independence), and computational efficiency have made it fundamental in scalable learning, pattern mining, and similarity search.

1. Formal Definitions and Construction

Let $X$ be a universe of $n$ items and $h : X \to (0,1)$ a hash function. The bottom-$k$ sample of $X$ is

$$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$

where $h_{(k)}$ is the $k$th smallest hash value among $\{h(x)\,:\,x\in X\}$ (Thorup, 2013). Thus, $S_k(X)$ consists of those $n$0 items in $n$1 with the lowest hashes, forming the "bottom-k sketch."

In the context of high-dimensional binary data, the one-permutation bottom-k scheme permutes the feature universe once using a random permutation $n$2. The permuted universe is partitioned into $n$3 contiguous blocks ("bins"). For each bin $n$4, the sketch records the smallest permuted index from the set $n$5 (the nonzero indices of a binary vector) that lands in bin $n$6 (Li et al., 2012).

Merge Operations

One key property is that sketches are composable:

$n$7

i.e., to obtain the sketch of a union, simply merge sketches of each set and extract the bottom $n$8 elements (Thorup, 2013).

2. Algorithms and Implementation

Heap-based bottom-k: Stream through elements of $n$9, maintaining a size-$h : X \to (0,1)$0 max-heap of the smallest $h : X \to (0,1)$1 hashes. Insert new items only if their hashes are among the $h : X \to (0,1)$2 lowest so far. Per-element computational cost is $h : X \to (0,1)$3 (Thorup, 2013).

One-permutation binning: For large, sparse binary matrices, a single permutation of the entire universe reduces the overall permuting cost by a factor of $h : X \to (0,1)$4. For each data vector $h : X \to (0,1)$5, iterate through nonzeros, assign to bins by permuted index, and keep the minimal value per bin. The pseudocode is as follows (Li et al., 2012):

$h_{(k)}$7

Handling empty bins: Either encode empty bins as zeros ("zero-coding"), or fill them with random values from the bin range ("random-coding"). Zero-coding is preferred as it preserves sparsity and matches well with linear learning pipelines (Li et al., 2012).

3. Statistical Properties: Unbiasedness and Concentration

Unbiasedness

Given sets $h : X \to (0,1)$6 and $h : X \to (0,1)$7, let $h : X \to (0,1)$8, $h : X \to (0,1)$9, and $k$0 the Jaccard similarity. For the one-permutation bottom-k sketch, define $k$1 as the number of bins for which both sketches are non-empty and match, and $k$2 as the number of empty bins. The estimator

$k$3

is unbiased: $k$4 (Li et al., 2012).

Variance and Error Bounds

• For classic bottom-k, the variance of subset frequency estimates is $k$5, even if $k$6 is only 2-independent (Thorup, 2013).
• For the one-permutation scheme, variance is at most $k$7, often slightly lower due to the sample-without-replacement effect.

4. Comparison with k-Permutation Minwise Hashing

Classic minwise hashing requires $k$8 independent hash functions (or permutations), significantly increasing preprocessing computational cost to $k$9, where $X$0 is the average number of nonzeros per vector. The one-permutation bottom-k scheme reduces this to $X$1, yielding a $X$2-fold reduction in preprocessing (Li et al., 2012).

Bias under Limited Independence

For $X$3-minwise, limited-independence hash families yield constant estimator bias that cannot be eliminated by increasing $X$4. In contrast, bottom-k hashing (even with only pairwise-independent hash functions) achieves the same expected error bounds as fully-random hash families, as demonstrated via union-bound arguments (Thorup, 2013).

Method	Preprocessing Cost	Independence Required	Bias (limited independence)
Bottom-k	$X$5	2-independent	Vanishing for any $X$6
k×minwise	$X$7	$X$8-independent	Constant, does not vanish

5. Applications

Set Similarity and Subset Estimation

Bottom-k sketches support unbiased estimation of Jaccard similarity and subset frequencies for potentially massive or distributed datasets, with efficient, scalable merging of sketches (Thorup, 2013).

Linear Learning with Hashed Features

In large-scale SVM and logistic regression tasks (e.g., webspam with $X$9M, avg. 4K nonzeros/doc), one-permutation bottom-k schemes with $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$0 or $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$1 and $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$2-bit feature compression achieve test accuracies matching or slightly exceeding the classical minwise baseline, at a tiny fraction of the preprocessing cost (Li et al., 2012).

Pattern Set Mining

For pattern set selection under reconstruction error objectives (e.g., in Boolean matrix factorization and database tiling), bottom-k sketches are used to estimate coverage and marginal gains rapidly. The HaPSi algorithm, for instance, uses precomputed bottom-k sketches per candidate pattern and merges them to estimate the coverage union size in $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$3 time. This leads to 50 to 100-fold speedups over greedy search, with reconstruction errors within 0.5–2% of those of the exact greedy method on real and synthetic datasets (Karjalainen et al., 11 Jul 2025).

Weighted Set Sums: Priority Sampling

Bottom-k techniques extend to weighted set estimation via priority sampling, where items are sampled according to priority scores $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$4 (for weights $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$5 and uniform $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$6). This enables unbiased and concentration-guaranteed estimation of weighted subset sums, adapting optimal variance properties for heavy-tailed distributions (Thorup, 2013).

6. Practical Considerations and Parameter Choices

• Typical sketch size $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$7 balances estimator variance and computational cost; choose $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$8 so that $$S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},$$9 for most sets to avoid empty bins (Li et al., 2012).
• For $h_{(k)}$0-bit compression, store only $h_{(k)}$1 bits per sketch entry for compactness.
• In scenarios with extreme data sparsity or high $h_{(k)}$2, using a small number $h_{(k)}$3 of permutations with $h_{(k)}$4 bins per permutation can further control the empty-bin probability (Li et al., 2012).
• In applications where patterns can cover “zeros” (approximate matching), correction terms are used to adjust estimated marginal gains (subtracting the number of newly covered zeros) (Karjalainen et al., 11 Jul 2025).

7. Empirical Findings and Impact

Experiments on public datasets demonstrate that one-permutation bottom-k hashing achieves equivalent (or better) accuracy compared to classical minwise hashing, with drastically lower computational cost (Li et al., 2012). In highly sparse settings such as the news20 corpus (avg. 500 nonzeros/document, $h_{(k)}$5M), bottom-k schemes show clear empirical superiority for large $h_{(k)}$6 due to reduced estimator variance.

In pattern set mining, bottom-k hashing dramatically accelerates greedy selection procedures, making previously intractable scales feasible with minimal loss in output quality (Karjalainen et al., 11 Jul 2025). In sum, the mergeable, efficient, and robust properties of bottom-k hashing render it a core primitive in large-scale, distributed, and streaming settings for both set and associated weighted estimations.