---
title: Bottom-k Hashing Techniques
url: https://www.emergentmind.com/topics/bottom-k-hashing
type: topic
---

# Bottom-k Hashing Techniques

Bottom-k hashing (also referred to as bottom-k sampling) is a probabilistic data summarization technique that produces compact, mergeable sketches of sets, enabling scalable approximation of set cardinalities, overlaps, and similarities. By maintaining the k elements with the lowest hash values (or permuted indices) from a set under a suitably chosen hash function or permutation, bottom-k hashing provides unbiased or nearly-unbiased estimators for Jaccard similarity, subset frequencies, and set unions. Its composability, strong theoretical guarantees (even for limited hash-function independence), and computational efficiency have made it fundamental in scalable learning, pattern mining, and similarity search.

## 1. Formal Definitions and Construction

Let $X$ be a universe of $n$ items and $h : X \to (0,1)$ a hash function. The bottom-$k$ sample of $X$ is
$$
S_k(X) = \left\{\, x \in X \;\mid\; h(x) \leq h_{(k)} \right\},
$$
where $h_{(k)}$ is the $k$th smallest hash value among $\{h(x)\,:\,x\in X\}$ [1303.5479]. Thus, $S_k(X)$ consists of those $k$ items in $X$ with the lowest hashes, forming the "bottom-k sketch."

In the context of high-dimensional binary data, the *one-permutation* bottom-k scheme permutes the feature universe once using a random permutation $\pi: \Omega \to \Omega$. The permuted universe is partitioned into $k$ contiguous blocks ("bins"). For each bin $j=1,\ldots,k$, the sketch records the smallest permuted index from the set $S$ (the nonzero indices of a binary vector) that lands in bin $j$ [1208.1259].

### Merge Operations

One key property is that sketches are composable:
$$
S_k(A \cup B) = S_k(S_k(A) \cup S_k(B)),
$$
i.e., to obtain the sketch of a union, simply merge sketches of each set and extract the bottom $k$ elements [1303.5479].

## 2. Algorithms and Implementation

**Heap-based bottom-k:** Stream through elements of $X$, maintaining a size-$k$ max-heap of the smallest $k$ hashes. Insert new items only if their hashes are among the $k$ lowest so far. Per-element computational cost is $O(\log k)$ [1303.5479].

**One-permutation binning:** For large, sparse binary matrices, a single permutation of the entire universe reduces the overall permuting cost by a factor of $k$. For each data vector $S$, iterate through nonzeros, assign to bins by permuted index, and keep the minimal value per bin. The pseudocode is as follows [1208.1259]:

```python
# Inputs: S - set of indices, k - number of bins, pi - permutation (array)
h = [EMPTY] * k
for x in S:
    y = pi[x]
    j = int(y * k / D)
    if h[j] == EMPTY or y < h[j]:
        h[j] = y
return h
```

**Handling empty bins:** Either encode empty bins as zeros ("zero-coding"), or fill them with random values from the bin range ("random-coding"). Zero-coding is preferred as it preserves sparsity and matches well with linear learning pipelines [1208.1259].

## 3. Statistical Properties: Unbiasedness and Concentration

### Unbiasedness

Given sets $S_1$ and $S_2$, let $a = |S_1 \cap S_2|$, $f = |S_1 \cup S_2|$, and $R = a / f$ the Jaccard similarity. For the one-permutation bottom-k sketch, define $N_{mat}$ as the number of bins for which both sketches are non-empty and match, and $N_{emp}$ as the number of empty bins. The estimator
$$
\hat R = \frac{N_{mat}}{k - N_{emp}}
$$
is unbiased: $E[\hat R] = R$ [1208.1259].

### Variance and Error Bounds

- For classic bottom-k, the variance of subset frequency estimates is $O(1/(\sqrt{fk}))$, even if $h$ is only 2-independent [1303.5479].
- For the one-permutation scheme, variance is at most $R(1-R)/k$, often slightly lower due to the sample-without-replacement effect.

## 4. Comparison with k-Permutation Minwise Hashing

Classic minwise hashing requires $k$ independent hash functions (or permutations), significantly increasing preprocessing computational cost to $O(k \cdot n \cdot nz)$, where $nz$ is the average number of nonzeros per vector. The one-permutation bottom-k scheme reduces this to $O(n \cdot nz) + O(D)$, yielding a $1/k$-fold reduction in preprocessing [1208.1259].

### Bias under Limited Independence

For $k$-minwise, limited-independence hash families yield constant estimator bias that cannot be eliminated by increasing $k$. In contrast, bottom-k hashing (even with only pairwise-independent hash functions) achieves the same expected error bounds as fully-random hash families, as demonstrated via union-bound arguments [1303.5479].

| Method                   | Preprocessing Cost        | Independence Required | Bias (limited independence)  |
|--------------------------|--------------------------|----------------------|-----------------------------|
| Bottom-k                 | $O(n\,nz) + O(D)$        | 2-independent        | Vanishing for any $k$       |
| k×minwise                | $O(k\,n\,nz)$            | $k$-independent      | Constant, does not vanish   |

## 5. Applications

### Set Similarity and Subset Estimation

Bottom-k sketches support unbiased estimation of Jaccard similarity and subset frequencies for potentially massive or distributed datasets, with efficient, scalable merging of sketches [1303.5479].

### Linear Learning with Hashed Features

In large-scale SVM and logistic regression tasks (e.g., webspam with $D\approx 16$M, avg. 4K nonzeros/doc), one-permutation bottom-k schemes with $k=256$ or $512$ and $b$-bit feature compression achieve test accuracies matching or slightly exceeding the classical minwise baseline, at a tiny fraction of the preprocessing cost [1208.1259].

### Pattern Set Mining

For pattern set selection under reconstruction error objectives (e.g., in Boolean matrix factorization and database tiling), bottom-k sketches are used to estimate coverage and marginal gains rapidly. The HaPSi algorithm, for instance, uses precomputed bottom-k sketches per candidate pattern and merges them to estimate the coverage union size in $O(k\,H)$ time. This leads to 50 to 100-fold speedups over greedy search, with reconstruction errors within 0.5–2% of those of the exact greedy method on real and synthetic datasets [2507.08745].

### Weighted Set Sums: Priority Sampling

Bottom-k techniques extend to weighted set estimation via *priority sampling*, where items are sampled according to priority scores $q_i = w_i / h_i$ (for weights $w_i$ and uniform $h_i$). This enables unbiased and concentration-guaranteed estimation of weighted subset sums, adapting optimal variance properties for heavy-tailed distributions [1303.5479].

## 6. Practical Considerations and Parameter Choices

- Typical sketch size $k\in[200,500]$ balances estimator variance and computational cost; choose $k$ so that $f/k \gtrsim 5$ for most sets to avoid empty bins [1208.1259].
- For $b$-bit compression, store only $b$ bits per sketch entry for compactness.
- In scenarios with extreme data sparsity or high $k$, using a small number $m>1$ of permutations with $k' = k/m$ bins per permutation can further control the empty-bin probability [1208.1259].
- In applications where patterns can cover “zeros” (approximate matching), correction terms are used to adjust estimated marginal gains (subtracting the number of newly covered zeros) [2507.08745].

## 7. Empirical Findings and Impact

Experiments on public datasets demonstrate that one-permutation bottom-k hashing achieves equivalent (or better) accuracy compared to classical minwise hashing, with drastically lower computational cost [1208.1259]. In highly sparse settings such as the news20 corpus (avg. 500 nonzeros/document, $D \approx 1.35$M), bottom-k schemes show clear empirical superiority for large $k$ due to reduced estimator variance.

In pattern set mining, bottom-k hashing dramatically accelerates greedy selection procedures, making previously intractable scales feasible with minimal loss in output quality [2507.08745]. In sum, the mergeable, efficient, and robust properties of bottom-k hashing render it a core primitive in large-scale, distributed, and streaming settings for both set and associated weighted estimations.

Source: https://www.emergentmind.com/topics/bottom-k-hashing