b-Bit MinHash: Compressed Minwise Hashing

• b-Bit MinHash is a compressed variant of minwise hashing that truncates hash signatures to lower storage requirements.
• It enables unbiased Jaccard similarity estimation by analytically correcting for collisions introduced by b-bit truncation.
• The method supports efficient kernel construction and scalable learning, particularly for high-dimensional sparse binary data.

b-Bit MinHash, or b-bit minwise hashing, is a compressed form of minwise hashing for estimating set resemblance and for building similarity-preserving representations of massive sparse binary data. For sets $S_1,S_2\subseteq \Omega=\{0,1,\dots,D-1\}$, its target similarity is the resemblance or Jaccard similarity,

$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$

with $f_1=|S_1|$, $f_2=|S_2|$, and $a=|S_1\cap S_2|$. Classical MinHash stores the minimum under each random permutation; b-Bit MinHash stores only the lowest $b$ bits of each such minimum, and then corrects analytically for the extra collisions introduced by truncation. The 2009 paper that established the method framed it as a storage-efficient alternative to ordinary MinHash, while later work showed that the same representation induces a valid positive definite kernel, an explicit sparse linear feature map, and effective large-scale learning algorithms for linear SVM and logistic regression (0910.3349, Li et al., 2011).

1. Origins and core construction

Classical minwise hashing applies a random permutation $\pi:\Omega\to\Omega$ and records

$z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$

Its defining identity is

$\Pr\bigl(z_1=z_2\bigr)=R.$

With $k$ independent permutations $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$0, the standard estimator is

$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$1

with variance

$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$2

The difficulty is storage: in the standard implementation each minhash is typically stored as a 64-bit value, so large $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$3 and large collections make sketch storage and repeated I/O expensive (0910.3349).

b-Bit MinHash changes only the stored representation. If $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$4 and $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$5 are the $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$6-th lowest bits of $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$7 and $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$8, then instead of testing full equality of $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$9 and $f_1=|S_1|$0, it tests whether the lowest $f_1=|S_1|$1 bits match. This reduces storage from 64 bits per sample to $f_1=|S_1|$2 bits per sample, a factor of $f_1=|S_1|$3 reduction (Li et al., 2011). The original theoretical paper further quantified the tradeoff in the sparse least favorable regime $f_1=|S_1|$4: for resemblance at least $f_1=|S_1|$5, using $f_1=|S_1|$6 may reduce storage space by at least a factor of $f_1=|S_1|$7 relative to 64-bit MinHash, or $f_1=|S_1|$8 relative to 32-bit MinHash (0910.3349).

The method was developed for binary or set-valued data in extremely high-dimensional sparse regimes. That specialization is essential: its most direct similarity target is resemblance, and much of its later learning theory, empirical evaluation, and systems engineering is explicitly about sparse binary vectors, shingled text, and similarly structured data (Li et al., 2011, Li et al., 2012).

2. Collision law, unbiased estimation, and variance

The central theoretical result of b-Bit MinHash is that the probability of a $f_1=|S_1|$9-bit collision remains an affine function of resemblance. Let

$f_2=|S_2|$0

and define $f_2=|S_2|$1, $f_2=|S_2|$2. For large $f_2=|S_2|$3,

$f_2=|S_2|$4

where

$f_2=|S_2|$5

with

$f_2=|S_2|$6

Thus truncation does not destroy identifiability of $f_2=|S_2|$7; it changes only the intercept and slope of the collision law (0910.3349, Li et al., 2011).

Using $f_2=|S_2|$8 independent permutations, the empirical $f_2=|S_2|$9-bit collision rate is

$a=|S_1\cap S_2|$0

and the unbiased resemblance estimator is

$a=|S_1\cap S_2|$1

Its variance is

$a=|S_1\cap S_2|$2

As $a=|S_1\cap S_2|$3, $a=|S_1\cap S_2|$4, so $a=|S_1\cap S_2|$5 and b-Bit MinHash recovers ordinary MinHash (0910.3349).

In the sparse limit emphasized repeatedly in later work, the collision law simplifies to

$a=|S_1\cap S_2|$6

so the extra collision probability becomes the intuitive random-match floor $a=|S_1\cap S_2|$7 (Li et al., 2011). The large-$a=|S_1\cap S_2|$8 approximation is a standing assumption of the theory, but the later learning paper notes that the approximation is highly accurate even for fairly small $a=|S_1\cap S_2|$9 (Li et al., 2011).

3. Positive-definite kernel and explicit feature map

A major later development was the realization that b-Bit MinHash is not only a similarity estimator but also a valid kernel construction. For sets $b$0, let $b$1 under one permutation. The following matrices are all positive definite:

1. the resemblance matrix $b$2, with

$b$3

2. the minwise hashing matrix $b$4, with

$b$5

3. the b-bit minwise hashing matrix $b$6, with

$b$7

The proof is constructive. For b-Bit MinHash,

$b$8

so each object is represented by a one-hot vector over $b$9 categories, and $\pi:\Omega\to\Omega$0 is a Gram matrix. The resemblance matrix is positive definite because $\pi:\Omega\to\Omega$1, and expectation preserves positive definiteness (Li et al., 2011).

This observation yields the explicit linearization used in large-scale learning. For each of the $\pi:\Omega\to\Omega$2 retained $\pi:\Omega\to\Omega$3-bit values $\pi:\Omega\to\Omega$4, encode $\pi:\Omega\to\Omega$5 as a one-hot vector in $\pi:\Omega\to\Omega$6, then concatenate the $\pi:\Omega\to\Omega$7 blocks. The resulting expanded feature map has dimension

$\pi:\Omega\to\Omega$8

and exactly $\pi:\Omega\to\Omega$9 ones. Inner products in that expanded space count $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$0-bit collisions and therefore approximate resemblance. This is the basis for integrating b-Bit MinHash with linear SVM, logistic regression, and related linear learners while approximating a nonlinear resemblance kernel (Li et al., 2011).

The resulting preprocessing-and-learning pipeline is simple: start from sparse binary vectors, generate $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$1 minhashes, retain only the lowest $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$2 bits, store the compact representation, expand each code to a one-hot block at runtime, and train a linear solver such as LIBLINEAR. The 2011 learning paper reports that only a very minimal modification of LIBLINEAR was needed (Li et al., 2011).

4. Large-scale learning, empirical performance, and systems practice

The learning papers evaluate b-Bit MinHash primarily on the webspam dataset, with $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$3, $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$4, and about 24GB in LIBSVM format, and later on an expanded RCV1 dataset of about 200GB with about $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$5 dimensions (Li et al., 2011, Li et al., 2011).

Dataset	Scale	Reported outcome
webspam	$z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$6 examples, $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$7 dimensions, about 24GB	$z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$8 and $z_1=\min(\pi(S_1)),\qquad z_2=\min(\pi(S_2)).$9–$\Pr\bigl(z_1=z_2\bigr)=R.$0 gave test accuracy very close to the original data
webspam	$\Pr\bigl(z_1=z_2\bigr)=R.$1, $\Pr\bigl(z_1=z_2\bigr)=R.$2	about 70MB effective storage versus roughly 24GB original input
expanded RCV1	about 200GB, about $\Pr\bigl(z_1=z_2\bigr)=R.$3 features	with $\Pr\bigl(z_1=z_2\bigr)=R.$4, $\Pr\bigl(z_1=z_2\bigr)=R.$5, linear SVM and logistic regression already exceeded 90% test accuracy
expanded RCV1	$\Pr\bigl(z_1=z_2\bigr)=R.$6, $\Pr\bigl(z_1=z_2\bigr)=R.$7	test accuracy exceeded 95% for both linear SVM and logistic regression

On webspam, the reported speedups were substantial. For linear SVM, training time dropped from about 100 seconds on the original data to about 3–7 seconds near $\Pr\bigl(z_1=z_2\bigr)=R.$8, excluding preprocessing; loading original data took about 12 minutes versus about 10 seconds for hashed data; and testing time dropped from about 100–200 seconds to about 1–2 seconds. For logistic regression, training time dropped from about 1000 seconds to roughly 30–50 seconds when $\Pr\bigl(z_1=z_2\bigr)=R.$9. The same study reports that direct nonlinear SVM training with the resemblance kernel was still running after a week, whereas using b-bit hashed representations made the learning task feasible, with $k$0, $k$1 yielding test accuracy essentially matching the best linear baseline on the original data (Li et al., 2011).

Comparisons with Vowpal Wabbit (VW) and random projections are a second major empirical theme. The theory paper shows that VW has the same variance as random projections when the sign distribution is $k$2, and that on binary data b-Bit MinHash is usually far more accurate at equal storage (Li et al., 2011). On webspam, 8-bit MinHash with $k$3 achieved about the same test accuracy as VW with $k$4 (Li et al., 2011). On the 200GB expanded RCV1 dataset, merely 30 b-bit hashed values per data point achieved similar accuracies to VW with $k$5 hashed values per data point (Li et al., 2011).

The main practical obstacle is preprocessing, because one must compute many minima per example. The 2012 systems paper addressed three deployment bottlenecks: preprocessing cost, applicability to online learning, and replacement of full random permutations by simple hash functions. It reported GPU preprocessing speedups of 20–80, making preprocessing substantially smaller than data loading time, and showed that simple 2-universal and 4-universal hash families produced learning results very similar to those obtained with fully random permutations on sparse data. In online learning, where data are loaded for many epochs, the compressed representation reduced per-epoch loading time enough to yield training-time ratios of 10.05 on webspam and 28.91 on expanded RCV1, with loading-time ratios of 8.95 and 29.07, respectively (Li et al., 2012).

5. Statistical refinements and theoretical scope

The original resemblance estimator uses only whether two retained $k$6-bit values are equal. Later work showed that this discards information. For each permutation, the pair $k$7 of retained $k$8-bit values lies in a $k$9 contingency table. The 2011 estimator paper therefore proposed a hierarchy of maximum-likelihood estimators: the full estimator $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$00, using all $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$01 cells; $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$02, using the $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$03 diagonal cells plus $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$04 and $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$05; $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$06, using the diagonal cells plus one lumped off-diagonal cell; and the practical three-cell estimator $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$07, using only

$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$08

The paper’s main empirical conclusion is that the biggest gains occur when resemblance is low but containment is high, and that for b-bit hashing the full MLE can improve over the equality-only estimator by 5-fold to 100-fold, while the three-cell estimator often preserves most of that gain (Li et al., 2011).

A separate line of theory studied b-bit min-wise hashing as a statistically analyzable dimension-reduction method for sparse regression and classification. For a sparse design matrix $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$09 with average number of nonzero entries per row $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$10, the 2013 theory paper showed that ordinary least squares, ridge regression, and $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$11-penalized logistic regression on the hashed design can achieve vanishing prediction error asymptotically as long as

$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$12

The same paper also showed that regression on the reduced design can approximate not only ordinary linear models but also more flexible structures, including row-normalized signals and interaction models, while retaining non-asymptotic error bounds driven by sparsity rather than by the ambient dimension $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$13 (Shah et al., 2013).

The b-bit viewpoint also reappears in later privacy work. The 2020 local-differential-privacy paper explicitly builds on Li and König’s range compression: if $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$14, then each range-$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$15 MinHash entry is a $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$16-bit value, and in the experimentally favored regime $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$17 the released sketch is effectively a privatized 1-bit MinHash signature. That paper treats range-$$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$18 MinHash as a generalization of b-Bit MinHash and derives unbiased estimators that debias both compression collisions and privacy noise (Aumüller et al., 2020).

6. Limitations, misconceptions, and later developments

b-Bit MinHash is specialized rather than universal. Its strongest results are for binary or naturally binarized data, very high-dimensional sparse vectors, and resemblance-style similarity. Its formulas assume large $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$19, though later work reports that the approximation is highly accurate even for smaller $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$20. Preprocessing requires generating multiple minhashes, and true random permutations on huge universes are expensive, which is why later systems work relied on universal hash families and GPU acceleration (Li et al., 2011, Li et al., 2012).

A persistent misconception is that b-Bit MinHash is merely a static similarity-estimation trick. That is only partly correct. It is also a positive definite kernel with an explicit finite-dimensional feature map, which is why it can support large-scale supervised learning (Li et al., 2011). The opposite misconception is that because it stores only a few bits, it should be naturally suited to streaming or sketch algebra. Later papers explicitly reject that interpretation. HyperMinHash states that unlike comparable Jaccard fingerprints in sub-logarithmic space such as b-bit MinHash, it retains MinHash’s features of streaming updates, unions, and cardinality estimation; it also recommends b-bit MinHash when the goal is only a one-time Jaccard fingerprint of a set (Yu et al., 2017). MaxLogHash similarly presents b-bit MinHash as highly memory-efficient for static high-similarity estimation but unsuitable for streaming sets (Wang et al., 2019). SetSketch makes the same contrast in another form, noting that b-bit minwise hashing reduces MinHash values to a few bits in a finalization step, but after finalization the sketch cannot be further aggregated or merged (Ertl, 2021).

Several later variants address operational gaps rather than replacing the core method outright. Pb-Hash partitions a large $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$21-bit hash into $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$22 chunks of $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$23 bits rather than discarding most bits; its analysis shows an accuracy loss from correlation, but often not much for $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$24 (Li et al., 2023). C-MinHash rigorously reduces the number of permutations needed for full MinHash from $$R(S_1,S_2)=\frac{|S_1\cap S_2|}{|S_1\cup S_2|}=\frac{a}{f_1+f_2-a},$$25 to two via a circulant construction; that result is directly relevant to upstream hash generation, although it does not analyze b-bit truncation itself (Li et al., 2021). A dynamic threshold filter for MinHash-style estimators was also later proposed for image similarity search and explicitly stated to extend to b-Bit Minwise Hashing whenever the estimator satisfies a binomial distribution (Long et al., 2018).

The resulting historical pattern is consistent. b-Bit MinHash remains the canonical compressed MinHash method for static sparse binary data, especially when the task is Jaccard-style fingerprinting or resemblance-based learning. Later work mainly clarifies what is gained by this aggressive truncation—dramatic storage reduction, strong learning performance, and explicit kernel structure—and what is lost—mergeability, streaming updates, and part of the statistical information available in full hashes.