---
title: 'MultiSet-MinHash: Weighted Similarity Estimation'
url: https://www.emergentmind.com/topics/multiset-minhash
type: topic
---

# MultiSet-MinHash: Weighted Similarity Estimation

MultiSet-MinHash refers to a class of randomized hashing and sketching techniques that generalize the MinHash algorithm for sets to multisets (weighted sets), targeting efficient similarity estimation—especially the extension of Jaccard similarity—on high-velocity, high-cardinality data streams and weighted data representations. These methods are foundational for approximate similarity search and large-scale set comparisons where element multiplicity is integral, such as network traffic analytics, text mining, and recommendation systems.

## 1. Formal Definitions and Problem Context

Let $U$ be a universe of elements. A multiset $A$ over $U$ is represented by a *multiplicity function* $m_A : U \to \mathbb{Z}_+$, assigning each $x \in U$ a nonnegative integer count $m_A(x)$. The *support* is $\operatorname{Supp}(A) = \{x : m_A(x) > 0\}$, with cardinality $|A| = \sum_{x \in U} m_A(x)$. For two multisets $A, B$, define:
- Multiset union: $(A \cup B)(x) = \max(m_A(x), m_B(x))$
- Multiset intersection: $(A \cap B)(x) = \min(m_A(x), m_B(x))$

The *multiset Jaccard similarity* is:
\[
J(A,B) = \frac{|A \cap B|}{|A \cup B|} = \frac{\sum_x \min(m_A(x), m_B(x))}{\sum_x \max(m_A(x), m_B(x))}
\]
This generalizes the classical set Jaccard similarity to count vectors and forms the theoretical baseline for MultiSet-MinHash [2405.19711][1811.04633].

## 2. MultiSet-MinHash Algorithms

Multiple algorithmic paradigms extend MinHash to weighted sets:

### 2.1 Quantization-Based Methods

The quantization approach, typified by Haveliwala et al., "explodes" each weighted element into $m_A(x)$ subelements, transforming a multiset into a binary set whose standard MinHash approximates the generalized Jaccard similarity. With scaling factor $C$, each real weight $\tilde{S}_k$ becomes $S_k = \lfloor C\tilde{S}_k \rfloor$, and MinHash is run on the expanded universe. Collision probability approximates $J_{\text{gen}}$ up to quantization error. Complexity is $O(D \sum_k S_k)$ for $D$ sketches, which is prohibitive for high-multiplicity data [1811.04633].

### 2.2 Consistent Weighted Sampling (CWS) and Variants

Active-index methods such as Consistent Weighted Sampling (CWS), ICWS, and PCWS, avoid universe inflation by generating, per coordinate, a "landmark" subelement proportional to its weight. The hash for each multiset is generated so that
\[
\Pr[h(A) = h(B)] = J_{\text{gen}}(A, B)
\]
where $J_{\text{gen}}$ is the coordinate-wise min/max similarity [1811.04633]. Modern variants (ICWS, 0-bit CWS, CCWS, I²CWS) further optimize sampling and reduce storage or computational cost while maintaining unbiasedness or near-unbiasedness.

### 2.3 Maximally Consistent (P-MinHash) Sampling

For nonnegative weight vectors $x \in \mathbb{R}_+^n$, the P-MinHash algorithm samples index $i$ with probability proportional to $x_i$, using either exponential random variables in sparse settings (i.e., $e_i = -\ln(U_i)/x_i$ for $U_i \sim \mathrm{Uniform}(0,1]$), or A* sampling for continuous/dense data. The induced collision probability
\[
\Pr[H(x) = H(y)] = \sum_{i=1}^n \frac{1}{\sum_{j=1}^n \max(x_j/x_i, y_j/y_i)}
\]
defines a scale-invariant, distribution-sensitive similarity generalizing set Jaccard [1809.04052].

## 3. Streaming and Sketching Architectures

Efficient real-time estimation of multiset Jaccard similarity on streams necessitates constant-memory, single-pass data structures. SimiSketch [2405.19711] introduces a streaming-ready sketch for multisets:

- **Count-Min (CM) Based Sketch:** Architected as a $k \times \ell$ counter array. Each stream increment updates $k$ hash buckets, and similarity estimation is derived from the per-cell min/max statistics across corresponding buckets. It yields a controlled over-estimate of $J(A,B)$, with bias bounded in terms of heavy hitters and bucket count.

- **Count Sketch (CS) Based Sketch:** Each update increments/decrements buckets using sign hashes $s_i(x) \in \{\pm1\}$. The estimator computes, per cell, normalized min/max of signed counts only if all contributions share the sign, leading to high accuracy for "heavy" items.

- **SimiSketch (Combined Sketch):** Integrates a pair of counters per cell—one CM-type, one CS-type—combining their outputs to weight local similarity estimates with estimated bucket maxima. Critically, SimiSketch incorporates the SALSA self-sizing framework, allowing counters/buckets to merge adaptively on overflow, maximizing efficiency under fixed memory constraints [2405.19711].

The table below summarizes principal MultiSet-MinHash streaming variants:

| Method      | Sketch Structure         | Theoretical Guarantee      |
|-------------|-------------------------|---------------------------|
| Quantization| Expanded binary MinHash | Bias from quantization    |
| CM-Sketch   | $k\times\ell$ counts    | Over-estimate, O($\varepsilon$) bias |
| CS-Sketch   | $k\times\ell$ signed    | Unbiased for heavy items  |
| SimiSketch  | Joint CM,CS + SALSA     | $O(\varepsilon)+o(1)$ bound |

## 4. Theoretical Analysis and Guarantees

MultiSet-MinHash algorithms provide explicit error bounds and provable optimality under certain settings:

- **CM-Stream Sketch:** Given sufficiently large $\ell=O((1/\varepsilon)\log(1/\delta))$, the probability that the estimated similarity exceeds the true $J(A,B)$ by $2\varepsilon+\delta$ decays exponentially in $\ell$ [2405.19711].
- **SimiSketch:** Under "heavy tail" assumptions (few items dominate total count), the error is $O(\varepsilon) + o(1)$ with probability at least $1-\delta$ by choosing sketch parameters accordingly.
- **P-MinHash collision optimality:** The scale-invariant similarity $J(x,y)$ is provably Pareto-optimal among all sampling-based LSH schemes with matching marginals [1809.04052].

A plausible implication is that for streaming and large-scale data, sketches like SimiSketch yield maximally tight error bounds in fixed memory, provided the data are heavy-tailed or highly skewed.

## 5. Empirical Performance and Practical Recommendations

Comprehensive evaluations on synthetic Zipfian data, real-world network traces, and text corpora validate MultiSet-MinHash streaming sketches:

- **Accuracy:** SimiSketch reduces relative error by up to $42\times$ over prior methods at identical memory usage [2405.19711].
- **Throughput:** Processing speed increases up to $360\times$ compared to MinHash or MaxLogHash baselines.
- **Resource Efficiency:** Accurate estimation ($<1\%$ error) is achieved with $40\,\text{KB}$; graceful degradation is observed to as low as $10\,\text{KB}$.
- **Parameter Choices:** $k$ in $[4,16]$ suffices for practical error rates, and SALSA block merging nearly doubles effective counter space without degrading update throughput.

Empirical results confirm that SimiSketch and advanced CWS variants (notably PCWS, I²CWS) are the methods of choice for high-dimensional, streaming or otherwise large-scale multiset similarity estimation [2405.19711][1811.04633].

## 6. Comparison to Prior Art and Extensions

Classical MinHash is binary—it cannot incorporate multiplicities. Quantization methods introduce infeasible overhead for large or real-valued weights. HyperLogLog and MaxLogHash techniques address cardinality, not count similarity. CWS and its descendants offer unbiased, exact similarity for real weights and have become the standard for static data [1811.04633][1809.04052].

SimiSketch, as a streaming MultiSet-MinHash, is distinguished by
- One-pass, exact multiplicity processing,
- CM/CS-based bias/variance tradeoff,
- Logarithmic scaling in $\varepsilon^{-1}$ and $\delta^{-1}$ for memory-efficiency,
- Native support for merging sketches in distributed settings (lossless under SALSA-aligned merges),
- Applicability to weighted Jaccard, locality-sensitive hashing, and general similarity tasks [2405.19711].

Potential research directions include robust support for arbitrary weights (weighted Jaccard), adaptive per-row sizing, and improved variance reduction schemes. Distributed LSH and similarity search over sketches, and more refined error analysis under skewed or adversarial workloads, also remain open [2405.19711][1811.04633].

## 7. Use Cases and Future Outlook

MultiSet-MinHash forms the computational backbone for streaming similarity in domains characterized by repeated item occurrence and dimensional scale—e.g. high-speed clickstreams, IP flow records, and probabilistic text representations. The correctness, memory footprint, and update efficiency of streaming sketches (notably SimiSketch) make them particularly suited for one-pass analytics where element deletion is unnecessary and accurate, low-latency similarity is critical.

Future research is likely to extend these models to general weighted and signed set functions, tighter concentration bounds, fully adaptive or hierarchical sketch structures, and seamless integration into broader approximate query processing architectures [2405.19711][1811.04633][1809.04052].

Source: https://www.emergentmind.com/topics/multiset-minhash