MultiSet-MinHash: Weighted Similarity Estimation

• MultiSet-MinHash is a hashing and sketching technique that extends classical MinHash to handle multisets with weighted elements.
• It utilizes quantization, Consistent Weighted Sampling, and P-MinHash methods to estimate generalized Jaccard similarity for high-cardinality data.
• Streaming variants like SimiSketch deliver tight error bounds and superior throughput, making them ideal for real-time analytics in network traffic and text mining.

MultiSet-MinHash refers to a class of randomized hashing and sketching techniques that generalize the MinHash algorithm for sets to multisets (weighted sets), targeting efficient similarity estimation—especially the extension of Jaccard similarity—on high-velocity, high-cardinality data streams and weighted data representations. These methods are foundational for approximate similarity search and large-scale set comparisons where element multiplicity is integral, such as network traffic analytics, text mining, and recommendation systems.

1. Formal Definitions and Problem Context

Let $U$ be a universe of elements. A multiset $A$ over $U$ is represented by a multiplicity function $m_A : U \to \mathbb{Z}_+$, assigning each $x \in U$ a nonnegative integer count $m_A(x)$. The support is $\operatorname{Supp}(A) = \{x : m_A(x) > 0\}$, with cardinality $|A| = \sum_{x \in U} m_A(x)$. For two multisets $A, B$, define:

• Multiset union: $(A \cup B)(x) = \max(m_A(x), m_B(x))$
• Multiset intersection: $A$0

The multiset Jaccard similarity is: $A$1 This generalizes the classical set Jaccard similarity to count vectors and forms the theoretical baseline for MultiSet-MinHash (Dong et al., 2024, Wu et al., 2018).

2. MultiSet-MinHash Algorithms

Multiple algorithmic paradigms extend MinHash to weighted sets:

2.1 Quantization-Based Methods

The quantization approach, typified by Haveliwala et al., "explodes" each weighted element into $A$2 subelements, transforming a multiset into a binary set whose standard MinHash approximates the generalized Jaccard similarity. With scaling factor $A$3, each real weight $A$4 becomes $A$5, and MinHash is run on the expanded universe. Collision probability approximates $A$6 up to quantization error. Complexity is $A$7 for $A$8 sketches, which is prohibitive for high-multiplicity data (Wu et al., 2018).

2.2 Consistent Weighted Sampling (CWS) and Variants

Active-index methods such as Consistent Weighted Sampling (CWS), ICWS, and PCWS, avoid universe inflation by generating, per coordinate, a "landmark" subelement proportional to its weight. The hash for each multiset is generated so that

$A$9

where $U$0 is the coordinate-wise min/max similarity (Wu et al., 2018). Modern variants (ICWS, 0-bit CWS, CCWS, I²CWS) further optimize sampling and reduce storage or computational cost while maintaining unbiasedness or near-unbiasedness.

2.3 Maximally Consistent (P-MinHash) Sampling

For nonnegative weight vectors $U$1, the P-MinHash algorithm samples index $U$2 with probability proportional to $U$3, using either exponential random variables in sparse settings (i.e., $U$4 for $U$5), or A* sampling for continuous/dense data. The induced collision probability

$U$6

defines a scale-invariant, distribution-sensitive similarity generalizing set Jaccard (Moulton et al., 2018).

3. Streaming and Sketching Architectures

Efficient real-time estimation of multiset Jaccard similarity on streams necessitates constant-memory, single-pass data structures. SimiSketch (Dong et al., 2024) introduces a streaming-ready sketch for multisets:

• Count-Min (CM) Based Sketch: Architected as a $U$7 counter array. Each stream increment updates $U$8 hash buckets, and similarity estimation is derived from the per-cell min/max statistics across corresponding buckets. It yields a controlled over-estimate of $U$9, with bias bounded in terms of heavy hitters and bucket count.
• Count Sketch (CS) Based Sketch: Each update increments/decrements buckets using sign hashes $m_A : U \to \mathbb{Z}_+$0. The estimator computes, per cell, normalized min/max of signed counts only if all contributions share the sign, leading to high accuracy for "heavy" items.
• SimiSketch (Combined Sketch): Integrates a pair of counters per cell—one CM-type, one CS-type—combining their outputs to weight local similarity estimates with estimated bucket maxima. Critically, SimiSketch incorporates the SALSA self-sizing framework, allowing counters/buckets to merge adaptively on overflow, maximizing efficiency under fixed memory constraints (Dong et al., 2024).

The table below summarizes principal MultiSet-MinHash streaming variants:

Method	Sketch Structure	Theoretical Guarantee
Quantization	Expanded binary MinHash	Bias from quantization
CM-Sketch	$m_A : U \to \mathbb{Z}_+$1 counts	Over-estimate, O($m_A : U \to \mathbb{Z}_+$2) bias
CS-Sketch	$m_A : U \to \mathbb{Z}_+$3 signed	Unbiased for heavy items
SimiSketch	Joint CM,CS + SALSA	$m_A : U \to \mathbb{Z}_+$4 bound

4. Theoretical Analysis and Guarantees

MultiSet-MinHash algorithms provide explicit error bounds and provable optimality under certain settings:

• CM-Stream Sketch: Given sufficiently large $m_A : U \to \mathbb{Z}_+$5, the probability that the estimated similarity exceeds the true $m_A : U \to \mathbb{Z}_+$6 by $m_A : U \to \mathbb{Z}_+$7 decays exponentially in $m_A : U \to \mathbb{Z}_+$8 (Dong et al., 2024).
• SimiSketch: Under "heavy tail" assumptions (few items dominate total count), the error is $m_A : U \to \mathbb{Z}_+$9 with probability at least $x \in U$0 by choosing sketch parameters accordingly.
• P-MinHash collision optimality: The scale-invariant similarity $x \in U$1 is provably Pareto-optimal among all sampling-based LSH schemes with matching marginals (Moulton et al., 2018).

A plausible implication is that for streaming and large-scale data, sketches like SimiSketch yield maximally tight error bounds in fixed memory, provided the data are heavy-tailed or highly skewed.

5. Empirical Performance and Practical Recommendations

Comprehensive evaluations on synthetic Zipfian data, real-world network traces, and text corpora validate MultiSet-MinHash streaming sketches:

• Accuracy: SimiSketch reduces relative error by up to $x \in U$2 over prior methods at identical memory usage (Dong et al., 2024).
• Throughput: Processing speed increases up to $x \in U$3 compared to MinHash or MaxLogHash baselines.
• Resource Efficiency: Accurate estimation ($x \in U$4 error) is achieved with $x \in U$5; graceful degradation is observed to as low as $x \in U$6.
• Parameter Choices: $x \in U$7 in $x \in U$8 suffices for practical error rates, and SALSA block merging nearly doubles effective counter space without degrading update throughput.

Empirical results confirm that SimiSketch and advanced CWS variants (notably PCWS, I²CWS) are the methods of choice for high-dimensional, streaming or otherwise large-scale multiset similarity estimation (Dong et al., 2024, Wu et al., 2018).

6. Comparison to Prior Art and Extensions

Classical MinHash is binary—it cannot incorporate multiplicities. Quantization methods introduce infeasible overhead for large or real-valued weights. HyperLogLog and MaxLogHash techniques address cardinality, not count similarity. CWS and its descendants offer unbiased, exact similarity for real weights and have become the standard for static data (Wu et al., 2018, Moulton et al., 2018).

SimiSketch, as a streaming MultiSet-MinHash, is distinguished by

• One-pass, exact multiplicity processing,
• CM/CS-based bias/variance tradeoff,
• Logarithmic scaling in $x \in U$9 and $m_A(x)$0 for memory-efficiency,
• Native support for merging sketches in distributed settings (lossless under SALSA-aligned merges),
• Applicability to weighted Jaccard, locality-sensitive hashing, and general similarity tasks (Dong et al., 2024).

Potential research directions include robust support for arbitrary weights (weighted Jaccard), adaptive per-row sizing, and improved variance reduction schemes. Distributed LSH and similarity search over sketches, and more refined error analysis under skewed or adversarial workloads, also remain open (Dong et al., 2024, Wu et al., 2018).

7. Use Cases and Future Outlook

MultiSet-MinHash forms the computational backbone for streaming similarity in domains characterized by repeated item occurrence and dimensional scale—e.g. high-speed clickstreams, IP flow records, and probabilistic text representations. The correctness, memory footprint, and update efficiency of streaming sketches (notably SimiSketch) make them particularly suited for one-pass analytics where element deletion is unnecessary and accurate, low-latency similarity is critical.

Future research is likely to extend these models to general weighted and signed set functions, tighter concentration bounds, fully adaptive or hierarchical sketch structures, and seamless integration into broader approximate query processing architectures (Dong et al., 2024, Wu et al., 2018, Moulton et al., 2018).