Coordinated Weighted Sampling

• Coordinated weighted sampling is a probabilistic method that produces compact, overlapping summaries for large weighted datasets, enabling accurate estimation of aggregate statistics.
• It uses shared random seeds to ensure heavy items across various weight assignments are sampled together, thereby reducing variance and storage needs.
• The technique underpins scalable similarity search, approximate query processing, and combinatorial optimization in streaming and distributed environments.

Coordinated weighted sampling (CWS) is a probabilistic data summarization technique that generates compact, reusable representations (“sketches”) of large, weighted datasets while coupling the sampling process across related datasets or functions. By sharing randomization (such as hash seeds) across instances, CWS enables efficient, accurate, and unbiased estimation of a wide variety of aggregate statistics—including set similarity, sums under multiple weightings, and complex combinatorial functions—using samples orders-of-magnitude smaller than the raw data and with strong relative error guarantees. CWS is foundational for scalable similarity search, approximate query processing, large-scale statistical analytics, and combinatorial optimization in streaming and distributed environments.

1. Conceptual Foundations and Problem Statement

CWS generalizes classical sampling by introducing coordination: for a universe of keys $I$ and a collection of weight assignments $\mathcal{W} = \{b_1,\dots,b_m\}$, it draws random numbers (ranks or seeds) shared across all assignments for each key $i\in I$. The core construction is as follows:

• For each key $i$, generate a seed $u_i$—for example, $u_i \sim \text{Uniform}(0,1)$ via a hash of $i$.
• For each assignment $b\in \mathcal{W}$, compute a sampling rank or threshold as a function of $w^{(b)}(i)$ and $u_i$.
• Build fixed-size or Poisson-based sketches (e.g., bottom-$\mathcal{W} = \{b_1,\dots,b_m\}$0) for each $\mathcal{W} = \{b_1,\dots,b_m\}$1 using these coordinated ranks.

The coordination ensures that heavy items under multiple assignments are likely to be included together, enabling overlap-sensitive analytics such as weighted set intersection, $\mathcal{W} = \{b_1,\dots,b_m\}$2-difference, max-min-dominance, and more (0906.4560).

For weighted sets $\mathcal{W} = \{b_1,\dots,b_m\}$3 with nonnegative weights, the generalized Jaccard similarity is defined as: $\mathcal{W} = \{b_1,\dots,b_m\}$4 CWS schemes build sketches such that: $\mathcal{W} = \{b_1,\dots,b_m\}$5 allowing unbiased estimation of $\mathcal{W} = \{b_1,\dots,b_m\}$6 using collisions over $\mathcal{W} = \{b_1,\dots,b_m\}$7 hash functions (Raff et al., 2018).

2. Algorithms and Sketch Construction

Several CWS methods exist, optimized for accuracy, speed, or reusability:

Consistent Weighted Sampling (CWS) and Variants:

• ICWS: For each feature $\mathcal{W} = \{b_1,\dots,b_m\}$8 and hash $\mathcal{W} = \{b_1,\dots,b_m\}$9, draw $i\in I$0, $i\in I$1, compute $i\in I$2, $i\in I$3, and score $i\in I$4. The minimum $i\in I$5 determines the sampled feature for each hash. Coordination is imposed across all datasets by sharing $i\in I$6 (Raff et al., 2018).
• 0-bit CWS: The output is reduced to just the index of the selected feature (dropping the ticket component) without significant effect on estimation variance.
• SCWS: A computational simplification exploiting pre-computed tables of the random factor $i\in I$7, enabling per-sketch construction up to 20$i\in I$8 faster than ICWS, while preserving accuracy and theoretical guarantees. The index into the table is computed deterministically as $i\in I$9 (Raff et al., 2018).

Threshold/Probability and Priority-Sampling (Inner Product, General Linear Statistics):

• Threshold Sampling: Given vector $i$0, specify a threshold $i$1, and include index $i$2 if $i$3.
• Priority Sampling: For sketch of size $i$4, compute priority ranks $i$5 and select the $i$6 smallest, using the shared $i$7 for coordination.
• Both methods achieve unbiased estimators for bilinear and $i$8 statistics with per-instance variance $i$9, where $u_i$0 is the overlap support (Daliri et al., 2023).

Multi-Objective Sampling:

• When a universe of statistics $u_i$1 is of interest, a coordinated multi-objective sample $u_i$2 efficiently supports accurate estimation for all $u_i$3, with overhead proportional to the maximum per-key coverage rather than $u_i$4 (Cohen, 2015).

The table below summarizes key CWS sketch families and their distinguishing features:

Algorithm	Coordination Mechanism	Application Scope
ICWS/SCWS	Shared seeds, explicit rank transform	Weighted Jaccard, minhash-based similarity
Threshold/Priority Sampling	Hash-based per-index sharing	Inner product, linear/semilinear aggregates
Multi-Objective	Unified random seeds	Many statistics, optimization objectives

3. Estimation Theory and Statistical Guarantees

CWS sketches admit unbiased estimators for a wide range of functions $u_i$5:

• For sum aggregates, the classic Horvitz–Thompson estimator weights by the reciprocal of inclusion probability, which is computable due to coordination.
• For more complex statistics (e.g., $u_i$6 difference, max/min dominance), estimators depend on multi-sketch certification and the combinatorial structure of the sampling set (see $u_i$7-set and $u_i$8-set dependence for max/min aggregates) (0906.4560).

Variance and concentration properties:

• For bottom-$u_i$9 or threshold CWS, the variance of segment-sum estimation is bounded tightly by the sample size and the effective “relative weight” of the query set.
• CWS strictly dominates independent sketching for overlap-sensitive statistics, often reducing variance by orders of magnitude as the number of weight assignments increases (0906.4560).
• The J-estimator provides a universal, variance-competitive construction for arbitrary item functions admitting unbiased, nonnegative estimators (Cohen et al., 2012).

Statistical guarantees:

• For segment sum estimation via coordinated Poisson-pps, the coefficient of variation (CV) is

$u_i \sim \text{Uniform}(0,1)$0

where $u_i \sim \text{Uniform}(0,1)$1 is the relative weight and $u_i \sim \text{Uniform}(0,1)$2 the disparity. This demonstrates improved error scaling compared to non-coordinated approaches (Cohen, 2015).

4. Implementation, Efficiency, and Practical Considerations

CWS methods are designed for streaming and distributed environments:

• Seed generation via hash functions enables reproducible, low-collision coordination across distributed streams (Cohen et al., 2012).
• Bottom-$u_i \sim \text{Uniform}(0,1)$3 and Poisson-pps CWS sketches are naturally mergeable: sketches built over different data shards can be composed with $u_i \sim \text{Uniform}(0,1)$4 per-sample cost.
• SCWS achieves massive per-sample speedup through table-lookup, requiring only one floating-point multiplication per feature-hash combination (Raff et al., 2018).
• Storage is minimized by tracking only sampled items, associated seeds, and assignment-specific weights, with empirical union-sketch size typically much less than the product of sketch size and number of assignments.

Priority and threshold CWS sketches for high-dimensional vectors are constructed in $u_i \sim \text{Uniform}(0,1)$5 or $u_i \sim \text{Uniform}(0,1)$6 time (for $u_i \sim \text{Uniform}(0,1)$7, $u_i \sim \text{Uniform}(0,1)$8 sketch size). In practice, they significantly outperform CountSketch and JL-projection for inner-product and sparse subset overlap tasks, achieving $u_i \sim \text{Uniform}(0,1)$9–$i$0 lower error (Daliri et al., 2023).

Empirical evaluations on network, text, and financial data confirm that coordination achieves both tight accuracy and storage compression—e.g., in multi-assignment aggregates, variance reduction from $i$1 to $i$2 over independence has been observed (0906.4560).

5. Expressiveness, Generalizations, and Multi-Objective Settings

The expressive power of coordinated weighted sampling is characterized by the class of functions for which unbiased, nonnegative, and bounded estimators exist. These are determined by the convex lower-hull of the function’s value space under the CWS sampling process (Cohen et al., 2012). For every function satisfying certain lower-bound properties, the J-estimator yields an estimator competitive within a constant factor of the minimum possible variance, encompassing distinct counts, min/max, $i$3-difference, and more.

Multi-objective CWS supports simultaneous estimation of numerous statistics using coordinated samples. With proper design (e.g., universal monotone or capping samples), the required sketch size scales as $i$4 for all monotone functions over $i$5 items, enabling broad application to metric clustering, centrality, moments, and threshold statistics, all from a single compact summary (Cohen, 2015).

6. Applications and Comparative Impact

CWS is foundational in the following domains:

• Similarity search: Weighted minhash and CWS enable accurate approximations of set and vector similarities for massive datasets (Raff et al., 2018).
• Approximate query processing: Efficient inner product and $i$6 statistics over dynamic datasets, join-size/correlation estimation for unjoined tables, and range queries (Daliri et al., 2023).
• Multi-stream, multi-objective analytics: Simultaneous summarization for streaming snapshots, large-scale monitoring, and multi-assignment aggregates (0906.4560).
• Combinatorial optimization: CWS-based samples enable scalable objective estimation for $i$7-means and $i$8-median clustering, and centrality analysis (Cohen, 2015).

Compared to uncoordinated sampling and classical linear sketching (e.g., JL, CountSketch), CWS provides improved variance scaling for sparse and overlap-centric queries, superior flexibility for multi-query use, tight size guarantees, and efficient streaming composability.

7. Theoretical Limits, Extensions, and Open Directions

The theoretical boundaries of coordinated weighted sampling are precisely characterized: only functions with sufficient lower bound and convexity properties admit unbiased, nonnegative estimators, with variance-characterization via convex hull methods (Cohen et al., 2012). Recent work has extended these principles to inner product sketches using threshold and priority CWS, optimizing for $i$9 error rather than classical union or intersection sizes (Daliri et al., 2023). Multi-objective and universal CWS structures continue to support emerging applications in optimization meta-algorithms, streaming clustering, and sublinear-time analytics (Cohen, 2015).

A plausible implication is that as workloads demand more diverse and compositional statististical summaries, coordinated weighted sampling will remain central to streaming and sketching theory, offering optimal tradeoffs between space, accuracy, and query expressiveness.