---
title: Probability Proportional to Size Without Replacement
url: https://www.emergentmind.com/topics/probability-proportional-to-size-without-replacement-ppswor
type: topic
---

# Probability Proportional to Size Without Replacement

Probability Proportional to Size Without Replacement (PPSWOR) is a fundamental statistical sampling scheme wherein units are drawn sequentially from a finite population without replacement, and at each draw, the probability of selection is proportional to a user-defined “size” or weight associated with each unit. PPSWOR is central to a spectrum of applications, including complex survey design, randomized experiments, distributed data summarization, scalable learning algorithms, and theoretical analyses of dependent selection processes.

## 1. Formal Definitions, Probabilistic Structure, and Canonical Algorithms

Given a population \( U = \{1, 2, \dots, N\} \) with non-negative weights \( w_1, \dots, w_N \), PPSWOR refers to sampling a subset \( S \subset U \) of fixed size \( n \) such that, at each selection step, an element \( i \) is chosen with probability \( w_i \) divided by the sum of weights of remaining (unselected) elements. Formally, if \( R_{k-1} \) denotes the set of as-yet-unsampled units at step \( k \), the conditional probability of selecting \( i \in R_{k-1} \) is:
\[
\Pr\{j_k = i \mid R_{k-1} \} = \frac{w_i}{\sum_{h \in R_{k-1}} w_h}
\]
Classic PPSWOR schemes include:
- **Hájek’s rejective (conditional Poisson) sampling**: Equivalent to repeatedly drawing Bernoulli indicators with Poissonized marginal probabilities, then conditioning on the sample having fixed size; yields computationally tractable expressions for inclusion probabilities via elementary symmetric polynomials [1005.4107].
- **Successive (sequential) PPSWOR**: Sampling with replacement, discarding repeats, until \( n \) distinct items are drawn; admits an explicit albeit complex sum-of-products representation for inclusion probabilities.
- **Streaming and algorithmic variants**: Schemes such as the EB-PPS algorithm provide exact PPSWOR (i.e., sticking to prescribed proportionality in marginal inclusion probabilities) while controlling sample size in a single pass over weighted streams [2105.10809].

For any unit \( i \), the first-order inclusion probability under PPSWOR with sample size \( n \) and weights \( \alpha = (\alpha_1, \dots, \alpha_N) \) (normalized, so \( \sum_i \alpha_i = 1 \)) is:
- **Rejective**: \( \pi_i^R(n) = \frac{\alpha_i\,e_{n-1}(\alpha_{-i})}{e_n(\alpha)} \), where \( e_k(\cdot) \) is the \( k \)th elementary symmetric sum.
- **Successive**: \( \pi_i^S(n) = \frac{1}{n} \sum_{k=1}^n \pi_{i,k} \), with \( \pi_{i,k} \) the intricate sum detailed in [1005.4107].

## 2. Key Theoretical Properties: Uniformity, Proportionality, and Majorization

PPSWOR methods, in contrast to with-replacement sampling, induce dependencies and negative correlations among the inclusion indicators of different units. The inclusion probabilities inherit important structural properties:

- **Majorization order**: For both rejective and successive PPSWOR, as \( n \) increases, the per-sample inclusion vector \( p(n) = (\pi_i(n)/n) \) becomes more uniform, i.e., majorizes those at smaller \( n \). For fixed \( n \) and draw probabilities \( \alpha \), the inclusion probabilities produced by rejective sampling are always more uniform (in majorization sense) than those from successive sampling, confirming the conjecture of Hájek [1005.4107].
- **Kullback-Leibler divergence**: Inclusion probabilities from successive PPSWOR are always closer (in KL divergence) to the original drawing probabilities \( \alpha \) than those from rejective PPSWOR. That is, \( p^S(n) \) is more “proportional” to \( \alpha \), while \( p^R(n) \) is more “uniform” [1005.4107].
- **Negative correlation**: In any PPSWOR scheme, inclusion indicators for distinct units exhibit negative dependence; \( \Pr[i \in S,\, j \in S] \leq \Pr[i \in S]\Pr[j \in S] \) [2410.06868].

These facts have critical implications for variance reduction, estimator design, and the analysis of sampling-based algorithms.

## 3. Algorithms, Computational Complexity, and Streaming Schemes

Computing inclusion probabilities in PPSWOR depends on the chosen variant and the population size.

- **Rejective sampling**: Efficient dynamic programming via recursion on symmetric polynomials computes all inclusion probabilities in \( O(Nn) \) time [1005.4107].
- **Successive sampling**: The sum-of-products formula requires potentially exponential time; practical methods include Monte Carlo simulation, sandwiching by majorization bounds, or iterative refinement [1005.4107].
- **Streaming EB-PPS**: The EB-PPS algorithm maintains a “latent sample” representation, updating inclusion probabilities and sample content in \( O(1) \) amortized time per item. At each step, proportionality factor \( \rho \) is updated as \( \rho = \min\{1/\max_i w_i,\, n/\sum_j w_j\} \), and inclusion probabilities are set as \( \pi_i = \rho w_i \) [2105.10809].

Modern applications employ sketch-based algorithms, such as bottom-\( k \) and \( \ell_p \)-residual heavy hitter sketches, to perform PPSWOR efficiently in high-dimensional streaming or distributed environments [2007.06744].

| Algorithm Type        | Input Model      | Complexity                 |
|----------------------|------------------|----------------------------|
| Rejective (Hájek)    | Static           | \( O(Nn) \)                |
| Successive (sequential) | Static       | \( O(Nn^2) \) (DP); MC feasible for large N |
| EB-PPS (streaming)   | Streaming        | \( O(1) \) amortized per item  |

## 4. Statistical Inference: Estimation, Variance, and Design-based/Bayesian Methods

PPSWOR underpins the design of unbiased estimators in finite population sampling, most notably the Horvitz-Thompson (HT) estimator. For population mean or total estimation:

- **HT estimator**: If \( \pi_i \) is the inclusion probability of unit \( i \), the estimator is \( \hat{Y}_{HT} = \sum_{i \in S} y_i / \pi_i \).
- **Variance**: The variance involves both first- and second-order inclusion probabilities \( \pi_{ij} \). For example, in cluster-randomized experiments, closed forms for the variance are available under PPSWOR, and conservative unbiased estimators (Sen-Yates-Grundy) can be employed even when joint probabilities are approximated [2002.08009].
- **Bayesian model-based inference**: For two-stage cluster sampling by PPSWOR, Bayesian frameworks account for uncertainty in cluster size (for units not sampled), providing integrated, design-aware posterior inference with efficiency gains over classical approaches [1710.00959].

Practical computation of joint inclusion probabilities and variances can rely on analytical approximations (e.g., Hájek’s, Berger-Sitter) or simulation; specialized software, such as the R package **TeachingSampling**, supports standard PPSWOR algorithms including Sunter’s method [2002.08009].

## 5. Large Deviations, Concentration, and New Theoretical Bounds

Recent advances provide sharp deviation inequalities for PPSWOR:

- **Martingale and pivotal methods**: The Deville–Tillé pivotal method generates the fixed-size sample as a vector-martingale, providing the basis for Freedman (Bennett-type) [2411.03955], Chernoff-type (Kullback-Leibler), and Hoeffding-Azuma concentration bounds for the number of sampled units falling in any subset \( A \) of the population:
  \[
  \Pr\left\{ \frac{|S \cap A|}{n} \geq a + \delta \right\} \leq \exp\left(-\frac{n\delta^2}{2(\eta + \delta/3)}\right)
  \]
  for subset weight \( a = \sum_{i \in A} w_i \), variance parameter \( \eta = a - n \sum_{i \in A} w_i^2 \), and sample size \( n \) [2411.03955]. These bounds provide quantitative guarantees for the concentration of PPSWOR samples and adapt classical hypergeometric tail estimates to the unequal-probability setting.
- **Relation to classical SRSWOR**: For equal weights, these new martingale-based bounds recover the established hypergeometric large deviations.

## 6. Applications in Experimental Design, Urn Models, and Combinatorial Optimization

PPSWOR is the de facto standard in a variety of domains:

- **Complex surveys and randomized experiments**: In cluster-randomized trials, PPSWOR at the cluster level (with further SRS within clusters) produces unbiased and location-invariant estimators of treatment effects, with robust variance properties [2002.08009].
- **Generalized urn models**: PPSWOR is realized as sequential sampling in two-color (or multi-color) Pólya–Eggenberger urn models, with arbitrary weight sequences. This framework unifies the classical hypergeometric case and allows both normal and non-normal (theta-function) limits, depending on weight structure [1003.1603].
- **Online algorithms and rounding**: In online bipartite (and stochastic) matching, PPSWOR serves as an efficient rounding scheme for fractional matchings, yielding competitive guarantees (≥0.707 in stochastic and ≥0.513 in adversarial models), and is essential for negative dependence and correlated selection properties [2410.06868].
- **Sketch-based data summarization**: Bottom-\( k \) and order-statistics-based PPSWOR constructs, extended to \( \ell_p \)-weighted objectives and signed data, underlie scalable randomized algorithms for large-scale machine learning and data analytics [2007.06744].

## 7. Extensions, Limitations, and Recent Algorithmic Advances

While classic PPSWOR achieves fixed sample size and size-proportional marginal probabilities asymptotically, exact proportionality cannot, in general, be simultaneously maintained with strict sample size constraints if some weights are large relative to \( n \) [2105.10809]. The EB-PPS algorithm introduces a controlled tradeoff: it enforces exact PPS in marginal probabilities, caps (but does not always fix) the sample size, and provides a simple amortized \( O(1) \)-time, one-pass streaming implementation. This method strictly generalizes fixed-size and pure-PPS sampling, yielding maximal expected sample size and minimal variance under an explicit sampling budget.

Modern sketch-based PPSWOR algorithms are composable, memory-efficient (\( O(n) \) space), and extend naturally to non-linear and signed weighting scenarios. These advances guarantee worst-case bias and variance bounds near those of classical PPSWOR, and in certain cases outperform with-replacement sampling in both estimator stability and representation diversity [2007.06744].

## References

- [1005.4107] On the inclusion probabilities in some unequal probability sampling plans without replacement
- [1003.1603] On Sampling without replacement and OK-Corral urn models
- [2105.10809] Exact PPS Sampling with Bounded Sample Size
- [2002.08009] The Benefits of Probability-Proportional-to-Size Sampling in Cluster-Randomized Experiments
- [1710.00959] Bayesian Inference under Cluster Sampling with Probability Proportional to Size
- [2411.03955] Large Deviations Inequalities for Unequal Probability Sampling Without Replacement
- [2410.06868] Online Matching Meets Sampling Without Replacement
- [2007.06744] WOR and \( p \)’s: Sketches for \( \ell_p \)-Sampling Without Replacement

Source: https://www.emergentmind.com/topics/probability-proportional-to-size-without-replacement-ppswor