Mixture of Geometric Distributions

• Mixture of Geometric Distributions is a family of discrete probability laws constructed by integrating geometric PMFs over a mixing measure, offering a unified framework for various stochastic models.
• It generalizes classical geometric and negative binomial distributions, with diverse applications including differential privacy, renewal processes, and overdispersed count modeling.
• Analytic tools such as probability generating functions and moment analysis provide insights into its structural properties, limit theorems, and explicit constructions for privacy mechanisms.

A mixture of geometric distributions is a family of discrete probability laws where the distribution of a random variable is constructed by integrating the probability mass function (PMF) of a geometric distribution over a mixing measure on the interval $(0,1)$. This structure arises naturally both as a unifying probabilistic framework in renewal and random-sum theories, as well as in contemporary applications such as differential privacy and flexible count modeling. Mixtures of geometric distributions generalize the geometric and negative binomial families and exhibit rich analytical, structural, and asymptotic properties.

1. Definition and Structural Properties

Let $\mu$ be a probability measure on $(0,1)$ or $[0,1]$. A random variable $X$ is called a $\mu$-mixture of geometric distributions if its PMF is given by

$$P\{X = k\} = \int_0^1 y\,(1-y)^k\,\mu(dy),\quad k \in \mathbb{Z}_{\geq 0}.$$

For $X$ supported on $\mathbb{Z}_{\geq 1}$, the convention is $P\{X=n\} = \int_0^1 x\,(1-x)^{n-1}\,\mu(dx)$ for $n \geq 1$. The variable $y$ (or $x$) is interpreted as the "success probability" parameter of the geometric law, drawn at random from the mixing measure $\mu$.

Key analytic objects for such mixtures include:

• Probability generating function (PGF):

$$G_X(z) = \int_0^1 \frac{y}{1 - (1-y)z}\, \mu(dy), \quad |z| < 1.$$

• Moments:

$\mathbb{E}[X] = \int_0^1 \frac{1-y}{y}\, \mu(dy).$

Higher moments and joint moments are similarly computable by standard techniques involving reordering the integrations.

• Renewal structure: If $\{N_j\}$ is an i.i.d. sequence with common law $P\{N_j = n\}$ as above, the corresponding renewal function $U(n) = \Pr(n \in T)$ (where $T$ is the set of renewal epochs) satisfies

$U(n) = \int_0^1 x^n\,\nu(dx),$

for a uniquely determined probability measure $\nu$ related to $\mu$ by an explicit transform (Enriquez et al., 2020).

2. Mixtures in Classical and Generalized Models

Mixtures of geometric distributions arise naturally as the limiting case or core analytical engine for several notable discrete distributions and stochastic models:

• Generalized Negative Binomial (GNB) distributions: For parameters $\alpha, r, \mu > 0$, the GNB law is defined via a mixed Poisson model with a generalized gamma (GG) mixing law:

$$P\{N_{r, \alpha, \mu}=k\} = \frac{1}{k!}\int_{0}^{\infty}e^{-z}z^k\,g^*(z; r, \alpha, \mu) dz,$$

where $g^*$ is the GG density (Korolev et al., 2017). For $0 < r \leq 1, 0 < \alpha \leq 1$, this distribution can be represented as a mixture of geometric laws:

$$P\{N_{r,\alpha,\mu} = k\} = \int_0^1 y\,(1-y)^k\, dF_{Y_{r,\alpha,\mu}}(y),$$

where $Y_{r,\alpha,\mu}$ is itself a nontrivial scale mixture of strictly stable and gamma distributions.

• Piecewise mixtures for differential privacy: The symmetric "piecewise mixture" of geometric laws is constructed by fusing two geometric PMFs, with exponential (geometric) decay rates $\epsilon$ and $r\epsilon$ inside and outside a cutoff $\pm c_t$, respectively (Smith et al., 2017). The resulting mixture has explicit expressions for its normalization, moments, and entropy, and is parametrized by three parameters ($\epsilon$, $r$, $c_t$), yielding closed-form control of privacy and utility measures.

3. Analytic and Limit Theorems

A central property of mixtures of geometric laws is their analytic tractability:

• Renewal process solution: For a renewal process with inter-arrival law a mixture of geometric distributions, the renewal function's generating series is

$$U(z) = \frac{1}{1 - G_X(z)} = \int_0^1 \frac{\nu(dx)}{1 - xz},\quad |z| < 1,$$

where $\nu$ is a measure explicitly determined by $\mu$ through a Stieltjes transform involution (Enriquez et al., 2020). For atomic $\mu$, $\nu$ is also atomic with one additional atom; when $\mu$ is absolutely continuous on intervals, so is $\nu$.

• Limit theorems for random sums: Let $X_j$ be i.i.d., and $N_{r,\alpha,\mu}$ be independent. Several regimes are established:
  • For light-tailed $X_j$ and suitable scaling of $\mu_n$, normalized random sums $\frac{1}{n} \sum_{j=1}^{N_{r,\alpha,\mu_n}} X_j$ converge to a GG law, which is itself a mixture of geometric laws for allowed parameter ranges.
  • If $X_j$ is in the domain of attraction of a stable law (one-sided or symmetric), the scaled sums converge to (gamma- or scale-) mixtures of stable or Linnik-type laws (Korolev et al., 2017).
  • In the central limit regime, normal scale-mixtures ("generalized variance-gamma" laws) arise as limits.
• Random Poisson Theorem: In mixed-binomial models (Bernoulli trials with random success probability), as the random parameter $Y_n \to 0$ appropriately, the distribution of the number of successes converges to a mixed-Poisson law under scaling (Korolev et al., 2017).

4. Explicit Constructions: Piecewise Mixtures

A particularly flexible class of mixture distributions arises in privacy-preserving mechanisms:

• Piecewise-Mixture Geometric Mechanism: The PMF is defined by

$$P\{X = k\} = \begin{cases} a_{2,g}\frac{\alpha_2-1}{\alpha_2+1} \alpha_2^{-|k|}, &|k|\leq c_t \ a_{1,g}\frac{\alpha_1-1}{\alpha_1+1} \alpha_1^{-|k|}, &|k| > c_t \end{cases}$$

where $\alpha_2 = e^{\epsilon}$, $\alpha_1 = e^{r\epsilon}$, with explicit normalization constants $a_{2,g}, a_{1,g}$. Moments such as $\mathbb{E}[|X|]$, variance, and entropy are provided in closed form.

• Parameter effects: $\epsilon$ controls inner decay, $r\epsilon$ outer decay, $c_t$ the core cutoff. For $r > 1$, the tails are lighter beyond the core, yielding better accuracy for bounded outputs while maintaining $\max\{\epsilon, r\epsilon\}$-differential privacy. If $r \to \infty$, the mechanism degenerates to a truncated law lacking differential privacy.
• Tail probabilities and accuracy: For $k > c_t$, $\Pr(|X| > k) = a_{1,g}\alpha_1^{-k}$ supplies explicit accuracy guarantees. For a batch of $K$ independent observations, the tail bound $$\Pr(\max_{1\leq i\leq K}|X_i| \geq T) \leq K a_{1,g} \alpha_1^{-T}$$ allows tuning of parameters to meet target misclassification rates or privacy budgets.

5. Applications and Connections

Mixtures of geometric distributions are prevalent in multiple domains:

• Differential privacy: Piecewise-mixture mechanisms provide increased flexibility, enabling practitioners to select three parameters to balance privacy loss and utility, compared to classical mechanisms with only one parameter (Smith et al., 2017).
• Stochastic processes and random polymers: In renewal theory, mixtures of geometric laws yield solvable classes with explicit representations for both discrete (renewal epochs) and continuous (exponential-mixture) cases and provide a rigorous basis for random polymer partition functions, including the case of generalized arcsine mixing measures and explicit free energy computations (Enriquez et al., 2020).
• Flexible count models in statistics: GNB laws have found empirical success in modeling count data with overdispersion unaccounted for by classical negative binomial laws, e.g., in modeling wet and dry spells in meteorology. The interpretation as a mixture of geometric distributions accounts for environmental variation in "success probability," matching observed overdispersion (Korolev et al., 2017).
• Limit theorems: The analytic and limit structure of mixtures of geometric distributions yields a wide range of tractable forms, including scale mixtures of strictly stable, Laplace, Linnik, and Mittag-Leffler laws, connecting discrete and continuous limit phenomena.

6. Special Cases and Explicit Measures

Several notable explicit cases arise:

• Generalized arcsine law: For $$\mu_v(dx) = \frac{\sin(\pi v)}{\pi} x^{-v}(1-x)^{v-1} dx,\, 0 < v < 1$$, the inter-arrival law and all objects of the associated renewal process admit closed forms. The tilted renewal measure $\nu_{v,\beta}$ and associated partition functions of the pinned polymer model are explicitly computable, with density and point-mass contributions that depend on $v$ and $\beta$ (Enriquez et al., 2020).
• Atomic mixtures: When $\mu$ is atomic, the resulting renewal measure $\nu$ is also atomic, allowing exact computation of moments and partition functions.
• GNB scale-mixing: The structural random variable for the mixing, $Y_{r,\alpha,\mu}$, admits nontrivial representations in terms of scale mixtures involving one-sided strictly stable distributions and independent gamma laws, enabling explicit probabilistic and computational work (Korolev et al., 2017).

7. Analytical and Applied Significance

Mixtures of geometric distributions provide fundamental analytical tractability (integral representations, closed-form moments), structural and limit universality (via analytic continuations, Stieltjes transforms, and involutive dualities), and substantial practical flexibility (for privacy, overdispersed count data, and solvable polymer models). The explicit description of all core properties in terms of the mixing measure establishes a unified analytical machinery that is extensible to a variety of domains (renewal theory, privacy, statistical modeling) and able to interpolate smoothly between classical geometric, negative binomial, and much more general discrete laws. This suggests that further applications and theory may emerge wherever heterogeneity in geometric-like success parameters is essential or analytically tractable.