Pairwise-Connected Distributions

• Pairwise-connected distributions are multivariate models with fixed univariate and bivariate marginals, providing a foundational structure for constraint-based analysis.
• They exhibit rich geometric, entropic, and dependence properties across domains such as statistical physics, network inference, and extreme value modeling.
• Methodologies like maximum-entropy estimation, information-diagram linear programming, and tree-structured graphical models offer practical frameworks for constructing and inferring these distributions.

A pairwise-connected distribution is any multivariate probability distribution constrained such that all one-dimensional marginals and all two-dimensional (bivariate) marginals—typically encoded as means and pairwise correlations, or in some generalizations, as mutual informations or tail dependencies—are fixed to specified target values. These families are central to statistical physics, network inference, extremal dependence modeling, high-dimensional nonparametric Bayes, and information theory. Despite the simplicity of the pairwise constraint, their properties exhibit rich geometric and functional structure, spanning highly constrained maximum-entropy models (such as the Ising exponential family), to highly degenerate constructions with logarithmic entropy, and extending to complex dependence structures in extreme value theory. Theoretical and algorithmic developments elucidate the range of entropic, structural, and computational phenomena that emerge in the study of pairwise-connected distributions.

1. Formal Definitions and Parametric Structure

Let $\boldsymbol{\sigma} = (\sigma_1, \ldots, \sigma_N)$ be $N$ binary or categorical variables. A distribution $P(\boldsymbol{\sigma})$ is pairwise-connected if it satisfies:

• $\mathbb{E}_P[\sigma_i] = \mu_i$ for all $i$ (fixed means),
• $\mathbb{E}_P[\sigma_i \sigma_j] = \nu_{ij}$ for all $i < j$ (fixed pairwise correlations), for some prescribed $\{\mu_i\}$ and $\{\nu_{ij}\}$ (Albanna et al., 2012).

In more abstract settings, the pairwise constraint may be defined via fixed bivariate marginals, prescribed mutual informations $I(X_i;X_j)$, or, in the context of extremal models, fixed upper-tail dependence coefficients $\lambda_{ij}$ (2208.02627, Martin et al., 2016).

Maximum-entropy models within the pairwise-connected class take the form of an exponential family:

$$P_{\mathrm{max}}(\boldsymbol{\sigma}) = \frac{1}{Z(\mathbf{h},\mathbf{J})} \exp\left[ \sum_i h_i \sigma_i + \sum_{i<j} J_{ij} \sigma_i \sigma_j \right],$$

where $Z$ normalizes the distribution, and $\{h_i, J_{ij}\}$ are Lagrange multipliers chosen to satisfy the constraints (Albanna et al., 2012).

In extremal graphical models, pairwise-connectedness can refer to prescribing all bivariate max-stable marginals, achieved via Markov trees or tree-structured graphical models in the extremes domain (2208.02627, Lalancette, 2023).

2. Entropy Structure and Support Constraints

The entropy of pairwise-connected distributions, subject only to first- and second-order constraints, ranges from $O(\log N)$ (minimum entropy) to $O(N)$ (maximum entropy) as $N\to\infty$, illustrating the vastness of the admissible solution space (Albanna et al., 2012). Specifically:

• The maximum-entropy (Ising) model achieves the upper bound, with entropy linear in system size.
• The minimum-entropy distribution must be supported on at most $1 + N + N(N-1)/2$ states, leading to an upper bound $$\widetilde{S}_2 \leq \log_2 [1 + N(N+1)/2] \sim 2 \log_2 N$$.
• Explicit constructions (prime-shifting, Hadamard arrays) demonstrate that $O(\log N)$ entropy is achievable, matching the support-size bound.

If additional symmetries (e.g., exchangeability) are imposed, the entropy grows linearly with $N$, closing the gap. Absent higher-order constraints, the pairwise specification leaves the entropy essentially unconstrained for large systems.

In the setting of multivariate Pareto or extremal graphical models, the pairwise interaction property leads to unique parametric structures (e.g., Hüsler–Reiss family) but does not substantially restrict entropy in the classical Shannon-theoretic sense (Lalancette, 2023).

3. Connection to Maximum-Entropy and Exponential Families

The principle of maximum entropy selects a unique distribution among all pairwise-connected distributions by maximizing

$$S[P] = -\sum_{\boldsymbol{\sigma}} P(\boldsymbol{\sigma}) \log_2 P(\boldsymbol{\sigma})$$

subject to the linear constraints above (Albanna et al., 2012). The resulting Ising exponential family is completely specified by the solution to a convex optimization problem over $\{h_i, J_{ij}\}$.

In contrast, if the constraints are given in terms of univariate entropies and pairwise mutual informations, the maximum-entropy distribution lacks a closed-form exponential family structure. Instead, it resides in a transcendental implicit family where the sufficient statistics depend on the unknown marginals:

$$\log p(\mathbf{x}) = \tilde\gamma + \sum_{i<j} \beta_{ij} \log p(x_i,x_j) - \sum_{i=1}^N \Bigl[\alpha_i + \sum_{j \neq i}\beta_{ij}\Bigr] \log p(x_i)$$

(Martin et al., 2016). Efficient computation is enabled by a linear program over the atoms of the information diagram representing shared and unique information.

4. Methodologies for Model Construction and Inference

Methodologies for constructing pairwise-connected distributions include:

• Moment-based exponential families: Solving for the Ising or generalized Potts parameters via maximum-likelihood or pseudolikelihood estimation (Feinauer et al., 2020).
• Information-diagram linear programming: Maximizing entropy with constraints on univariate entropies and pairwise mutual informations by linear programming over $2^N - 1$ information atoms (Martin et al., 2016).
• Extremal graphical models with tree dependence: Constructing Markov trees over a selected graph, ensuring that each edge reproduces a target bivariate extreme-value margin (2208.02627).
• Dependent nonparametric Bayes via shared-atom DPs: Realizing pairwise dependence among random densities by constructing specific dependence in Dirichlet process weights, sharing common global support (Hatjispyros et al., 2015).
• Optimal transport structures: Defining joint couplings for collections of distributions such that every pair achieves given optimal transport bounds, formalized as “pairwise multi-marginal OT” (Li et al., 2019).

Parameter estimation in the exponential family case is tractable and interpretable; with mutual information constraints or in nonparametric mixtures, algorithms rely on convex programming or Markov chain Monte Carlo sampling with slice augmentation.

5. Geometric and Measure-Theoretic Extensions

Beyond discrete variables, pairwise-connectedness generalizes to geometric and functional contexts:

• On spheres and homogeneous spaces, two sets are pairwise-connected if their distributions of Euclidean distances coincide. For example, on $S^n$ any two complementary sets of equal area have identical within-set distance distributions, and, more generally, the difference in such density functions depends only on the area difference (García-Pelayo, 2016).
• In high-dimensional optimal transport, pairwise-connectedness is reframed as the existence of a coupling of distributions such that each bivariate projection matches a prescribed transport cost up to distortion $r$ (Li et al., 2019). The existence, construction, and distortion bounds of such couplings depend sensitively on the geometry of the underlying space (e.g., snowflake metrics, ultrametrics, finite combinatorial structures).

These generalizations link pairwise-connectedness to advanced topics in geometry and probability, such as measure-preserving transformations, concentration of measure, and bi-Lipschitz embeddings.

6. Limitations, Diagnostics, and Implications

Pairwise-connected distributions, while parsimonious, may fail to capture relevant higher-order structure:

• In extreme value applications, tree-based models can only capture dependence along edges; the global goodness-of-fit can be assessed via the discrepancy $$D_T = \sum_{i<j, (i,j) \notin E} |\hat\lambda_{ij}^M - \hat\lambda_{ij}|$$ over non-edges (2208.02627).
• In entropy-based models, the excess entropy left unconstrained by pairwise information can be substantial unless additional symmetries or higher-order statistics are enforced (Albanna et al., 2012).
• In network inference, high values of the “explained” multi-information fraction suggest that a pairwise network model is adequate, but low values signal irreducible multi-way dependencies (Martin et al., 2016).

In nonparametric Bayes, the common-atom construction is theoretically sufficient for inducing pairwise dependence among random densities, without requiring more complex architectures; this enables efficient computation of $L_p$ distances and collapses the high-dimensional dependency structure to a manageable form (Hatjispyros et al., 2015).

A plausible implication is that, in high-dimensional applications, care must be taken in interpreting the validity or sufficiency of pairwise models—the presence of matching low-order statistics does not guarantee absence of high-order, synergistic, or combinatorially constrained structure (Albanna et al., 2012, Martin et al., 2016).

7. Applications and Domain-Specific Models

Pairwise-connected distributions are exploited in diverse fields:

• Statistical physics and neuroscience: The Ising maximum-entropy model underlies network inference from neural data; entropy gap analysis informs the detectability of collective behavior (Albanna et al., 2012, Feinauer et al., 2020).
• Extreme value theory: Tree-structured and graphical Pareto models encapsulate extremal dependence via pairwise margins, with the Hüsler–Reiss family uniquely characterizing pairwise-interaction graphical extremes (2208.02627, Lalancette, 2023).
• Machine learning and network reconstruction: Pseudolikelihood and hybrid energy-based models allow for efficient inference in high dimensions, with the explicit identification of pairwise terms and implicit absorption of higher-order interactions in learnable residuals (Feinauer et al., 2020).
• Optimal transport: Embedding probability measures with prescribed pairwise transport costs finds applications in locality-sensitive hashing and robust multi-distribution matching (Li et al., 2019).
• Nonparametric Bayesian inference: Random measure models with explicit pairwise dependence are deployed in density estimation and model selection for complex and exchangeable data (Hatjispyros et al., 2015).
• Geometric measure theory: The concept extends to distributions of pairwise distances in manifolds and stratified spaces, enabling structural inference from geometric or topological data (García-Pelayo, 2016).

These instances illustrate the fundamental relevance of pairwise-connected distributions as both a modeling assumption and a technical construct, providing a bridge between tractability and expressiveness across statistical and computational disciplines.