Wyner's Lossy Common Information

• Wyner’s lossy common information is defined as the minimum common rate in the Gray–Wyner network required to achieve the joint rate-distortion limit under prescribed distortions.
• It leverages rate-distortion theory with optimized auxiliary variables, exhibiting distinctive regimes such as low-distortion plateaus and high-distortion conditions where shared descriptions become dominant.
• Explicit evaluations for sources like bivariate Gaussian and doubly symmetric binary models illustrate its operational significance and the tradeoff geometry in multimodal coding.

Wyner’s lossy common information is the distortion-constrained Gray–Wyner analogue of Wyner’s classical common information. For a correlated source pair $(X,Y)$ reconstructed subject to distortions $(D_1,D_2)$, it is the minimum rate on the common branch of the lossy Gray–Wyner network among all operating points that still achieve the minimum possible total transmit rate, namely the joint rate-distortion function $R_{X,Y}(D_1,D_2)$. In the modern single-letter formulation, the quantity is expressed as an infimum of $I(X,Y;U)$ over auxiliaries $U$ attached to rate-distortion optimal reconstructions and constrained by Wyner-type conditional independence. The lossy quantity reduces to ordinary Wyner common information at zero distortion, but its distortion dependence is nontrivial: low-distortion plateaus, high-distortion regimes in which the entire optimal description becomes common, and nonmonotone intermediate behavior all occur in explicit source families (Viswanatha et al., 2014, Xu et al., 2013).

1. Operational formulation in the lossy Gray–Wyner network

The operational setting is the lossy Gray–Wyner network. A memoryless pair $(X,Y)$ is encoded into a common message $S_0$ and two private messages $S_1,S_2$; decoder 1 reconstructs $\hat X^n$, decoder 2 reconstructs $\hat Y^n$, and the reconstructions must satisfy prescribed single-letter distortion constraints. In the formulation emphasized by Viswanatha, Akyol, and Rose, the lossy Gray–Wyner region $(D_1,D_2)$0 is the convex closure of all rate triples satisfying

$(D_1,D_2)$1

together with

$(D_1,D_2)$2

Its minimum sum rate is the joint rate-distortion function, so the plane

$(D_1,D_2)$3

is the lossy Pangloss plane (Viswanatha et al., 2014).

Wyner’s lossy common information is the minimum admissible common rate on that plane. One operational definition is

$(D_1,D_2)$4

and

$(D_1,D_2)$5

Equivalent notational variants appear in later papers: $(D_1,D_2)$6 in the lossy-source-coding interpretation of Xu, Liu, and Chen, and $(D_1,D_2)$7 in more recent Gray–Wyner formulations, but the role is the same—the smallest shared rate compatible with optimal joint coding (Viswanatha et al., 2014, Xu et al., 2013).

This operational definition is deliberately asymmetric with respect to common versus private description. The total rate is held at its absolute minimum, and only then is the common branch minimized. That is the precise lossy analogue of the classical lossless Wyner point on the Gray–Wyner boundary.

2. Single-letter characterization and equivalent formulations

The main single-letter theorem states

$(D_1,D_2)$8

where the infimum is over all joint distributions $(D_1,D_2)$9 satisfying

$R_{X,Y}(D_1,D_2)$0

and where $R_{X,Y}(D_1,D_2)$1 is rate-distortion optimal at $R_{X,Y}(D_1,D_2)$2, i.e.

$R_{X,Y}(D_1,D_2)$3

The interpretation is that $R_{X,Y}(D_1,D_2)$4 is a common latent variable extracted from a jointly optimal lossy reproduction pair; among all such latents that render the reconstructions conditionally independent, one minimizes the information revealed about the source pair (Viswanatha et al., 2014).

The optimizer has additional Markov structure. The cited characterization records

$R_{X,Y}(D_1,D_2)$5

or equivalently

$R_{X,Y}(D_1,D_2)$6

These factorization properties are useful computationally because they isolate the common branch from source-specific reconstruction structure (Viswanatha et al., 2014).

A complementary characterization, derived in the lossy source-coding interpretation of Xu, Liu, and Chen, expresses the same quantity through conditional rate-distortion functions: $R_{X,Y}(D_1,D_2)$7 subject to

$R_{X,Y}(D_1,D_2)$8

with

$R_{X,Y}(D_1,D_2)$9

The same work also gives an equivalent reproduction-based characterization $I(X,Y;U)$0 using a rate-distortion achieving pair $I(X,Y;U)$1 and proves

$I(X,Y;U)$2

At zero distortion the lossy quantity collapses to ordinary Wyner common information: $I(X,Y;U)$3 Thus the lossy object is a genuine extension, not a separate measure (Xu et al., 2013).

3. Distortion regimes and explicit source families

The bivariate Gaussian example is the canonical explicit evaluation. Let $I(X,Y;U)$4 be zero-mean, unit-variance jointly Gaussian with correlation coefficient $I(X,Y;U)$5, and let the distortion measure be mean-squared error. With $I(X,Y;U)$6, the lossy Wyner common information is

$I(X,Y;U)$7

Several qualitative features follow directly. In the low-distortion regime $I(X,Y;U)$8, the lossy quantity equals the lossless Wyner common information. In the highly distorted regime $I(X,Y;U)$9, all of the information is sent on the shared branch, so $U$0. In the intermediate regimes, the lossy common information is strictly larger than the lossless value. The same paper notes that the lossy common information is generally neither convex nor monotone in the distortions (Viswanatha et al., 2014).

The earlier lossy-source-coding interpretation had already identified a nontrivial distortion region in which the common rate is constant. Under mild regularity assumptions—$U$1, reproduction alphabets equal to source alphabets, and strictly positive single-letter distortions with zero self-distortion—there exists a strictly positive distortion surface $U$2 such that

$U$3

For a bivariate Gaussian pair with covariance

$U$4

the same work gives

$U$5

and states that in the low-distortion region

$U$6

one has

$U$7

For symmetric distortion $U$8,

$U$9

The doubly symmetric binary source exhibits an analogous low-distortion plateau, together with regions where $(X,Y)$0 and a region $(X,Y)$1 in which only the bounds

$(X,Y)$2

are known (Xu et al., 2013).

4. Tradeoff geometry and relation to other common-information notions

Lossy Wyner common information is one endpoint of a broader Gray–Wyner transmit–receive tradeoff. The relevant aggregate rates are

$(X,Y)$3

At minimum transmit rate,

$(X,Y)$4

the minimum receive rate is

$(X,Y)$5

and this point is the Wyner operating point $(X,Y)$6. At the opposite extreme, when

$(X,Y)$7

the minimum transmit rate is

$(X,Y)$8

which is the lossy Gács–Körner point $(X,Y)$9. The contour connecting these points provides a unified operational picture in which Wyner’s and Gács–Körner’s notions are distinct extremal projections of the same lossy Gray–Wyner geometry (Viswanatha et al., 2014).

A recent formulation sharpens the comparison by inserting the mutual information of the optimal lossy reconstructions between the two common-information notions: $S_0$0 Here $S_0$1 is lossy Gács–Körner common information, $S_0$2 is lossy Wyner common information, and $S_0$3 is a joint rate-distortion optimal reproduction pair. The upper bound is derived from the Wyner Markov constraints

$S_0$4

while the lower bound uses the lossy Gács–Körner Markov structure. Equality

$S_0$5

holds if and only if the sources admit a separable common-part representation

$S_0$6

with

$S_0$7

This identifies the precise structural regime in which the three quantities coincide (Andrade, 6 Jul 2025).

5. Gaussian structure, computation, and coding constructions

For jointly Gaussian sources with quadratic distortion, later work reframed lossy common information as a corner-point computation inside a general Gray–Wyner rate-distortion optimization. The weighted objective

$S_0$8

generates supporting hyperplanes of the Gaussian Gray–Wyner region. Setting

$S_0$9

selects Wyner’s common-information corner, yielding a unified computational recipe rather than a separate case analysis for each distortion regime (Chen et al., 2020).

A different Gaussian line of work uses canonical variables and weak stochastic realizations. For jointly Gaussian vectors, canonical-variable decomposition isolates correlated modes with canonical correlation coefficients

$S_1,S_2$0

In that representation,

$S_1,S_2$1

and the same value is obtained by the lossy Gray–Wyner common rate throughout the distortion region

$S_1,S_2$2

The optimal Gaussian auxiliary is achieved by a latent $S_1,S_2$3 with identity covariance in canonical coordinates, which realizes conditional independence of the two source blocks while minimizing $S_1,S_2$4 (Charalambous et al., 2020).

Explicit coding constructions are also available. Polar codes achieve the entire best-known lossless Gray–Wyner region for the doubly symmetric binary source and, in the low-distortion lossy regime, realize the fact that the common information remains equal to the lossless value. For Gaussian sources, polar lattices achieve the best-known lossy Gray–Wyner region in the bivariate case and show that extracting Wyner’s common information is equivalent to rate-distortion-optimal compression of a single Gaussian source: for two sources, the relevant scalar is

$S_1,S_2$5

and for $S_1,S_2$6 equicorrelated Gaussian sources it becomes

$S_1,S_2$7

In that $S_1,S_2$8-source model, the common information is

$S_1,S_2$9

These constructions turn the Gray–Wyner common branch from an existence theorem into an explicit extractable object (Shi et al., 2016).

6. Relaxations and learnable formulations

Several later developments study objects that are not identical to formal lossy Wyner common information, but are closely related through residual dependence or trainable Gray–Wyner architectures. A central relaxation is

$\hat X^n$0

with $\hat X^n$1. The lower-bound paper for classical Wyner common information uses this object as a relaxed common-information functional, and explicitly notes that it is conceptually similar to lossy settings because exact conditional independence is replaced by controlled residual dependence. The same relaxation underlies Common Information Components Analysis, where $\hat X^n$2 is interpreted as a resolution level or compression level; in the jointly Gaussian case, the resulting extraction procedure reduces exactly to CCA, with the top $\hat X^n$3 CCA components solving the relaxed common-information problem at the corresponding resolution (Sula et al., 2021, Gastpar et al., 2020).

Representation-learning work has turned Wyner’s optimization into trainable latent-variable objectives. The variational Wyner model is built around the constrained problem

$\hat X^n$4

with a shared latent $\hat X^n$5 and private latents $\hat X^n$6. That work explicitly states that it does not define a dedicated lossy common-information objective, but it cites the lossy source-coding interpretation of Xu et al. and adopts an objective with a clear rate–distortion flavor: distribution matching plus a common-information regularizer on the shared bottleneck (Ryu et al., 2019).

More directly, a recent learnable Gray–Wyner network uses the lossy quantity itself to characterize how much information should be placed in a common channel versus task-specific channels. In that formulation,

$\hat X^n$7

where the infimum is over $\hat X^n$8 such that

$\hat X^n$9

and

$\hat Y^n$0

The same work writes Gray and Wyner’s contour optimization as

$\hat Y^n$1

and, under deterministic encoder–decoder families, as the entropy objective

$\hat Y^n$2

A learnable Lagrangian then balances estimated rates and distortions, moving toward the Wyner endpoint when transmit rate is prioritized and toward the Gács–Körner endpoint when receive rate is prioritized. This suggests a direct operational migration of lossy common information from classical multiterminal information theory into learned codecs and task-conditioned representations (Andrade et al., 29 Jan 2026).

Wyner’s lossy common information therefore occupies a distinctive position. It is simultaneously a corner point of the lossy Gray–Wyner rate region, a latent-variable minimization over rate-distortion optimal reconstructions, a source of explicit Gaussian formulas and coding constructions, and a template for modern relaxed and learnable common-representation objectives. Its central organizing principle remains unchanged across these formulations: identify the smallest truly shared description compatible with optimal joint lossy coding.