Joint Rate Distortion Function (RDF)

• Joint RDF is defined as the infimum of mutual information between a correlated source tuple and its reconstruction under prescribed fidelity constraints.
• For jointly Gaussian sources, the problem admits a covariance-domain convex formulation and semidefinite-programming computation that can meet Gray’s lower bound exactly in specific regimes.
• Extensions to causal, multiterminal, and semantic settings reveal versatile applications that optimize compression strategies under various realizability and dependency constraints.

to=arxiv_search.search 天天乐购彩票json {"query":"(Stylianou et al., 2021) Joint Rate Distortion Function correlated multivariate Gaussian individual fidelity criteria", "max_results": 5} The joint rate–distortion function (joint RDF) is the infimum of the mutual information between a correlated source tuple and its reconstruction tuple under prescribed fidelity constraints, and it quantifies the minimum total coding rate required for joint lossy compression. For a pair $(X_1,X_2)$ with individual distortion limits $(\Delta_1,\Delta_2)$, the central object is

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$

where the admissible set fixes the $(X_1,X_2)$-marginal and enforces $\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$, $i=1,2$. In the multivariate jointly Gaussian, square-error setting, this optimization admits structural characterizations, a covariance-domain convex formulation, a semidefinite-programming computation, and closed-form regimes in which Gray’s lower bound is exact (Stylianou et al., 2021).

1. Definition, scope, and basic information-theoretic structure

For abstract source alphabets, let

$$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$

with reconstructions $\hat X_i$ taking values in $\hat{\mathbb X}_i$, and distortion functions

$$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$

The joint RDF with two individual distortion criteria is

$(\Delta_1,\Delta_2)$0

where $(\Delta_1,\Delta_2)$1 contains all joint laws $(\Delta_1,\Delta_2)$2 having fixed $(\Delta_1,\Delta_2)$3-marginal $(\Delta_1,\Delta_2)$4 and satisfying

$(\Delta_1,\Delta_2)$5

Equivalently, one may optimize over the forward test channel $(\Delta_1,\Delta_2)$6 subject to the same distortion constraints (Stylianou et al., 2021).

For Euclidean alphabets and quadratic fidelity,

$(\Delta_1,\Delta_2)$7

and

$(\Delta_1,\Delta_2)$8

The single-letter constraints become

$(\Delta_1,\Delta_2)$9

A recurring misconception is that the joint RDF is simply the sum of two individual RDFs. For correlated sources, the joint RDF is in general smaller than $$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$0, because the encoder exploits correlation. In a specific Gaussian distortion region, the identity

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$1

holds, matching Gray’s lower bound with equality (Stylianou et al., 2021).

2. Structural properties of optimal reconstructions and test channels

A central structural result for quadratic distortion is that optimal reconstructions can be restricted to conditional means. Define

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$2

Then

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$3

with equality if $$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$4 almost surely. In the Euclidean quadratic case,

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$5

for any measurable $$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$6. Hence the optimization can be restricted to reconstructions satisfying

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$7

This is the key conditional-mean structure of the optimal realization (Stylianou et al., 2021).

For jointly Gaussian sources with quadratic distortion, the minimizing joint law $$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$8 is jointly Gaussian. Accordingly, the optimal forward test channel is Gaussian, the optimal reproductions are jointly Gaussian with the sources, and the conditional means are linear. In stacked form,

$$R_{X_1,X_2}(\Delta_1,\Delta_2) = \inf_{\mathcal{M}(\Delta_1,\Delta_2)} I(X_1,X_2;\hat X_1,\hat X_2),$$9

the optimal realization can be parameterized as

$(X_1,X_2)$0

where $(X_1,X_2)$1, $(X_1,X_2)$2, and $(X_1,X_2)$3 is independent of $(X_1,X_2)$4 (Stylianou et al., 2021).

The nonanticipative extension retains an analogous structural emphasis but imposes causal factorizations. For a tuple of random processes with individual fidelity criteria, the joint NRDF minimizes a directed-information-type quantity over reproduction kernels satisfying a sequential nonanticipativity constraint, and the optimal test channel factors causally in time (Charalambous et al., 2021). Closely related sequence-based causal RDF formulations for $(X_1,X_2)$5 and $(X_1,X_2)$6 use the causal product kernel

$(X_1,X_2)$7

and define

$(X_1,X_2)$8

as the causal or realizable RDF (Stavrou et al., 2012).

3. Gaussian covariance-domain characterization and convex optimization

For the Gaussian specialization,

$(X_1,X_2)$9

and the error vector

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$0

has covariance

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$1

For jointly Gaussian $\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$2,

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$3

Therefore,

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$4

subject to the realization constraints and

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$5

When $\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$6, the realization constraints simplify to

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$7

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$8

with the additional semidefinite constraint

$\mathbb E[d_{X_i}(X_i,\hat X_i)]\le \Delta_i$9

Hence the feasible set reduces to

$i=1,2$0

This converts the Gaussian joint RDF into a convex optimization problem in the error covariance: $i=1,2$1 Convexity follows because $i=1,2$2 is convex on positive definite matrices, and the feasible set is defined by affine trace constraints and semidefinite inequalities (Stylianou et al., 2021).

An explicit semidefinite-programming form uses the selection matrices

$i=1,2$3

so that

$i=1,2$4

The computation then becomes a semidefinite program with log-det objective and linear matrix inequality constraints (Stylianou et al., 2021).

4. Closed-form regimes, canonical variables, and Gray-type equalities

A distinguished distortion region is the positive surface region

$i=1,2$5

In this region, the optimal error covariance is block-diagonal: $i=1,2$6 The distortions are therefore split equally among components, and the cross-error covariance vanishes. In this regime,

$i=1,2$7

which yields

$i=1,2$8

Thus Gray’s lower bound holds with equality there (Stylianou et al., 2021).

Outside that region, the optimal $i=1,2$9 can be full rather than block-diagonal, and the KKT structure is correspondingly more intricate. In the $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$0 numerical example of (Stylianou et al., 2021), the distortion pair $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$1 produces

$$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$2

consistent with the equal-allocation closed form, whereas the pair $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$3 yields a full non-block-diagonal optimal error covariance.

Canonical-variable methods sharpen this picture. In the canonical variable form, orthogonal transformations reduce the source covariance to

$$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$4

with $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$5 carrying canonical correlations. This makes explicit how each canonical correlated component contributes to the joint RDF (Stylianou et al., 2021). A later analysis of the same Gaussian problem uses Hotelling’s canonical variable form to derive an implicit characterization by a system of nonlinear equations and, in the symmetric-distortion case $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$6, an explicit representation in terms of two water-filling variables $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$7 and $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$8: $$X_1:\Omega\to\mathbb X_1,\qquad X_2:\Omega\to\mathbb X_2,$$9 with $\hat X_i$0 and $\hat X_i$1 piecewise determined by a water level $\hat X_i$2 satisfying $\hat X_i$3 (Stylianou et al., 22 Aug 2025).

5. Causal, multiterminal, and network generalizations

The joint RDF admits several non-equivalent generalizations once causality, common information, or decoder asymmetry is imposed. For sequence compression, the causal or realizable version restricts the joint conditional law to causal kernels and defines a sequence-level joint RDF

$\hat X_i$4

with the causal factorization

$\hat X_i$5

and a backward recursion for the optimal non-stationary reconstruction kernel (Stavrou et al., 2012). The nonanticipative formulation replaces mutual information by directed information and yields

$\hat X_i$6

together with a stationary closed form for the optimal causal Gibbs-type kernel (Charalambous et al., 2012).

In network source coding, the Gray–Wyner system introduces a common description $\hat X_i$7 and private reconstructions $\hat X_i$8. Its lossy region is described by

$\hat X_i$9

and a weighted joint RDF

$\hat{\mathbb X}_i$0

under the two distortion constraints. For jointly Gaussian sources with quadratic distortion, this weighted joint RDF has an explicit analytical form and recovers corner points such as Wyner’s lossy common information (Chen et al., 2020).

A different multiterminal generalization appears in the two-source Heegard–Berger problem with degraded reconstruction sets. There the rate–distortion function is a single-letter optimization over auxiliaries $\hat{\mathbb X}_i$1 and, in the common-reconstruction extension, $\hat{\mathbb X}_i$2, with the common description interpreted as $\hat{\mathbb X}_i$3 or $\hat{\mathbb X}_i$4. This formulation exposes when joint compression of $\hat{\mathbb X}_i$5 is strictly better than successive or separate compression (Benammar et al., 2015).

6. Computation, later developments, and contemporary uses

For finite alphabets, a general computational perspective treats the RDF as an entropy-regularized transport problem. In the Communication Optimal Transport formulation, one introduces slack variables $\hat{\mathbb X}_i$6 representing the reproduction marginal and solves

$\hat{\mathbb X}_i$7

subject to

$\hat{\mathbb X}_i$8

For joint discrete sources, the same structure applies verbatim once $\hat{\mathbb X}_i$9 and $$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$0 are replaced by joint alphabets, and the resulting model can be solved by an Alternating Sinkhorn algorithm with one-dimensional root-finding for the distortion dual variable (Wu et al., 2022).

A later vector-Gaussian analysis under individual component-wise quadratic distortion constraints emphasizes the role of the Hadamard lower bound and the semidefinite condition $$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$1. When that condition holds, the Hadamard rate is exact; when it fails, the optimal reconstruction covariance becomes singular and lower-dimensional reconstructions are essential. Within the scalable two-type correlation covariance framework, the probability of satisfying the semidefinite condition decays exponentially with source length, and explicit formulas quantify how correlations and distortion constraints trade off in the achievable compression rate (Chen et al., 6 Feb 2026).

The joint RDF framework has also been imported into semantic communication. For designed sources whose semantic object is a deterministic oracle allocation $$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$2, smooth concave utility together with deterministic common-category encoders reduces the semantic branch of the joint problem to the Shannon RDF of $$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$3; the SK exponential-tilting decoder specializes to conditional-mean decoding and the generalized Blahut–Arimoto iteration specializes to Lloyd–Max stationarity on $$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$4. When the second fidelity is aggregate verification rather than a monotone single-letter distortion, the joint problem leaves the single-letter admissible class and is characterized instead by a feasibility band

$$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$5

of width $$d_{X_i}:\mathbb X_i\times \hat{\mathbb X}_i\to [0,\infty),\qquad i=1,2.$$6 bits in partition cardinality (Armstrong, 9 Jun 2026).

Taken together, these developments identify the joint RDF as a family of closely related objects rather than a single formula. In the classical anticipative setting it is a mutual-information minimum under simultaneous fidelity constraints; in the Gaussian pair case it becomes a log-det semidefinite program with canonical-variable reductions and Gray-type equality regions; under nonanticipation it becomes a directed-information optimization tied to Bayesian filtering and source–channel matching; and in multiterminal or semantic settings it acquires auxiliary-variable, common-description, or feasibility-band structures. This suggests that the decisive modeling choice is not the presence of multiple sources alone, but which dependence, fidelity, and realizability constraints are imposed on the joint law of source and reproduction.