Rate-Distortion-in-Distortion (RDD) Theory

• Rate-Distortion-in-Distortion (RDD) is a framework that replaces the classical pointwise distortion with a Gromov-type distortion comparing pairwise metric structures.
• It minimizes mutual information I(X;Y) under a fidelity criterion that enforces preservation of internal distance structures, making it applicable to spaces with different dimensions and geometries.
• The framework employs computational techniques like alternating mirror descent to manage the quadratic complexity inherent in structural distortion evaluations.

Rate-Distortion-in-Distortion (RDD) denotes an extension of classical rate–distortion theory in which the usual expected pointwise distortion constraint is replaced by a Gromov-type distortion that compares pairwise distances in the source and reproduction spaces under a coupling induced by $P_{Y|X}$ (Chen et al., 13 Jul 2025). In this formulation, the objective remains the minimization of mutual information $I(X;Y)$, but the fidelity criterion measures preservation of metric structure rather than pointwise reconstruction accuracy. The framework is explicitly designed for source and reproduction spaces that may have different dimensions and different intrinsic geometries, and it is presented as both an informational and an operational rate–distortion function (Chen et al., 13 Jul 2025).

1. Definition and mathematical formulation

The RDD framework is stated for a source metric measure space $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$ and a reproduction metric space $(\mathcal{Y}, d_{\mathcal{Y}})$. The source random variable $X$ has law $P_X = P_{\mathcal{X}}$, the reproduction random variable $Y$ takes values in $\mathcal{Y}$, and the joint law is chosen through a conditional distribution $P_{Y|X}$, so that $P_{XY} = P_X \cdot P_{Y|X}$ (Chen et al., 13 Jul 2025).

The defining optimization problem replaces classical expected distortion by a distortion between distortions: $I(X;Y)$0 where $I(X;Y)$1, and $I(X;Y)$2 compares two distances rather than a source point and a reproduction point (Chen et al., 13 Jul 2025). The main case studied uses

$I(X;Y)$3

which yields

$I(X;Y)$4

This criterion is called Gromov-type distortion. Its central feature is that it does not require any direct metric between $I(X;Y)$5 and $I(X;Y)$6. Instead, it constrains the mismatch between the internal distance structures of the two spaces (Chen et al., 13 Jul 2025). A plausible implication is that the framework is naturally suited to settings in which pointwise fidelity is ill-posed but relational or geometric fidelity remains meaningful.

2. Relation to classical rate–distortion theory and Gromov–Wasserstein geometry

Classical rate–distortion theory minimizes mutual information subject to an expected distortion constraint of the form

$I(X;Y)$7

where $I(X;Y)$8 is a pointwise distortion measure (Chen et al., 13 Jul 2025). In RDD, the objective $I(X;Y)$9 is unchanged, but the fidelity constraint becomes pairwise and structural: $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$0 The paper explicitly characterizes this as a generalization of Shannon’s rate–distortion function in which the expected distortion constraint is replaced by the Gromov-type distortion (Chen et al., 13 Jul 2025).

The same structural quantity appears in the Gromov–Wasserstein (GW) distance. For a coupling $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$1, the Gromov-type distortion is

$(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$2

The squared GW distance is obtained by minimizing this expression over couplings (Chen et al., 13 Jul 2025). RDD therefore imports the GW structural distortion into an information-theoretic optimization, but with the additional objective of minimizing rate. The paper presents this as a structural rate–distortion theory situated between Shannon RD and Gromov–Wasserstein optimal transport (Chen et al., 13 Jul 2025).

A factual distinction from classical RD is dimensional flexibility. Because RDD compares only intrametric distances within $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$3 and within $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$4, it can be defined when the source and reproduction spaces have different dimensions or different types of metric structure (Chen et al., 13 Jul 2025). This suggests a direct relevance to point clouds, graphs, manifolds, and representation spaces in which a direct cross-space distortion is unavailable or unnatural.

3. Informational and operational interpretations

The RDD function is defined in the same informational form as a Shannon rate–distortion function: it is an infimum of mutual information under a fidelity constraint (Chen et al., 13 Jul 2025). The paper further states that encoding theorems substantiate its status as an operational RD function, not merely an informational one (Chen et al., 13 Jul 2025).

The coding theorem presented in the paper asserts that, for real numbers $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$5 and $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$6, the condition

$(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$7

is both necessary and sufficient for the existence of a sequence of encoders $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$8, decoders $(\mathcal{X}, d_{\mathcal{X}}, P_{\mathcal{X}})$9, and shared random variables $(\mathcal{Y}, d_{\mathcal{Y}})$0 such that

$(\mathcal{Y}, d_{\mathcal{Y}})$1

with $(\mathcal{Y}, d_{\mathcal{Y}})$2 independent of $(\mathcal{Y}, d_{\mathcal{Y}})$3, while the source and reconstruction vectors satisfy the Gromov-type distortion constraint with level $(\mathcal{Y}, d_{\mathcal{Y}})$4 for all $(\mathcal{Y}, d_{\mathcal{Y}})$5 (Chen et al., 13 Jul 2025).

The paper also gives an alternative interpretation through a two-dimensional source $(\mathcal{Y}, d_{\mathcal{Y}})$6 with independent copies of $(\mathcal{Y}, d_{\mathcal{Y}})$7, where the distortion measure becomes

$(\mathcal{Y}, d_{\mathcal{Y}})$8

In that construction, the resulting RD function coincides with $(\mathcal{Y}, d_{\mathcal{Y}})$9 (Chen et al., 13 Jul 2025). This places RDD inside standard source-coding logic while changing the notion of fidelity from pointwise approximation to structural preservation.

A related but distinct second-layer construction appears in “The Rate-Distortion Risk in Estimation from Compressed Data” (Kipnis et al., 2016). That work defines rate-distortion risk as the expected downstream inference loss under an RD-achieving distribution, which is another example of a task-level distortion induced by first-layer rate–distortion compression (Kipnis et al., 2016). The connection is conceptual rather than terminological: both formulations examine how classical RD behavior propagates into a more structured fidelity criterion.

4. Discrete formulation and alternating mirror descent

For computation, the paper studies the finite-alphabet case

$X$0

with source probabilities $X$1, conditional probabilities $X$2, and reproduction marginal

$X$3

The mutual information is then

$X$4

The discrete RDD problem becomes

$X$5

where

$X$6

and

$X$7

This distortion is quadratic in $X$8 and naively costs $X$9 per evaluation (Chen et al., 13 Jul 2025).

The paper reduces this complexity by decomposing the Gromov-type distortion into terms that can be evaluated through matrix multiplications, giving dominant cost $P_X = P_{\mathcal{X}}$0 per iteration (Chen et al., 13 Jul 2025). It then introduces a semi-relaxed formulation in which the marginal consistency constraint is temporarily relaxed; Theorem 2 states that the optimal solution of the semi-relaxed problem is also the optimal solution of the original RDD problem (Chen et al., 13 Jul 2025).

Because the RDD function cannot be solved analytically due to the high computational complexity associated with Gromov-type distortion, the paper develops an alternating mirror descent (AMD) algorithm using decomposition, linearization, and relaxation techniques (Chen et al., 13 Jul 2025). The $P_X = P_{\mathcal{X}}$1-update is an exponential-weighting step of the form

$P_X = P_{\mathcal{X}}$2

where $P_X = P_{\mathcal{X}}$3 and $P_X = P_{\mathcal{X}}$4 encode linearized contributions of the structural distortion, while the $P_X = P_{\mathcal{X}}$5-update is simply

$P_X = P_{\mathcal{X}}$6

The resulting procedure is reminiscent of entropy-regularized iterative updates, but its target constraint is the quadratic four-index Gromov-type distortion rather than classical expected distortion (Chen et al., 13 Jul 2025).

For comparison, classical RD and distortion-rate functions on finite alphabets admit Blahut–Arimoto-type procedures and constrained variants that directly target a prescribed distortion or rate (Chen et al., 2023). In the RDD setting, the computational difficulty comes from the structural distortion term rather than from the mutual-information objective.

5. Mixed constraints and empirical studies

The numerical experiments in the paper consider classical sources on uniform grids, including Gaussian, Laplacian, and uniform sources, with Euclidean $P_X = P_{\mathcal{X}}$7 distances and grid truncation to $P_X = P_{\mathcal{X}}$8 with $P_X = P_{\mathcal{X}}$9 (Chen et al., 13 Jul 2025). The experiments also include settings where the source and reproduction spaces have different dimensions, as well as point sets on a circle in $Y$0D and a sphere in $Y$1D, illustrating that the RDD formulation remains well-defined beyond same-dimensional Euclidean grids (Chen et al., 13 Jul 2025).

A further extension studied in the paper combines classical distortion and distortion-in-distortion through

$Y$2

with $Y$3 (Chen et al., 13 Jul 2025). The corresponding AMD update adds an extra factor

$Y$4

to the exponential-weighting rule (Chen et al., 13 Jul 2025). This fused formulation is motivated by fused Gromov–Wasserstein structure and interpolates between classical RD and purely structural RDD.

The reported observations are qualitative rather than asymptotic optimality claims. The AMD algorithm is described as showing stable convergence behavior on the tested discrete sources (Chen et al., 13 Jul 2025). In the mixed-constraint experiments, for fixed $Y$5, increasing $Y$6 increases the required rate $Y$7, and for fixed $Y$8, decreasing $Y$9 increases the rate (Chen et al., 13 Jul 2025). The paper also notes that, although the Gromov-type term does not come with a convexity guarantee, the simulations show monotonic behavior of $\mathcal{Y}$0 in both $\mathcal{Y}$1 and $\mathcal{Y}$2 (Chen et al., 13 Jul 2025). This suggests that structural preservation materially alters optimal coding behavior even when a classical pointwise distortion is retained.

6. Terminological ambiguity and adjacent uses of “RDD”

The acronym “RDD” is not unique in current information-theoretic literature. In “Rate-Distortion Dimension of Stochastic Processes,” RDD denotes rate–distortion dimension, defined for a stationary process $\mathcal{Y}$3 under squared-error distortion by

$\mathcal{Y}$4

when the limit exists (Rezagah et al., 2016). That paper proves, under stated regularity conditions, that the RDD of a stationary process equals its information dimension, thereby giving an operational interpretation to information dimension through low-distortion asymptotics of the rate–distortion function (Rezagah et al., 2016).

In a different line of work, “RDD: Pareto Analysis of the Rate-Distortion-Distinguishability Trade-off” uses RDD to mean rate–distortion–distinguishability, a three-way trade-off between rate, reconstruction distortion, and the distinguishability between compressed normal and anomalous signals (Enttsel et al., 29 Sep 2025). There the optimization augments classical RD with either an anomaly-agnostic or anomaly-aware distinguishability constraint, and the object of study is a Pareto surface in $\mathcal{Y}$5 space rather than a Gromov-type distortion (Enttsel et al., 29 Sep 2025).

These uses are conceptually adjacent but mathematically distinct. Rate–distortion dimension concerns small-$\mathcal{Y}$6 asymptotics of $\mathcal{Y}$7 for analog processes (Rezagah et al., 2016); rate–distortion–distinguishability concerns detection performance under compression (Enttsel et al., 29 Sep 2025); rate-distortion risk concerns inference loss under RD-achieving compression (Kipnis et al., 2016); and Rate-Distortion-in-Distortion concerns structural fidelity via distortions between distances (Chen et al., 13 Jul 2025). The overlap of acronyms is therefore a source of potential confusion rather than an indication of a shared formalism.

7. Limitations, scope, and open directions

The paper explicitly identifies computational complexity as the principal limitation of the RDD framework. Even with decomposition and AMD, the optimization remains computationally heavy for large-scale problems (Chen et al., 13 Jul 2025). The Gromov-type distortion is quadratic in the coupling and is not convex in general, so standard convex-analytic properties of classical RD do not immediately transfer (Chen et al., 13 Jul 2025).

Theoretical gaps are also stated directly. The paper notes that classical RD has strong convexity and continuity properties, whereas analogous properties for RDD—such as convexity in $\mathcal{Y}$8, differentiability, and explicit parametric forms for Gaussian sources—are mostly open (Chen et al., 13 Jul 2025). The experiments are mainly on discrete classical sources, grids, and spherical point sets, so real-world validations for point clouds, graphs, and representation-learning pipelines remain to be carried out (Chen et al., 13 Jul 2025).

Within the broader rate–distortion landscape, the RDD proposal belongs to a larger pattern of extending fidelity criteria beyond standard pointwise losses. For example, rate–distortion under an $\mathcal{Y}$9-insensitive distortion measure replaces absolute or squared error by a dead-zone loss and changes both the Shannon lower bound analysis and the shape of $P_{Y|X}$0 (Watanabe, 2013). A plausible implication is that RDD should be understood as one member of a family of nonclassical rate–distortion theories in which the fidelity criterion is tailored to structural, geometric, or task-level properties rather than direct reconstruction error.

The distinctive claim of Rate-Distortion-in-Distortion is therefore precise: the rate is still the mutual information $P_{Y|X}$1, but the distortion is a distortion of distances, and the resulting object is intended to quantify the minimum rate required to preserve metric structure up to a prescribed Gromov-type tolerance across spaces that need not share the same geometry or dimension (Chen et al., 13 Jul 2025).