Rate Distortion Perception Function Overview

• RDPF is an extension of Shannon’s rate–distortion theory that incorporates a perceptual-quality constraint comparing source and reconstruction distributions.
• It characterizes the tradeoff among compression rate, distortion, and perceptual realism using mutual information optimization under both fidelity and distributional mismatch constraints.
• Applications span image, audio, and neural coding where achieving realistic reconstructions challenges classical distortion-only approaches.

Searching arXiv for recent and foundational papers on the rate–distortion–perception function and closely related extensions. The Rate Distortion Perception Function (RDPF) is the information-theoretic object that extends Shannon’s rate-distortion theory by imposing, in addition to a distortion constraint, a perceptual-quality constraint formulated as a discrepancy between the source distribution and the reconstruction distribution. In its standard form, the RDPF is

$R(D,P)= \min_{p_{\hat X|X} I(X;\hat X) \quad \text{s.t.}\quad \mathbb E[\Delta(X,\hat X)]\le D,\;\; d(p_X,p_{\hat X})\le P,$

where the optimization is over stochastic reconstruction channels, $\Delta$ is a fidelity criterion, and $d(p_X,p_{\hat X})$ measures perceptual mismatch at the distributional level rather than samplewise error (Blau et al., 2019). The function formalizes a three-way tradeoff among compression rate, distortion, and perceptual realism, and has since been developed in operational coding theorems, exact special-case characterizations, numerical algorithms, Gaussian and multiterminal extensions, and constructive achievability schemes (Theis et al., 2021).

1. Definition and conceptual scope

The classical Shannon rate-distortion function is

$R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$

The RDPF generalizes this by adding a perceptual-quality constraint: $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$ Here, distortion is a full-reference quantity comparing $X$ and $\hat X$, whereas perception is a distributional realism quantity comparing $p_{\hat X}$ and $p_X$ (Blau et al., 2019).

This formulation, introduced by Blau and Michaeli, treats perceptual quality as closeness between the distribution of reconstructions and the distribution of natural or source signals. Perfect perceptual quality corresponds to

$p_{\hat X}=p_X,$

equivalently $\Delta$0. Examples of admissible divergences mentioned in the literature include KL, Wasserstein, and total variation (Blau et al., 2019). A recurring theme across later work is that the perception constraint acts on the marginal law of the reconstruction, not on the conditional reconstruction channel directly (Theis et al., 2021).

Operationally, the RDPF is the minimum mutual-information rate required to satisfy both a fidelity budget and a distribution-matching budget. This distinguishes it from formulations that optimize only distortion or only realism. A plausible implication is that the RDPF should be understood not merely as a modification of rate-distortion theory, but as a distinct constrained optimization in which the admissible set of test channels is restricted by the output law they induce.

2. Structural properties and the basic tradeoff geometry

Under mild assumptions, the RDPF has the same variational flavor as the classical RDF but with a stricter feasible set. Blau and Michaeli prove that $\Delta$1 is monotonically non-increasing in both $\Delta$2 and $\Delta$3, and convex if the perception divergence is convex in its second argument (Blau et al., 2019). In particular, if $\Delta$4, then

$\Delta$5

and if $\Delta$6, then

$\Delta$7

The same work proves that, under a nondegeneracy condition on the distortion, the perfect-perception curve and the unconstrained Shannon curve do not coincide: $\Delta$8 This is the formal statement behind the often-cited “elevation” of the rate-distortion curve under perceptual constraints (Blau et al., 2019). The interpretation is precise: tightening perceptual quality generally requires either higher rate or higher distortion.

For squared-error distortion, an explicit upper bound connects perfect perception to classical distortion-rate tradeoffs: $\Delta$9 Equivalently, if a rate $d(p_X,p_{\hat X})$0 achieves distortion $d(p_X,p_{\hat X})$1 without a perception constraint, then the same rate can achieve perfect perceptual quality with distortion at most $d(p_X,p_{\hat X})$2 (Blau et al., 2019). This does not collapse the tradeoff; it quantifies one specific constructive route to exact marginal realism.

Later work sharpened the geometric interpretation by identifying critical transitions. When perception is measured by Wasserstein distance, there exist minimal transition functions $d(p_X,p_{\hat X})$3 and $d(p_X,p_{\hat X})$4 such that for $d(p_X,p_{\hat X})$5, the perceptual constraint is inactive and $d(p_X,p_{\hat X})$6, while for $d(p_X,p_{\hat X})$7, the distortion constraint is inactive and $d(p_X,p_{\hat X})$8 (Chen et al., 2024). This suggests that the RDP surface is naturally partitioned into regions where one or both constraints are active.

3. Operational meaning and the role of randomness

A major early question was whether the mutual-information expression defining the RDPF is operationally attainable by actual source codes. A coding theorem for the RDPF answers this affirmatively for stochastic, variable-length codes with shared randomness (Theis et al., 2021). In that setting, if

$d(p_X,p_{\hat X})$9

then

$R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$0

The coding construction relies on Li and El Gamal’s Poisson functional representation. Given a target conditional law $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$1, shared randomness allows the encoder and decoder to simulate the desired reconstruction channel while communicating at rate asymptotically equal to its mutual information, up to a one-shot additive overhead: $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$2 in the one-shot setting (Theis et al., 2021). The asymptotic version follows by applying the same mechanism to the product channel $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$3.

Subsequent work clarified that the role of randomness is more subtle than the original coding theorem might suggest. For Polish alphabets, under convexity and quantization assumptions, deterministic codes achieve the RDPF for all interior points

$R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$4

so that

$R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$5

in that regime (Chen et al., 2022). The same paper shows that randomness becomes essential only in certain extreme cases, most notably $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$6, where one imposes perfect perceptual quality.

That work also distinguishes weak-sense perfect perceptual quality, meaning $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$7 for each coordinate, from strong-sense perfect perceptual quality, meaning $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$8 (Chen et al., 2022). With common randomness, both notions have the same tradeoff $R(D) = \min_{p_{\hat{X}|X} I(X,\hat{X}) \quad \textrm{s.t.} \quad \mathbb{E}[\Delta(X,\hat{X})] \le D.$9, but with private randomness the strong version is governed by

$\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$0

subject to the stated Markov and marginal constraints (Chen et al., 2022). This is not a terminological nuance; it is an operational distinction about which forms of exact realism are achievable with which classes of coders.

4. Exact characterizations and closed-form special cases

Several exact RDPF characterizations are known for particular source models and perception metrics. The foundational closed-form example is the Bernoulli source under Hamming distortion and total variation perception (Blau et al., 2019). For $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$1, $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$2, Blau and Michaeli derive a piecewise formula for $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$3 that explicitly exhibits three regimes: a Shannon RD branch, an intermediate genuine RDP branch, and a zero-rate branch.

This scalar Bernoulli result was later extended to Bernoulli vector sources $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$4 with independent $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$5, additive Hamming distortion, and a componentwise marginal-distribution perception measure

$\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$6

The exact vector RDPF is

$\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$7

where $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$8 is the scalar Bernoulli RDPF (Vippathalla et al., 21 Jan 2025). The $\begin{aligned} R(D,P) = &\min_{p_{\hat{X}|X} \,\, I(X,\hat{X}) \ &\,\, \text{s.t.} \,\, \mathbb{E}[\Delta(X,\hat{X})] \le D,\,\, d(p_X,p_{\hat{X}}) \le P. \end{aligned}$9-plane splits into three regions $X$0: an RD region where perception is inactive, a zero-rate region, and an intermediate region where both budgets are jointly active. This exact decomposition also induces a graph-source formulation for inhomogeneous Erdős–Rényi models by treating edge indicators as Bernoulli coordinates (Vippathalla et al., 21 Jan 2025).

For scalar Gaussian sources, extensive theory exists under MSE distortion. Under a jointly Gaussian restriction, exact or analytically characterized formulas are available for direct KL, reverse KL, geometric Jensen-Shannon divergence, squared Hellinger distance, and squared Wasserstein-2 distance (Serra et al., 2023). For example, under scalar Gaussian source $X$1 and $X$2 perception, the stagewise RDPF is piecewise, with the active-perception region determined by

$X$3

and the classical Gaussian RDF recovered outside $X$4 (Serra et al., 2023).

A later paper generalized Gaussian perception analysis to the full $X$5-divergence family, still under a jointly Gaussian restriction, producing a jointly Gaussian RDPF upper bound $X$6 and showing that the perception equality reduces to a scalar nonlinear equation

$X$7

whose roots determine the reconstruction variance (Sourla et al., 23 Sep 2025). That paper recovers previously known cases such as KL, reverse KL, and Hellinger, and gives an explicit Pearson-divergence specialization (Sourla et al., 23 Sep 2025). This suggests that a substantial part of Gaussian RDP analysis can be reduced to a variance-allocation geometry shaped by the chosen divergence family.

5. Computational methods and algorithmic formulations

Beyond exact special cases, much of the RDPF literature concerns computation. For finite alphabets, one major line of work recasts the Wasserstein-perception RDPF as a Wasserstein barycenter-type optimization by introducing an auxiliary coupling $X$8 and a shared reconstruction marginal $X$9 (Chen et al., 2023). In discrete form, the optimization becomes

$\hat X$0

This lifted formulation turns the perception constraint into linear transport constraints and identifies $\hat X$1 as a barycenter-like reconstruction marginal balancing information/distortion and perceptual transport (Chen et al., 2023).

Because the transport block is not strictly convex, an entropy-regularized model adds

$\hat X$2

and the resulting alternating algorithm becomes Sinkhorn-like on the transport side and BA-like on the channel side (Chen et al., 2023). A related paper used the same barycenter perspective to characterize critical transition curves $\hat X$3 and $\hat X$4, and extended the computation to the distortion-rate-perception function (DRP), where distortion becomes the objective and rate a constraint (Chen et al., 2024).

A second major algorithmic line treats perception constraints given by $\hat X$5-divergences. For discrete memoryless sources, the RDPF remains a convex program, and the optimal conditional distribution takes the parametric form

$\hat X$6

where

$\hat X$7

encodes both distortion and perception penalties (Serra et al., 2024). The natural alternating-minimization scheme is implicit, so two implementable variants were proposed: a Newton-based Alternating Minimization (NAM) scheme for smooth divergences and a Relaxed Alternating Minimization (RAM) scheme for nonsmooth cases such as total variation (Serra et al., 2024). Both converge to globally optimal solutions under the stated necessary and sufficient conditions, and the paper provides sufficient conditions for exponential convergence in the number of iteration steps (Serra et al., 2024).

For Gaussian sources, computation proceeds differently. A multivariate Gaussian RDPF under MSE distortion and several perception metrics can be reduced to scalar Gaussian stagewise RDPFs along the eigenmodes of the covariance matrix, leading to a constrained allocation problem solved by block nonlinear Gauss-Seidel alternating minimization (Serra et al., 2023). In the Gaussian-process setting, the same philosophy appears in infinite dimension: the source covariance operator is diagonalized in the Karhunen–Loève basis, and the RDPF becomes an allocation across modes, yielding a perceptually constrained analogue of water-filling (Serra et al., 10 Jan 2025).

6. Extensions: Gaussian, secure, multiterminal, diffusion, and neural realizations

The RDPF has expanded well beyond its original point-to-point setting. For multivariate Gaussian sources under tensorizable distortion and perception metrics, the optimal jointly Gaussian solution resides in the eigenbasis of the source covariance matrix. The multivariate problem reduces to

$\hat X$8

where each $\hat X$9 is a scalar Gaussian stagewise RDPF (Serra et al., 2023). In the perfect realism regime $p_{\hat X}$0, the paper derives an analytical multivariate solution with per-mode distortions

$p_{\hat X}$1

which differs from classical Gaussian reverse water-filling and preserves all source components in the reconstruction (Serra et al., 2023).

For infinite-dimensional Gaussian processes, the same diagonalization principle holds in the Karhunen–Loève basis. The optimal reconstruction is itself Gaussian and shares the source covariance eigenspaces, reducing the Gaussian-process RDPF to a countable sum of scalar Gaussian RDPFs (Serra et al., 10 Jan 2025). Under proportional perceptual allocation, the resulting upper bound is tight in the perfect realism regime, and for stationary Gaussian processes on $p_{\hat X}$2 the asymptotic bound becomes an explicit frequency-domain integral involving the source spectral density (Serra et al., 10 Jan 2025).

Security has also been incorporated. In the secure noiseless setting, the exact region is characterized by

$p_{\hat X}$3

subject to $p_{\hat X}$4 and a realism constraint $p_{\hat X}$5 (Åhlgren et al., 22 Apr 2026). A striking conclusion is that, in the noiseless case, the common randomness required to satisfy realism already suffices for strong secrecy, so the secrecy constraint introduces no new rate inequality beyond $p_{\hat X}$6 (Åhlgren et al., 22 Apr 2026). Broadcast-channel and side-information variants show how secrecy modifies the common-randomness budget in more general settings (Åhlgren et al., 22 Apr 2026).

The multiterminal extension is the Gray–Wyner network. For correlated sources $p_{\hat X}$7, the first-order asymptotic optimal rate region with perception constraints is

$p_{\hat X}$8

where the private branches are governed by conditional rate-distortion-perception functions (Yang et al., 16 Jan 2026). This is the multiterminal analogue of the point-to-point RDPF: perception constraints do not destroy the Gray–Wyner decomposition but replace conditional rate-distortion functions by conditional RDPFs (Yang et al., 16 Jan 2026).

On the constructive side, diffusion-based traversal of the RDP surface has recently been proposed. A training-free framework combines reverse channel coding (RCC) to control rate with a score-scaled probability flow ODE decoder to control the distortion-perception tradeoff (Wang et al., 4 Mar 2026). In the scalar Gaussian case, the full scheme achieves the known information RDPF asymptotically, while in the multivariate Gaussian AWGN case the diffusion decoder is optimal for the distortion-perception frontier (Wang et al., 4 Mar 2026). This is not a new universal closed form for arbitrary sources, but it is a constructive realization mechanism with exact Gaussian optimality guarantees.

A further constructive direction uses neural lattice transform coders with shared dithering. That work argues that realizing RDP-optimal behavior in practice requires both efficient packing, implemented through lattice coding, and shared randomness, implemented through shared dithering (Lei et al., 21 Mar 2025). It analyzes infinite shared randomness and zero shared randomness settings and shows optimality in both cases for the studied scenarios (Lei et al., 21 Mar 2025). This suggests that, in neural compression, the RDPF is not merely a benchmark but also a design principle linking geometry, randomness, and perceptual realism.

In aggregate, these developments show that the RDPF has evolved from a point-to-point mutual-information optimization into a broader theory encompassing coding theorems, exact discrete and Gaussian solutions, numerical optimal transport and alternating-minimization methods, strong and weak perfect realism, multiterminal source coding, secrecy, diffusion-based achievability, and constructive neural codecs. A plausible implication is that future work will increasingly treat the RDPF not as an isolated function $p_{\hat X}$9, but as a family of rate regions and algorithmic realizations adapted to network structure, source geometry, and the chosen notion of perceptual similarity.