Rate-Distortion Linear Programming

• Rate-Distortion Linear Programming is an optimization framework that reformulates rate-distortion tradeoffs as linear programs to precisely quantify performance limits in networked systems with privacy and side-information constraints.
• It provides both achievable coding schemes and converse bounds, utilizing convex optimization to improve traditional methods in source coding and private information retrieval.
• The approach scales to large instances by integrating convex hull filtering and advanced solvers, enabling practical optimization for heterogeneous databases and privacy leakage scenarios.

Rate-Distortion Linear Programming (LP) is a collection of techniques for characterizing, bounding, or optimizing the fundamental rate-distortion tradeoffs in networked information systems by recasting the optimization as a finite or infinite-dimensional linear program. These LP methods provide both achievable schemes and converses for source coding scenarios—potentially with privacy or side-information constraints—yielding explicit or efficiently computable characterizations that subsume or improve on earlier bounds. Key applications include lossy private information retrieval with leakage constraints and multiterminal rate-distortion with heterogeneous, variable side information.

1. Principles of Rate-Distortion LP Formulation

At its core, rate-distortion LP involves encoding the tradeoff between communication rate (R), fidelity (distortion, D), and potentially privacy or leakage (L) as optimal solutions to a linear program over probability distributions, auxiliary variables, and distortion allocations.

In the archetypal single-server, lossy weakly-private information retrieval (LWPIR) scenario, for $M$ files of infinite length, the minimum achievable download rate $R^*(D, L; M)$ for average distortion $D$ and allowed maximal leakage $L$ is given by a convex optimization: $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$ subject to normalization over queries, leakage constraint $\max_{m, q} P_{Q|M}(q|m) \leq L$, and aggregate distortion constraint $$\frac{1}{M} \sum_{m,q} P_{M,Q}(m,q) D_q^{(m)} \leq D$$ (Yakimenka et al., 2021). $R_X(D)$ denotes the canonical scalar rate-distortion function for the file source.

LP methods can also be used to bound the rate-distortion function in distributed source coding settings featuring variable decoder side information (Unal et al., 2016).

2. Rate-Distortion-Leakage LP for Weakly-Private Retrieval

In lossy single-server LWPIR, the LP framework enables precise characterization of the tradeoff among rate, distortion, and privacy leakage. For infinite blocklengths ($B \to \infty$), the key optimization variables include the query distribution $P_{Q|M}(q|m)$ and per-file distortion allocations $R^*(D, L; M)$0, subjected to:

• Query normalization: $R^*(D, L; M)$1
• Leakage: $R^*(D, L; M)$2
• Average distortion: $R^*(D, L; M)$3

Two fundamental operational regimes emerge:

• No-leakage ($R^*(D, L; M)$4): $R^*(D, L; M)$5, resulting in $R^*(D, L; M)$6.
• No-privacy ($R^*(D, L; M)$7): queries fully reveal the user’s request, yielding $R^*(D, L; M)$8.

For finite $R^*(D, L; M)$9, explicit LPs over $D$0 and auxiliary variables $D$1 (linearizing the leakage constraint) determine the minimum rate for given $D$2, based on response-function analysis and convex hull filtering to reduce the dimensionality (Yakimenka et al., 2021).

3. LP Bounds in Rate-Distortion with Variable Side Information

In multiterminal rate-distortion with variable side information at decoders, LP methods supply both upper (achievability) and lower (converse) bounds tightly related to structured random binning, simultaneous decoding, and entropy submodularity.

• Achievability LP: For a source $D$3 and $D$4 decoders with side information $D$5, rates $D$6 (for auxiliary random variables $D$7) are chosen to jointly satisfy:
  • Each $D$8
  • Decoder $D$9 constraints for each $L$0: $L$1
  • Optimizing the sum-rate yields $L$2, whose lower-convex envelope characterizes the achievable rate (Unal et al., 2016).
• Converse LP: Introducing auxiliary set functions $L$3 on side-information marginals, LP constraints encode:
  • Initialization, nonnegativity, slope, monotonicity, and submodularity
  • Distortion constraints via single-decoder rate-distortion functions $L$4
  • This yields explicit lower bounds on $L$5, often tight when side-information structure is degraded or less noisy ordered.

4. Closed-Form and Explicit LP Solutions for Small Instances

For small database instances, the rate-distortion LP yields closed-form, piecewise-linear solutions. For example, with $L$6 files of one bit each under Hamming distortion:

• Enumerate all set partitions and corresponding response functions
• Compute $L$7 for each
• Retain convex hull vertices for query space $L$8
• The LP over $L$9, $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$0 (with $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$1) produces: $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$2 This mechanism readily generalizes to slightly larger $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$3, where symbolic solutions or commercial LP packages suffice (Yakimenka et al., 2021).

5. Scaling and Algorithmic Aspects for Large Instances

The space of all response functions grows super-exponentially with $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$4, necessitating scalable methods for practical databases. Techniques include:

• Restriction to product response-functions built from optimal compressors of small blocks (e.g., found for $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$5)
• Convex hull filtering (e.g., Quickhull) to prune to polytope vertices, reducing to order hundreds of queries
• Solving the pruned LP (thousands of variables) with industrial solvers (Gurobi, CPLEX) in seconds

On databases of $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$6 files, $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$7 bits ($$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$8), evaluated schemes include repetition of block-optimal LPs, nonconstructive random coding bounds, and a hybrid search plus LP approach. LP-based methods achieve within $$R^*(D, L; M) = \min_{P_{Q|M}, \{D_q^{(m)}\}} \sum_{q \in \mathcal Q} P_Q(q) \sum_{m=1}^M R_X(D_q^{(m)})$$9–$\max_{m, q} P_{Q|M}(q|m) \leq L$0 bits of the infinite-blocklength lower bound $\max_{m, q} P_{Q|M}(q|m) \leq L$1; repetition approaches are up to $\max_{m, q} P_{Q|M}(q|m) \leq L$2 bits looser (Yakimenka et al., 2021).

6. Conceptual Links to Index Coding and Submodularity

Rate-distortion LP approaches generalize and subsume submodularity-based index coding bounds by moving from cardinality and inclusion constraints (zero-error setting) to mutual information, distortion, and leakage formulations. The converse LP in multiterminal rate-distortion reduces to the Blasiak–Madden–Sridhar LP in lossless index coding, and similar entropy-submodularity constraints underlie both frameworks (Unal et al., 2016).

The “odd-cycle Gaussian” rate-distortion example demonstrates that LP bounds recover the precise minimum rate where prior minimax approaches do not: for $\max_{m, q} P_{Q|M}(q|m) \leq L$3 odd,

$\max_{m, q} P_{Q|M}(q|m) \leq L$4

This reveals the power and generality of rate-distortion LP methods, particularly in multiterminal or non-degraded side-information settings.

7. Extensions and Exactness Regimes

Rate-distortion LP frameworks extend to sources in vector-Gaussian, exponential family, and continuous domains via continuity arguments and Carathéodory-type bounds. Whenever side-information induces a degraded or less noisy structure, the LP achieves a single-letter minimizer and is tight. Combined algorithmic tactics include subgradient ascent over auxiliary distributions and interior-point LP solvers for the rate-distortion tradeoff, enabling application beyond the classic settings (Unal et al., 2016).

These LP methodologies establish a unified, computationally tractable scaffold for characterizing and optimizing rate-distortion tradeoffs—integrating achievable coding schemes, converses, and privacy-leakage constraints across classical, networked, and private information retrieval settings.