Action-Dependent Source Coding

• Action-dependent source coding is a framework where terminals actively select actions that affect side information availability and quality during encoding and decoding.
• The methodology employs a two-stage coding scheme combining action signaling and Wyner–Ziv refinement to optimize the rate–distortion–cost trade-off.
• Coordinated actions in sensor networks and multi-terminal systems demonstrate practical benefits in reducing compression rates under cost- and distortion-constrained environments.

Action-dependent source coding is a generalization of classical rate-distortion theory in which one or more terminals may actively select “actions” that affect the availability, quality, or nature of side information during the encoding or decoding process. Such actions influence the joint distribution from which potentially compressible data and side information are drawn, and typically incur a cost—motivating a fundamental three-way trade-off between compression rate, reconstruction distortion, and action cost. This framework subsumes classical scenarios such as Wyner-Ziv coding and Slepian-Wolf coding, while enabling novel operational regimes, particularly for systems such as sensor networks, adaptive measurement, and active learning, where information acquisition and actuation are intertwined with compression.

1. Formal Models and Problem Statements

The general model of action-dependent source coding consists of an i.i.d. source sequence $X^n \sim \prod_{i=1}^n P_X(x_i)$, alongside a sequence of actions $A^n$ taken at one or multiple terminals—commonly the decoder (0904.2311), encoder (Sabag et al., 2014), or both. Actions $A_i \in \mathcal{A}$ may be selected as functions of the observed or received data (possibly causally, non-causally, or in an adaptive fashion), and induce, via a channel $P_{Y|X,A}$, a side information sequence $Y^n$ that is used to aid lossy or lossless reconstruction. Each action $a$ incurs a per-symbol cost $\Lambda(a)$, subject to an average cost constraint $C$. Distortion constraints are as in classical settings.

For example, in the “vending machine” scenario (0904.2311), after receiving the message $M$, the decoder chooses $A^n = f(M)$; $A^n$0 becomes available via $A^n$1; and the reconstruction $A^n$2. The encoder-action variant allows $A^n$3 to be a function of $A^n$4, influencing both the side-information statistics and, indirectly, the achievable compression limits.

Multi-terminal generalizations (Chia et al., 2011) allow for multiple decoders, each with different action-dependent side-information channels, and possibly differing cost constraints or action strategies.

2. Rate–Distortion–Cost Characterizations

The core theoretical tool in action-dependent source coding is the single-letter characterization of the achievable rate–distortion–cost region. In the fundamental decoder-action model (0904.2311, Trillingsgaard et al., 2013), the minimal achievable rate for given average distortion $A^n$5 and average cost $A^n$6 is: $A^n$7 subject to: $A^n$8 where $A^n$9 deterministically, and the joint law is $A_i \in \mathcal{A}$0.

The first term $A_i \in \mathcal{A}$1 corresponds to the description rate needed to coordinate the actions between encoder and decoder; the second term $A_i \in \mathcal{A}$2 is the Wyner–Ziv term for refinement under action-dependent side information.

For the encoder-action model, the rate expressions generally differ, e.g. in the lossless case (0904.2311, Sabag et al., 2014): $A_i \in \mathcal{A}$3 Correlated-source and network extensions (Sabag et al., 2014) yield multi-dimensional rate regions dependent on the distribution of action and the network’s structure.

3. Coding Schemes and Achievability

The canonical coding scheme for action-dependent source coding is a two-stage (or layered) construction (0904.2311, Trillingsgaard et al., 2013, Kittichokechai et al., 2012):

1. Action code: The encoder transmits a description of the chosen action sequence at rate $A_i \in \mathcal{A}$4 (covering/typicality arguments).
2. Refinement code: Given $A_i \in \mathcal{A}$5, the side information $A_i \in \mathcal{A}$6 is available. The encoder uses Wyner–Ziv coding (random binning) to refine the source at rate $A_i \in \mathcal{A}$7 (conditional on actions).
3. The decoder reconstructs $A_i \in \mathcal{A}$8 from the received message, measures $A_i \in \mathcal{A}$9, and joint-typicality decodes $P_{Y|X,A}$0 for final reconstruction.

Practical code constructions based on LDGM codes and multiplexed arrays of Wyner–Ziv codes, as well as Blahut–Arimoto-type algorithms for numerically computing $P_{Y|X,A}$1, exist and have been shown to approach the theoretical bounds (Trillingsgaard et al., 2013).

In settings with in-block memory or causally-controllable side information, codetree-based representations merge the decision processes for actions and reconstruction and can be optimized analogously, using Markovian or block-wise dependencies (Simeone, 2013).

4. Special Cases, Extensions, and Examples

Classical settings are recovered as special cases:

• Wyner–Ziv problem: $P_{Y|X,A}$2 (no actions), reduces to $P_{Y|X,A}$3.
• Greedy action policies: Choosing the same “best” action for all symbols is generally sub-optimal, except in degenerate cases. Adaptive, source-dependent action selection strictly improves rate–cost–distortion trade-offs (0904.2311, Sabag et al., 2014).
• Multi-terminal/robust scenarios: The region for two decoders observing different channels, or in robust (Heegard-Berger-Kaspi) settings, involves expanded mutual information expressions and require base-layer/redundancy embedding (Ahmadi et al., 2011, Chia et al., 2011).
• Networked and cascade settings: In network source coding, actions modulate the correlation structure across links and terminals, and random linear network coding remains optimal for extending action-dependent joint statistics downstream (Sabag et al., 2014, Ahmadi et al., 2012).

Binary and Gaussian examples concretely demonstrate strict gains over separate source coding or time-shared policies, particularly for lossy source coding with constrained observation cost (0904.2311, Trillingsgaard et al., 2013).

5. Impact of Action Coordination and Cost Constraints

Coordinating the decoder’s actions with the encoder’s message (e.g., allowing the action sequence $P_{Y|X,A}$4 to be a function of the message) enables substantial rate reduction by conditioning action selection on the source (0904.2311, Trillingsgaard et al., 2013). Causal adaptation of actions to observed side information provides further reduction only within blocks in models with in-block memory; cross-block adaptation does not enlarge achievable regions (Simeone, 2013).

Cost constraints on actions induce a trade-off: higher action budgets allow selection of better side-information channels, decreasing the required source coding rate; conversely, when action cost is limited, the coding rate must increase to compensate for weaker side information (0904.2311, Trillingsgaard et al., 2013).

6. Multi-Terminal and Network Generalizations

The action-dependent framework generalizes to multi-terminal systems including multiple decoders and distributed sources (Chia et al., 2011, Sabag et al., 2014). When actions are undertaken at the decoder, the rate–cost region is determined by the worst-case decoder and the mutual information between sources, actions, and side information. When the encoder selects actions, parts of the information can “ride on” or be inferred via side information at other nodes, relaxing some rate constraints.

In networked settings, cut-set constraints are modified to account for action-coordaining information; random linear network coding is optimal for reliably multicasting the action-dependent joint sources given cut constraints that explicitly incorporate action information (Sabag et al., 2014).

7. Operational Significance and Applications

Action-dependent source coding enables unified management of sensing, measurement, and communication costs in resource-constrained environments, supporting active compression in sensor networks, compressed sensing with acquisition cost, and adaptive data acquisition (0904.2311, Sabag et al., 2014). The field offers explicit guidance and rate–cost trade-offs for when adaptive, source-aware action selection justifies its coordination overhead, and for the design of efficient schemes in both centralized and decentralized, networked, and cascade architectures. The paradigm exposes new design degrees of freedom beyond classical coding theory, where measurement and actuation co-design with compression is both necessary and beneficial.