---
title: Implicit Channel Capacity
url: https://www.emergentmind.com/topics/implicit-channel-capacity
type: topic
---

# Implicit Channel Capacity

Searching arXiv for relevant papers on implicit channel capacity and related capacity formulations.
Implicit channel capacity denotes a class of capacity notions in which the communication medium is not treated merely as an explicit, analytically tractable transition law. In the cited literature, this occurs in two principal ways. First, the channel may be *implicit* because only input-output samples are available, so capacity must be estimated and optimized without explicit probability density functions. Second, communication may be *implicit* because it is embedded in another dynamical system, such as an LQG control loop, where messages are conveyed through control actions while a primary control objective must still be met [2107.03084] [2509.16146]. Closely related work on finite-state channels, input constraints, fading-memory channels, and computability provides the surrounding theory for lower bounds, dual characterizations, universal achievability, and impossibility results [2001.03423] [0807.0042] [1202.0417] [1601.06101].

## 1. Definitions and formal scope

For a memoryless vector channel, the basic capacity problem is
\[
C = \max_{p_X(\mathbf{x})} I(X;Y),
\]
with the difficulty that, for most channels, one must both compute \(I(X;Y)\) and maximize it over input distributions [2107.03084]. In the sample-based setting, the channel law may be unknown or highly complex, and only paired input-output observations are assumed available.

For input-driven finite-state channels, the state evolves as a deterministic, time-invariant function of the previous state and the current input,
\[
s_t = f(s_{t-1}, x_t),
\]
and the multi-letter capacity with known initial state is
\[
C = \lim_{N \to \infty} \max_{P(x^N \mid s_0)} \frac{1}{N} I(X^N; Y^N \mid s_0).
\]
This formulation captures channels whose memory is embedded in the state induced by the input constraint [2001.03423].

For implicit communication in LQG control, the relevant object is not an unconstrained mutual-information maximization. Instead, the controller transmits messages through the plant while keeping the average quadratic control cost within a slack \(V \ge 0\) above the optimal LQG cost:
\[
J_n \leq J_n^* + V.
\]
The corresponding implicit channel capacity is
\[
C(V) = \sup\left\{ R \;|\; \exists\ \text{admissible}\ (2^{nR}, n)\ \text{codes with}\ P_e^{(n)} \to 0,\ J_n \leq J_n^* + V \right\}.
\]
This definition makes the control-communication trade-off part of the capacity notion itself [2509.16146].

| Setting | Capacity expression | Defining feature |
|---|---|---|
| Sample-based implicit channel | \(C = \max_{p_X(\mathbf{x})} I(X;Y)\) | Capacity learned from samples |
| Input-driven FSC | \(\lim_{N\to\infty}\max_{P(x^N\mid s_0)} \frac{1}{N} I(X^N;Y^N\mid s_0)\) | State driven deterministically by input |
| LQG implicit communication | \(C(V)\) under \(J_n \le J_n^* + V\) | Communication through control actions |

## 2. Sample-based capacity learning for implicit channels

A central modern formulation treats an implicit channel as a black-box memoryless channel for which explicit densities are unavailable, but input-output samples can be generated. The main obstacles are precisely the two classical tasks in capacity analysis: estimating the mutual information and optimizing it over the input law [2107.03084].

The paper "Discriminative Mutual Information Estimators for Channel Capacity Learning" introduces the Discriminative Mutual Information Estimator (DIME). The estimator uses discriminator networks, as in adversarial learning, to estimate the density ratio needed for mutual information from paired samples drawn from the joint law \(p_{XY}\) and unpaired samples drawn from the product of marginals \(p_X p_Y\). In the indirect version, i-DIME, a GAN-type discriminator separates dependent and independent input-output pairs, and the optimal discriminator satisfies
\[
D^*(\mathbf{x},\mathbf{y}) =
\frac{p_X(\mathbf{x}) p_Y(\mathbf{y})}
{p_{XY}(\mathbf{x},\mathbf{y}) + p_X(\mathbf{x}) p_Y(\mathbf{y})},
\]
so that the density ratio is recovered as
\[
\frac{p_{XY}(\mathbf{x},\mathbf{y})}{p_X(\mathbf{x}) p_Y(\mathbf{y})}
=
\frac{1 - D^*(\mathbf{x},\mathbf{y})}{D^*(\mathbf{x},\mathbf{y})}.
\]

The direct version, d-DIME, replaces the GAN-type objective with an \(\alpha\)-parameterized value function for which the optimal discriminator is proportional to the density ratio:
\[
D^*(\mathbf{x},\mathbf{y}) = \alpha \cdot \frac{p_{XY}(\mathbf{x},\mathbf{y})}{p_X(\mathbf{x}) p_Y(\mathbf{y})}.
\]
The corresponding mutual-information estimator can be written either as an expectation under the joint law or through the optimal value of the discriminator objective:
\[
I_{dDIME}(X;Y) = \frac{\mathcal{J}_\alpha(D^*)}{\alpha} + 1 - \log(\alpha).
\]
The paper emphasizes that these constructions require no explicit channel model; sample pairs suffice [2107.03084].

Capacity optimization is then handled by the cooperative framework CORTICAL. A generator \(G\) produces candidate channel inputs, and a discriminator \(D\) estimates the mutual information for the induced input law. The optimization objective is
\[
\mathcal{J}_\alpha(G, D)
=
\alpha \mathbb{E}_{p_Z} \left[ \log D(G(\mathbf{z}), H(G(\mathbf{z}))) \right]
-
\mathbb{E}_{p_Z} \left[ D(G(\mathbf{z}), \pi(H(G(\mathbf{z})))) \right],
\]
with capacity represented as
\[
C = \max_G \max_D \frac{\mathcal{J}_\alpha(G, D)}{\alpha } + 1 - \log (\alpha).
\]
Simulation results on AWGN channels show that d-DIME achieves accuracy on par with or exceeding MINE, with generally lower variance, especially in higher dimensions; in continuous and discrete settings, the generator learns Gaussian inputs and capacity-approaching constellations such as standard PSK constellations [2107.03084].

## 3. Finite-state structure, reverse directed information, and constrained optimization

For channels with memory, explicit capacity computation often shifts from direct optimization of \(I(X^N;Y^N)\) to lower bounds, dynamic programming, and dual formulations. In input-driven finite-state channels, a key lower bound uses reverse directed information:
\[
I(X^N; Y^N) = I(Y^N \to X^N) + I(X^{N-1} \to Y^N),
\]
hence
\[
I(X^N; Y^N) \ge I(Y^N \to X^N).
\]
This leads to the lower bound
\[
C \geq \sup_{\{P(x_t \mid s_{t-1})\}}
\liminf_{N \to \infty}
\frac{1}{N} \sum_{t=1}^N I(X_t, S_{t-1}; Y_t \mid X^{t-1}),
\]
which can be cast as the average reward of an infinite-horizon dynamic program whose state is the belief \(\beta_{t-1}(s_{t-1}) = P(s_{t-1}\mid x^{t-1})\), action is \(P(x_t \mid s_{t-1})\), and reward is \(I(X_t,S_{t-1};Y_t\mid x^{t-1})\) [2001.03423].

The Bellman equation is solved explicitly for several runlength-limited channels. For \((d,\infty)\)-RLL \(\mathrm{BSC}(p)\),
\[
C_{d,\infty}^{\mathrm{BSC}(p)} \geq \max_{a \in [0,1]}
\frac{h_b(ap + (1-a)(1-p)) - h_b(p)}{ad + 1}.
\]
For \((d,k)\)-RLL \(\mathrm{BEC}(\epsilon)\),
\[
C_{d,k}^{\mathrm{BEC}(\epsilon)} \geq C_{d,k}\cdot (1-\epsilon),
\]
where \(C_{d,k}\) is the noiseless \((d,k)\)-RLL capacity expressed by an optimization over \(a_d,\ldots,a_{k-1}\). The same work also gives a single-letter lower bound based on Markov input processes constructed from a finite \(\mathscr{V}\)-graph:
\[
C \ge I_Q(X;Y\mid S,V)
\]
for connected input-driven FSCs with irreducible and aperiodic Markov input distributions [2001.03423]. For \(k<\infty\), the resulting DP-based lower bounds recover classic Markov-bound lower bounds such as those of Zehavi and Wolf, while extending them to general parameters.

A complementary line of work handles *input constraints* through the Lagrangian Converse Proof. Instead of naively maximizing \(I(X;Y)\) over a constrained set, the capacity is characterized by the minimum of a Lagrange dual function,
\[
C = \min_{\lambda \ge 0} L(\lambda,\rho_0), \qquad
L(\lambda,\rho_0)=\max_{p_X}\big[I(X;Y)-\lambda(\mathbb{E}[\alpha(X)]-\rho_0)\big].
\]
For convex problems this equals the standard constrained maximization. For non-convex problems it can be strictly larger than the naive constrained expression because time-sharing is then essential, and the dual formulation incorporates that automatically [0807.0042].

## 4. Biological signal transduction as an implicit communication channel

An influential biochemical example is the BIND channel, a discrete-time finite-state Markov channel modeling ligand-receptor signal transduction. The receptor state and the output coincide, with states \(U\) (unbound) and \(B\) (bound), and the input alphabet is a finite set of ligand concentrations \(\{c_L,c_2,\ldots,c_{m-1},c_H\}\) [1411.1650].

The transition matrix for input concentration \(c_j\) is
\[
\mathbf{P}_{Y|X = c_j} =
\begin{bmatrix}
1 - \alpha_j & \alpha_j \\
\beta & 1-\beta
\end{bmatrix},
\]
where \(\alpha_j = k_+ c_j \Delta t\) depends on the input when the receptor is unbound, while \(\beta = k_- \Delta t\) is the unbinding probability and is independent of the input when the receptor is bound. This asymmetry is decisive: only the unbound state is input-sensitive.

For IID inputs, the stationary probability of the unbound state is
\[
p_Y(U) = \frac{1}{1+\bar{\alpha}_p/\beta},
\qquad
\bar{\alpha}_p = \sum_{j=1}^m \alpha_j p_j,
\]
and the per-step mutual information is
\[
\mathcal{I}(X;Y)
=
\frac{H(\bar{\alpha}_p)-\sum_{j=1}^m p_j H(\alpha_j)}
{1+\bar{\alpha}_p/\beta}.
\]
The capacity-achieving input distribution is IID and places all probability on the minimum and maximum concentrations:
\[
C = \max_{p_H}
\frac{H(p_L\alpha_L+p_H\alpha_H)-[p_LH(\alpha_L)+p_HH(\alpha_H)]}
{1+(p_L\alpha_L+p_H\alpha_H)/\beta}.
\]
The proof that feedback does not increase capacity exploits the fact that the BIND channel is a unit-output-memory finite-state channel and uses results of Chen and Berger. An earlier arXiv version states the same central result in binary-input form: the capacity-achieving input distribution is iid, and feedback does not increase the capacity [1305.2245] [1411.1650].

In the continuous-time limit of short time steps and rapid ligand release, the BIND channel approaches Kabanov’s Poisson channel, with per-unit-time capacity
\[
C_{\mathrm{Kabanov}(c)} =
\frac{(c+1)^{1+1/c}}{e} -
\left(1 + \frac{1}{c}\right)\ln(c+1).
\]
This connects a biologically realistic refractory-state model to a classical continuous-time information-theoretic limit [1411.1650].

## 5. Implicit communication in LQG control systems

The most explicit recent formulation of implicit channel capacity appears in linear quadratic Gaussian control. Here the control system itself serves as the communication channel: the controller transmits messages through the control inputs, and the receiver observes the resulting system state, possibly noisily. Communication is implicit because no explicit communication channel is used, and because information transmission occurs while the controller simultaneously maintains the control cost within a prescribed level [2509.16146].

The plant evolves as
\[
x_{t+1} = Ax_t + Bu_t + w_t,
\]
with \(w_t \sim \mathcal{N}(0,\Psi_w)\), and the control cost is
\[
J_n = \frac{1}{n}\mathbb{E}\!\left[
\sum_{t=1}^{n}(x_t^\top F x_t + u_t^\top G u_t) + x_{n+1}^\top F x_{n+1}
\right].
\]
The central trade-off is between reliable communication rate and excess control cost \(V\).

When both controller and receiver have noiseless state observations, the implicit channel capacity admits the closed form
\[
C(V) = \frac{1}{2}\sum_{i=1}^{d_1}\log\!\left(1+\frac{\phi_i}{\lambda_i}\right),
\]
where \(\lambda_i\) are the eigenvalues of \(\Psi_w\), \(\phi_i\) depend on the allowable control loss through a waterfilling-type condition, and the capacity-achieving input policy is
\[
u_t = -Kx_t + B^\dagger s_t,
\]
with \(s_t \sim \mathcal{N}(0,\Phi)\) independent of \(x_t\) and \(w_t\). In the scalar case,
\[
C(V) = \frac{1}{2}\log\!\left(1+\frac{V}{J^* + B^{-2}G\Psi_w}\right).
\]
The paper states that this capacity formula exactly matches that of a memoryless Gaussian MIMO channel with noise covariance \(\Psi_w\) and a weighted input power constraint determined by the allowable control loss \(V\) [2509.16146].

When the controller has noiseless observations but the receiver observes
\[
z_t = Dx_t + v_t, \qquad v_t \sim \mathcal{N}(0,\Psi_v),
\]
capacity is characterized by a convex optimization involving a DARE:
\[
C_{\mathrm{nor}}(V)
=
\max_{\Phi \succeq 0}
\frac{1}{2}
\log
\frac{\det(\Psi_v + D\Sigma D^\top)}
{\det(\Psi_v + D\Psi_\sigma D^\top)}
\]
subject to the Riccati constraint for \(\Sigma\) and
\[
\operatorname{Tr}((\Gamma+\hat G)\Phi)\le V.
\]
When both controller and receiver have noisy observations, the paper establishes a lower bound achieved by linear-Gaussian policies of the form
\[
u_t = -K \check{x}_{t|t} + B^\dagger s_t.
\]

A striking structural result is the separation principle: when the controller has noiseless access to the state, the capacity-achieving policy superimposes an independent Gaussian codeword on the optimal LQG control input without loss of optimality. Under this policy the implicit channel is equivalently translated into
\[
y_t = s_t + w_t,
\]
so existing Gaussian MIMO channel codes can be used directly [2509.16146].

## 6. Universality, operational benchmarks, and computability limits

When the channel law is completely unknown and may have memory, the classical Shannon capacity need not be operationally attainable. A distinct capacity notion is therefore introduced through the Iterative Finite Block (IFB) and Arbitrary Finite Block (AFB) capacities. IFB capacity is the supremum of rates achievable by any block coding system used over consecutive blocks, while AFB capacity modifies the benchmark so that the reference block code must work when inserted at arbitrary times and states of the channel [1202.0417].

For fading-memory channels, the main theorem gives adaptive-rate systems with feedback and common randomness whose rate satisfies
\[
R_{UNI}[N] \geq \max \Big[ IFB - \delta_N^{IFB},\; AFB - \delta_N^{AFB} \Big],
\]
with \(\delta_N^{IFB}\to 0\) for fading-memory channels and \(\delta_N^{AFB}\to 0\) for any causal channel. The fading-memory condition is formulated by
\[
\big\| P(Y_n^m \mid X_1^\infty, Y_1^{n-L-1})
-
P_n(Y_n^m \mid X_{n-L}^\infty) \big\|_1 \le h,
\]
which states that the effect of the distant past on future outputs becomes negligible [1202.0417]. In this sense, an operationally meaningful capacity can still be defined even without an explicit stationary channel model.

Against that positive result stands a sharp impossibility theorem for channels with memory. For information-stable finite state machine channels with rational product-form probabilities, known initial state, 10 input symbols, 62 states, and binary output, there is no algorithm that can always output a rational number \(c\) satisfying
\[
|C(\mathbf{N}) - c| \le \frac{1}{5}.
\]
More strongly, for a suitable subfamily \(\mathcal{S}_\lambda\), every channel has capacity either at least \(\lambda\) or at most \(\lambda/2\), and it is undecidable to determine which case holds. The proof reduces undecidable value problems for probabilistic finite automata to the computation of FSMC capacity, with the key equivalence
\[
C(\mathbf{V}_{\mathcal{A}}) = \mathrm{val}(\mathcal{A}) .
\]
Thus, memory can place channel capacity beyond algorithmic approximation, even for comparatively small state spaces [1601.06101].

A common misconception is that additional structure or feedback necessarily makes capacity both larger and easier to characterize. The cited literature shows a more differentiated picture. In the BIND channel, feedback does not increase capacity [1411.1650]. For fading-memory unknown channels, feedback and common randomness are crucial for universality [1202.0417]. For general FSMCs, however, memory can make the capacity uncomputable to fixed precision [1601.06101]. The resulting landscape is technically heterogeneous: implicit channel capacity may be learnable from samples, expressible through dynamic programming or convex optimization, reduced to classical Gaussian MIMO coding, or provably resistant to algorithmic computation, depending on how the channel is implicit and how memory, constraints, and auxiliary objectives enter the model.

Source: https://www.emergentmind.com/topics/implicit-channel-capacity