Nonlinear Capacity Method

• Nonlinear capacity method is a framework that defines rigorous upper and lower bounds on the information capacity of systems affected by nonlinearities and memory effects.
• It employs techniques such as sequence selection, non-Gaussian input shaping, and mismatched decoding metrics to optimize achievable rates and overcome traditional AWGN limits.
• Applications span optical fibers, silicon photonics, and neural networks, addressing challenges like phase-noise, energy conservation, and nonlinear distortions.

The nonlinear capacity method encompasses a class of mathematical and algorithmic frameworks designed to rigorously analyze, bound, or optimize the information-carrying capacity of communication systems, neural networks, and physical channels whose behavior is strongly affected by nonlinearity. These methods derive from and extend classical Shannon-theoretic tools but contend with nonlinear evolution equations, non-Gaussian statistics, memory effects, and optimized signaling/distribution strategies—often departing substantially from approaches valid for linear time-invariant or Gaussian-noise-limited systems.

1. Foundational Concepts and Nonlinear Channel Models

The nonlinear capacity method arises in settings where the channel (or system) cannot be accurately described by additive, linear, or memoryless models. In optical communications, the stochastic nonlinear Schrödinger equation (NLSE) governs the evolution of the complex field $A(z,t)$ in fiber, introducing phase and amplitude distortions via Kerr nonlinearity and amplification noise (Civelli et al., 2021). In silicon photonics, the presence of two-photon absorption and free-carrier effects introduces multiplicative and signal-dependent losses, requiring joint modeling of nonlinear dissipative mechanisms and noise (Dimitropoulos et al., 2014).

A key modeling step is the specification, often by first-principles physics, of a stochastic dynamical equation governing the system:

• Stochastic NLSE:

$$\frac{\partial A}{\partial z} + \frac{\alpha}{2} A + j \frac{\beta_2}{2} \frac{\partial^2 A}{\partial t^2} + j \gamma |A|^2 A = n(z,t)$$

• Discrete-time nonlinear models:

$$Y_k = x_k + Z_k,\quad Z_k = \tilde{Z}_k\,\sqrt{ASE + \eta \left(\overline{P}_k\right)^3}$$

with $\overline{P}_k$ the local averaged power over a finite memory (Agrell et al., 2014).

In neural and reservoir computing, the nonlinear system may be a recurrent state-space process:

$$x(t) = F(x(t-1), u(t)),\quad \hat{y}(t) = W^\top x(t)$$

with $F$ a potentially highly nonlinear map (Gonon et al., 2020, Castro et al., 3 Jun 2024).

2. Methods: Capacity Bounds, Input Optimization, and Decoding Metrics

The core of the nonlinear capacity method is constructing rigorous upper and lower bounds or achievable information rates (AIR) for the specified nonlinear channel.

Upper Bounds

Universal upper bounds leverage information-conserving properties of the physical system.

• Hamiltonian conservation: In the stochastic NLSE, the deterministic part is Hamiltonian (energy/entropy-preserving), so the capacity is maximally that of an AWGN channel with the same total SNR. For split-step discretizations:

$\mathrm{SE} \leq \log_2(1+\mathrm{SNR})$

where $\mathrm{SE}$ is the spectral efficiency in bits/s/Hz (Kramer et al., 2015, Yousefi et al., 2015).

Lower Bounds and Achievable Rates

• Sequence Selection Bound: Input sequences are selected based on their potential to induce less nonlinear interference, with a lower bound given by

$AIR_s \geq AIR_s^{(u)} + \frac{1}{2N} \log_2 \eta$

where $AIR_s^{(u)}$ is computed with the original (unbiased) distribution, and $\eta$ is the acceptance probability under the sequence selection metric (Civelli et al., 2021).

• Phase-noise separation and capacity: By explicitly modeling long-memory phase-noise due to XPM and exploiting its temporal correlations, one can almost separate and subtract this nonlinearity, leading to a significant increase in achievable capacity and effectively doubling the reach for fixed spectral efficiency (Dar et al., 2013).
• Finite-memory models: When channel memory is finite (e.g., nonlinear effects depend only on a window of neighboring symbols), the nontrivial dependence among input symbols can be exploited by blockwise shaping or heavy-tailed distributions, avoiding the “capacity collapse” seen in infinite-memory GN models (Agrell et al., 2014).
• Nonlinear analog preprocessing: In coarsely quantized SISO systems with receive-side analog nonlinearities, one constructs the effective channel by partitioning the analog input via nonlinear maps and quantization thresholds, bounding capacity by the cardinality of the induced partition (“associated code”), and maximizing mutual information over this reduced output space (Shirani et al., 2022).

Decoding Metrics

Due to the intractability of exact likelihood evaluation for most nonlinear channels, achievable rates are typically computed under mismatched decoding metrics, notably the AWGN metric, or using more elaborate nonlinear-aware decoders when tractable (Civelli et al., 2021).

3. Input Distribution and Sequence Design in Nonlinear Regimes

A key insight is that capacity in nonlinear channels can often be increased by non-standard, non-Gaussian input distribution design (“shaping”). Several methods emerge:

• Rejection sampling/sequence selection: Input sequences are drawn from an underlying i.i.d. source and only those with low nonlinear distortion (as measured by a surrogate distortion metric) are accepted, introducing a controlled bias in the input law and yielding higher achievable rates (Civelli et al., 2021).
• Blockwise and heavy-tailed shaping: For finite-memory nonlinearities, optimal input distributions may concentrate energy in a few positions per block, with the rest taking low power (satellite/sparse signaling), thus exploiting the local nature of the nonlinearity while maximizing entropy at the receiver (Agrell et al., 2014).
• Probabilistic constellation shaping and amplitude/phase separation: Numerical and analytical results suggest that in per-sample nonlinear channels, optimal input laws often become discrete in amplitude with uniform phase, closely mirroring Rician fading channel solutions (Yousefi et al., 2014).

4. Applications Beyond Optical Channels: Reservoir Computing and Neural Architectures

The nonlinear capacity method extends to quantification of information processing in nonlinear dynamical systems beyond communications.

• Memory and forecasting capacities in RNNs: One defines lag-specific memory and prediction capacities as the maximal squared correlation extractable via linear readouts from the network state, providing general upper bounds in terms of network dimension and input autocovariance structure (Gonon et al., 2020). For nonlinear reservoir and time-delay photonic systems, memory capacity is resolved into polynomial (linear, quadratic, cubic) components via readout training against orthogonal polynomial targets, guiding the design of systems balancing nonlinearity and long fading memory (Castro et al., 3 Jun 2024).
• Capacity allocation in deep networks: In neural architectures, capacity analysis tracks the allocation of model degrees of freedom (parameters) across input directions, propagating these allocations through nonlinear layers by linearization in an augmented space (including nonlinear activations), and establishing limits such as the Markovian or diffusive propagation of capacity in deep residual or pseudo-random networks (Donier, 2019).

5. Mathematical Structures: Variational, Path-Integral, and PDE Methods

The mathematical footing of nonlinear capacity analysis incorporates elements from stochastic processes, PDEs, and variational optimization:

• Path-integral and Fokker–Planck approaches: In per-sample zero-dispersion NLSE, both path-integral discretization and Fokker–Planck formalism deliver the conditional output density in closed or series form, which is then used to numerically or analytically bound the mutual information for specific amplitude and phase constraints (Yousefi et al., 2014, Panarin et al., 2016).
• Brunn-Minkowski-type inequalities for nonlinear capacity: In convex-geometry and potential theory, the nonlinear $p$-capacity associated with $p$-Laplace-type operators supports Brunn–Minkowski-type inequalities and associated Minkowski problems, with variational characterizations and precise regularity results (Akman et al., 2017).
• Simulator-based (data-driven) characterization: For flexible load ensembles or control systems with nonlinear device physics, the capacity region is defined as the set of admissible spectral densities under output QoS constraints, computed via simulation-driven testing and convex optimization over candidate spectra (Coffman et al., 2020).

6. Practical Implications, Limitations, and Extensions

Nonlinear capacity methods have demonstrated several key practical implications:

• Nonlinearity does not universally limit capacity: In many relevant regimes, exploiting structure (e.g., memory, input shaping, nonlinearity-induced phase-noise correlations) can restore or even exceed classical linear-AWGN capacity performance (Sorokina et al., 2013, Dar et al., 2013).
• Existence of ultimate bounds: In physically-constrained systems (Hamiltonian, energy-conserving), spectral efficiency cannot exceed that of the equivalent SNR-limited linear channel, and the proper accounting of bandwidth is essential (Kramer et al., 2015).
• Optimal signaling schemes are often highly non-Gaussian: For many nonlinear channels, capacity-achieving inputs may be discrete, sparse, or even block-structured, differing substantially from the classic Gaussian case (Agrell et al., 2014, Yousefi et al., 2014).
• Computation is typically nontrivial: Achievable rate computation often requires Monte Carlo, empirical periodogram, or path-integral evaluation, as closed-form expressions are rare except in certain asymptotics or approximations (Civelli et al., 2021, Coffman et al., 2020).

Limiting factors include:

• The computational cost of sequence selection or simulation-based region construction.
• The difficulty in constructing viable decoding metrics that balance tractability and tightness.
• Sensitivity to system model details, particularly in channels with unmodeled cross-interference, nonidealities, or strong memory.

Possible extensions include:

• Joint optimization of input distribution and decoding metrics for general nonlinear-metric channels (Civelli et al., 2021).
• Porting sequence selection and surrogate-metric approaches to other classes of nonlinear-memory channels (e.g., wireless, magnetic, memristive) (Civelli et al., 2021, Castro et al., 3 Jun 2024).
• Analytically characterizing finite-blocklength and finite-memory effects in coded systems (Agrell et al., 2014).
• Deepening the connection between high-dimensional convex geometry (Brunn–Minkowski) and nonlinear PDE-based capacity (Akman et al., 2017).

7. Summary Table: Key Features of Nonlinear Capacity Methods

Domain	Model Structure	Principal Capacity Methods
Optical fibers	NLSE, finite-memory	Sequence selection, path integral, AWGN metric bounds, phase-noise separation
Silicon photonics	Nonlinear loss/noise	Log-normal modeling, EPI and entropy bounds
Reservoir/NN	Nonlinear state space	Linearization in extended space, polynomial memory decomposition, network capacity propagation
Distributed PC	Algebraic/nonlinear	PIR-type coding with nonlinear message sets, entropy-based achievability/converse for private computation

Specific methodological contributions are detailed in (Civelli et al., 2021, Kramer et al., 2015, Dar et al., 2013, Dimitropoulos et al., 2014, Gonon et al., 2020, Coffman et al., 2020, Yousefi et al., 2014, Agrell et al., 2014, Sorokina et al., 2013, Castro et al., 3 Jun 2024).