GVAMP: Generalized VAMP for GLMs

• GVAMP is an iterative inference algorithm for generalized linear models that combines nonlinear scalar denoisers with a robust LMMSE module.
• It efficiently handles arbitrary separable likelihoods and right-rotationally invariant matrices, offering improved convergence under ill-conditioned settings.
• GVAMP’s framework enables extensions for bilinear recovery, parameter learning, and coding applications, demonstrating practical impact in areas such as compressed sensing and optical communications.

Generalized Vector Approximate Message Passing (GVAMP), also presented as GLM-VAMP in early work, is an expectation-consistent / expectation-propagation-style iterative inference method for generalized linear models (GLMs) in which an unknown vector $x$ is observed through a separable output channel acting on $z=Ax$. It extends Vector Approximate Message Passing (VAMP) from additive white Gaussian noise linear models to arbitrary separable likelihoods $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$, while retaining the robustness of VAMP to right-rotationally or unitarily invariant sensing matrices rather than restricting $A$ to the IID Gaussian regime required by classical GAMP analyses (Rangan et al., 2016, Schniter et al., 2016). Later work embedded GVAMP inside broader families such as generalized memory AMP (GMAMP), established Bayes-optimal fixed-point results under unitarily invariant matrices, and developed low-complexity, adaptive, bilinear, and coding-aware extensions (Tian et al., 2021, Liu et al., 2023).

1. Model class and conceptual foundations

GVAMP is formulated for the generalized linear model

$$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$

where $A$ is $M\times N$, the prior is separable $p(x)=\prod_{i=1}^N P_X(x_i)$, and the observation channel is separable $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$ (Tian et al., 2021). Equivalent formulations write the posterior as

$p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$

which makes explicit the factorization into a prior factor on $z=Ax$0, a likelihood factor on $z=Ax$1, and a linear consistency constraint between them (Schniter et al., 2016).

The matrix class that underlies GVAMP’s robustness is broader than the IID sub-Gaussian setting of AMP. In the VAMP literature, $z=Ax$2 is right-orthogonally invariant, meaning that in the singular value decomposition $z=Ax$3, the right singular vectors $z=Ax$4 are Haar distributed and the distribution of $z=Ax$5 is invariant to right multiplication by any fixed orthogonal matrix (Rangan et al., 2016). In the generalized-memory formulation, the analogous complex-valued statement is that

$z=Ax$6

with $z=Ax$7 and $z=Ax$8 unitary, mutually independent with $z=Ax$9, and $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$0 Haar distributed; this is the unitarily invariant setting (Tian et al., 2021).

This placement of GVAMP between GAMP and VAMP is central. Standard GAMP is efficient for GLMs but is derived for IID sensing matrices; VAMP is robust for right-orthogonally invariant matrices but was introduced for linear AWGN models. GVAMP combines the generalized-output capability of GAMP with the rotational-invariance robustness of VAMP by introducing an auxiliary output variable $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$1 and passing Gaussian-parameterized extrinsic messages between nonlinear scalar modules and a linear LMMSE module (Schniter et al., 2016, Rangan et al., 2016).

2. Core algorithmic structure

The canonical GVAMP architecture alternates between nonlinear estimators and a linear estimator. In the GLM-VAMP presentation of Schniter, Rangan, and Fletcher, the algorithm maintains pseudo-observations $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$2 for $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$3 and $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$4 for $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$5, applies scalar denoisers, and forms extrinsic messages through Onsager-like orthogonalization (Schniter et al., 2016).

For the $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$6-stream,

$p(y|z)=\prod_m p_{y|z}(y_m|z_m)$7

followed by

$p(y|z)=\prod_m p_{y|z}(y_m|z_m)$8

For the $p(y|z)=\prod_m p_{y|z}(y_m|z_m)$9-stream,

$A$0

and

$A$1

Here $A$2 is the prior denoiser and $A$3 is the output-channel denoiser, each acting componentwise under Gaussian pseudo-priors (Schniter et al., 2016).

The linear module enforces $A$4 through an LMMSE calculation. With the SVD $A$5, diagonal matrix $A$6 defined by

$A$7

the linear update is

$A$8

with average Jacobians

$A$9

Extrinsic feedback to the nonlinear modules is then formed by the same subtraction-and-rescaling pattern (Schniter et al., 2016).

A later unitarily invariant formulation describes GVAMP as a two-module non-memory iterative process with symbol-wise MMSE denoisers

$$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$0

and an LMMSE linear step

$$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$1

This form makes explicit the matrix inverse $$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$2, which is the chief source of GVAMP’s computational cost (Tian et al., 2021).

That cost is substantial. For the linear estimator, direct inversion requires $$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$3 per iteration, plus $$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$4 multiplications; an SVD-based variant has similar asymptotic burden in memory and still high runtime for large-scale systems (Tian et al., 2021). This computational profile is one of the main reasons later work sought memory-based approximations and low-complexity replacements.

3. State evolution, fixed points, and optimality claims

The original theoretical foundation comes from VAMP. For standard linear regression with right-orthogonally invariant $$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$5 and separable Lipschitz denoisers, VAMP admits a rigorous scalar state evolution, and its fixed points are consistent with replica predictions of the minimum mean-squared error (Rangan et al., 2016). GVAMP inherits this program in the GLM setting by replacing the Gaussian output model with a nonlinear output-channel module.

In the generalized-memory treatment, GVAMP’s state evolution under unitarily invariant $$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$6 tracks the pseudo-observation variances through scalar recursions

$$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$7

with

$$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$8

These recursions are rigorous under the rotational-invariance assumptions and the associated orthogonality conditions (Tian et al., 2021).

The strongest Bayes-optimality statement in the supplied literature is attached to the memory generalization rather than to the original 2016 GLM-VAMP paper. For unitarily invariant transformation matrices, Bayes-optimal GMAMP is proved to attain the replica minimum, i.e. Bayes-optimal, MSE if it has a unique fixed point, and its fixed points coincide with those of GVAMP’s state evolution (Tian et al., 2021). This places GVAMP’s fixed points inside a later, more explicit replica-optimal framework.

Model mismatch complicates this picture. In the mismatched VAMP/GVAMP analysis, state evolution must track the order parameters $$\Psi:\quad y = Q(z),\qquad \Gamma:\quad z = A x,\qquad \Phi:\quad x_i \sim P_X(x),$$9, $A$0, and $A$1, rather than a single MSE-like scalar. The fixed-point equations coincide with the replica-symmetric saddle-point equations, while the microscopic instability threshold of GVAMP matches the de Almeida–Thouless instability line (Takahashi et al., 2020). A common implication is that convergence failures are not purely numerical artifacts: in some regimes they correspond to replica-symmetry breaking and genuine algorithmic instability.

Theoretical coverage is therefore stratified. The original GLM-VAMP paper emphasizes the algorithmic construction and empirical robustness, while explicitly stating that “a rigorous justification of these models is postponed for future work” (Schniter et al., 2016). Subsequent work supplied more detailed asymptotic and fixed-point analyses, especially for memory extensions and mismatched inference (Tian et al., 2021, Takahashi et al., 2020).

4. Generalizations and algorithmic descendants

Several later algorithms should be understood as GVAMP extensions rather than unrelated methods. One line concerns parameter learning. EM-GVAMP combines expectation maximization with GVAMP when the prior $A$2 and/or channel $A$3 contain unknown deterministic parameters. After GVAMP supplies Gaussian approximate posterior moments, the M-step updates

$A$4

thereby yielding an empirical-Bayes variant of GVAMP (Metzler et al., 2018).

A second line concerns bilinear models. Bilinear Adaptive GVAMP (BAd-GVAMP) addresses measurements

$A$5

and decomposes inference into a componentwise MMSE output module on $A$6 plus a BAd-VAMP module on a pseudo-linear observation, with EM updates for $A$7, $A$8, and $A$9 (Meng et al., 2018). When $M\times N$0 is fixed, the model collapses to GVAMP on the GLM; when the output channel is linear Gaussian, BAd-GVAMP reduces to BAd-VAMP (Meng et al., 2018).

A third line concerns long-memory processing. GMAMP generalizes AMP/VAMP/GVAMP by allowing arbitrary-length memory in all local processors and imposing stricter orthogonality: the current output error must be orthogonal to the entire memory of input errors. In this framework, choosing memory lengths equal to $M\times N$1 and selecting LMMSE polynomial coefficients recovers GVAMP exactly (Tian et al., 2021). The same paper constructs a low-complexity Bayes-optimal GMAMP with memory linear recursions

$M\times N$2

thereby replacing the inverse-based GVAMP linear step with matrix-vector recursions of $M\times N$3 complexity per iteration (Tian et al., 2021).

A fourth line concerns coding and achievable rates. For generalized linear systems with unitarily invariant matrices, GOAMP/GVAMP has been analyzed through a variational state evolution obtained by transforming the fully unfolded state evolution into an equivalent single-input single-output recursion. Using that reduction and the I-MMSE lemma, the literature derives a maximum achievable rate and an optimal coding principle, then designs LDPC codes with reported gains of $M\times N$4 dB over existing methods (Liu et al., 2023). This suggests that GVAMP is not only a detector but also a basis for end-to-end coded-system design.

5. Applications and empirical behavior

GVAMP has been used across a broad range of GLM tasks. The original GLM-VAMP paper emphasizes robust regression, binary classification, quantized compressed sensing, phase retrieval, photon-limited imaging, and inference from neural spike trains as motivating examples (Schniter et al., 2016). In one-bit compressed sensing experiments with $M\times N$5, $M\times N$6, 16 nonzeros, and SNR $M\times N$7 dB, GLM-VAMP maintained accurate recovery up to $M\times N$8, whereas adaptively damped GAMP was accurate for $M\times N$9 and degraded for larger condition numbers; GLM-VAMP converged in approximately $p(x)=\prod_{i=1}^N P_X(x_i)$0–$p(x)=\prod_{i=1}^N P_X(x_i)$1 iterations with weak dependence on $p(x)=\prod_{i=1}^N P_X(x_i)$2, while AD-GAMP required approximately $p(x)=\prod_{i=1}^N P_X(x_i)$3–$p(x)=\prod_{i=1}^N P_X(x_i)$4 iterations when it converged (Schniter et al., 2016).

EM-GVAMP was demonstrated on phase retrieval with unknown measurement-noise variance. For the complex model $p(x)=\prod_{i=1}^N P_X(x_i)$5, $p(x)=\prod_{i=1}^N P_X(x_i)$6, the paper derives an EM update for $p(x)=\prod_{i=1}^N P_X(x_i)$7 using the GVAMP Gaussian posterior approximation and reports accurate estimation of unknown $p(x)=\prod_{i=1}^N P_X(x_i)$8 on large problems with $p(x)=\prod_{i=1}^N P_X(x_i)$9 having IID circular Gaussian entries (Metzler et al., 2018).

BAd-GVAMP extends these ideas to generalized bilinear recovery. The reported applications include quantized compressed sensing with matrix uncertainty, blind self-calibration from quantized measurements, and structured dictionary learning from quantized measurements. In the experiments summarized in the supplied material, BAd-GVAMP converged in approximately $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$0–$p(y|z)=\prod_{i=1}^M p(y_i|z_i)$1 outer iterations across tested scenarios, achieved near-oracle behavior in quantized compressed sensing, and remained effective under both 1-bit and multi-bit quantization (Meng et al., 2018).

More recent work places GVAMP in high-rate communication systems. For bandlimited direct-detection optical fiber channels, a GVAMP detector combined with multi-level coding and successive interference cancellation was reported to operate within about $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$2 bits per channel use of the real-alphabet coherent capacity for optically amplified links, to improve the best existing theory-based gap of $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$3 bpcu, and to deliver approximately $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$4 dB power-efficiency gain for bipolar over unipolar modulation in amplified links and $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$5 dB in unamplified links. In that setting, the receiver required approximately $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$6 iterations to achieve $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$7 bpcu with $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$8 multiplications per information bit (Plabst et al., 2 Aug 2025).

These results also clarify the empirical role of GVAMP’s assumptions. When the matrix model matches the right-rotational or unitary-invariance setting, GVAMP combines competitive accuracy with unusually strong robustness to ill-conditioning; when the model moves into bilinear, adaptive, or coded regimes, the literature tends to preserve GVAMP’s extrinsic-message structure while modifying the modules around it (Schniter et al., 2016, Meng et al., 2018).

6. Limitations, misconceptions, and related methods

GVAMP is often described informally as a “general-matrix GAMP,” but that formulation is too broad. Its strongest guarantees rely on right-rotationally invariant or unitarily invariant $p(y|z)=\prod_{i=1}^M p(y_i|z_i)$9, separable priors and output channels, and module regularity such as Lipschitz continuity and well-behaved derivatives (Rangan et al., 2016, Tian et al., 2021). It is therefore more accurate to view GVAMP as a rotational-invariance counterpart to GAMP than as a universally robust GLM solver.

A related misconception is that the original GLM-VAMP paper already supplied a full rigorous state-evolution theory for the generalized model. It did not: that paper explicitly deferred rigorous justification of the GLM extension, even while showing strong numerical robustness (Schniter et al., 2016). Rigorous scalar state evolution is classical for VAMP in the AWGN linear model, and later papers extend asymptotic analysis in several directions, but the theoretical development proceeded incrementally rather than all at once (Rangan et al., 2016, Takahashi et al., 2020).

Matrix structure remains a practical fault line. A unitary-transform alternative, GUAMP, was proposed for general measurement matrices, in particular highly correlated matrices. In quantized compressed sensing with doubly correlated $p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$0, the reported numerical evidence shows that GAMP diverges for moderate correlation $p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$1, GVAMP begins to diverge in the same regime, and both GAMP and GVAMP fail completely at larger $p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$2, such as $p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$3, while GUAMP remains stable and accurate (Zhu et al., 2022). This does not negate GVAMP’s value; it delineates the boundary of the variance-tracking mechanism on which GVAMP depends.

Complexity is the second major limitation. The inverse-based LMMSE step that gives GVAMP its robustness is also its bottleneck, with $p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$4 direct-inversion cost per iteration in the unitarily invariant formulation (Tian et al., 2021). Low-complexity descendants such as BO-GMAMP aim to preserve GVAMP’s fixed points while replacing inversions by memory recursions and matrix-vector products. A plausible implication is that much of the later AMP literature is best read as an effort to retain GVAMP’s extrinsic-information geometry while weakening its matrix assumptions or reducing its linear-module cost.

In the broader AMP taxonomy, AMP and GAMP remain attractive for IID Gaussian $p(x,z|y)\propto p(x)\,p(y|z)\,\delta(z-Ax),$5 because of simplicity and low cost; VAMP and GVAMP trade additional linear-algebra structure for greater robustness under rotational invariance; OAMP is closely related through orthogonality-based design; GMAMP adds explicit memory; EM-GVAMP addresses unknown parameters; and bilinear extensions such as BAd-GVAMP and BiG-VAMP transport the same message-passing logic to structured matrix-factorization problems (Schniter et al., 2016, Tian et al., 2021). Within that lineage, GVAMP occupies the pivotal position at which VAMP’s rotational-invariance machinery first meets the full generalized-linear observation model.