Variationally Inferred Parameterisation (VIP)

• Variationally Inferred Parameterisation (VIP) is a framework that generates smooth, adaptive transitions between centered and non-centered parameterizations in hierarchical Bayesian models.
• It embeds tunable reparameterizations via auxiliary variables and partial-centering weights, optimizing the inference geometry through joint variational parameter and lambda adjustment.
• By automating model-specific tuning, VIP improves the efficiency of both variational inference and MCMC, reducing manual intervention and addressing complex posterior geometries.

Variationally Inferred Parameterisation (VIP) is a mechanism for adaptively interpolating between centered and non-centered parameterizations in hierarchical Bayesian models, providing a systematic framework for mitigating the pathological posterior geometries—such as the "funnel"—that hinder efficient inference. By embedding per-latent-variable, continuously tunable reparameterizations into the generative model, VIP optimizes both the variational parameters and the parameterization itself to improve the geometry for variational inference and MCMC, all while requiring minimal intervention from the user and respecting the model/inference abstraction (Ko et al., 30 May 2025, Gorinova et al., 2019).

1. Definition and Rationale

VIP introduces, for each latent coordinate $i$ in hierarchical models of the form $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$, an auxiliary variable $\tilde{z}_i$ and a continuous partial-noncentering weight $\lambda_i \in [0,1]$. These are combined in a two-step generative reparameterization:

1. $$\tilde z_i\sim \mathcal{N}\bigl(\lambda_i f_i(\pi(z_i)),\; g_i(\pi(z_i))^{\lambda_i}\bigr)$$
2. $$z_i = f_i(\pi(z_i)) + g_i(\pi(z_i))^{1-\lambda_i}\bigl(\tilde z_i - \lambda_i f_i(\pi(z_i))\bigr)$$

When $\lambda_i=1$, this recovers the centered parameterization (CP); when $\lambda_i=0$, the non-centered parameterization (NCP) is recovered. VIP therefore defines a smooth path between CP and NCP for each latent. In practical hierarchical models, different latents can require different parameterization regimes for efficient inference, and manual tuning of these transformations is error-prone and model-specific. VIP automates this procedure by optimizing $\lambda$ jointly with variational parameters (Gorinova et al., 2019).

2. Mathematical Formulation

The family of partial-centering transformations is summarized by the bijective map $T_\lambda$:

$$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$0

Given a mean-field normal variational family $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$1, the evidence lower bound (ELBO) is maximized jointly in $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$2 and $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$3:

$$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$4

The pushforward of $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$5 through $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$6 induces a variational posterior $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$7 with density:

$$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$8

as the determinant of the Jacobian for each coordinate is $$z_i \sim \mathcal{N}\bigl(f_i(\pi(z_i)), g_i(\pi(z_i))\bigr)$$9 (Ko et al., 30 May 2025).

All gradient terms with respect to $\tilde{z}_i$0 can be computed pathwise; terms involving the variational density with respect to $\tilde{z}_i$1 vanish since $\tilde{z}_i$2 only depends on variational parameters, not on $\tilde{z}_i$3 (Gorinova et al., 2019).

3. Equivalence with Generalized Forward Autoregressive Flows

VIP in conjunction with a full-rank Gaussian