--- title: Amortized Factor Inference Networks url: https://www.emergentmind.com/topics/amortized-factor-inference-networks-afins type: topic --- # Amortized Factor Inference Networks Amortized Factor Inference Networks (AFINs) denote a class of amortized neural inference constructions in which reusable inference objects are learned from a factorized probabilistic specification rather than derived anew for each dataset. The literature suggests that the term is not restricted to a single canonical architecture. In "Inference Networks for Sequential Monte Carlo in Graphical Models" [1602.06701], AFINs are structured neural recognition models that implement a stochastic inverse of a directed graphical model and serve as proposal distributions for Sequential Monte Carlo (SMC). In "Amortized Factor Inference Networks for Posterior Inference" [2605.26419], the name is used for encode–merge–decode networks that accept a typed-factor model specification and observations and return parameters of a variational posterior. Related work explicitly places Bayes-factor estimation with Evidence Networks and local blanket-level inference with $\Delta$-AI within an AFIN-style pattern of amortizing factor-level probabilistic computations [2305.11241; 2310.02423]. ## 1. Scope and terminology The principal usages of the term organize around what is being amortized and which probabilistic object is produced. In all cases, the amortized artifact is intended for reuse across many inference instances, but the target ranges from proposal distributions to posterior parameterizations to scalar evidence ratios. | Paper | Amortized object | Primary target | |---|---|---| | [1602.06701] | Structured inverse recognition network | $q(x \mid y)$ for SMC proposals | | [2605.26419] | Encode–merge–decode posterior network | $q_\phi(z)$ from typed factors | | [2305.11241] | Evidence Network as an AFIN instance | Bayes factor or log Bayes factor | | [2310.02423] | $\Delta$-AI as an AFIN-style method | Local conditional or blanket-consistent sampler | In the 2016 formulation, the defining feature is inverse factorization of a known directed graphical model. The forward model $$ p(x, y) = \prod_{i=1}^N f_i\big(x_i \mid pa(x_i)\big)\,\prod_{j=1}^M g_j\big(y_j \mid pa(y_j)\big) $$ is converted into an inverse model whose conditionals are oriented from observations toward latents, and a structured neural density estimator approximates the inverse conditional needed for posterior sampling [1602.06701]. In the 2026 formulation, the defining feature is instead a typed-factor interface. A model instance is written as $$ p(z, y_{1:N} \mid t_0,\theta_0,\{t_n,\theta_n\}_{n=1}^N) = p(z \mid t_0,\theta_0)\prod_{n=1}^N p(y_n \mid z, t_n,\theta_n), $$ and a single trained network maps the specification $(t,\theta)$ together with observations to the parameters of a variational posterior, while remaining invariant to varying numbers of observations and equivariant in latent coordinates [2605.26419]. This multiplicity of usage makes one misconception worth correcting. AFINs are not, in the literature considered here, a single standardized neural module. Rather, the term is applied to a broader design principle: amortize inference objects that respect factorized probabilistic structure, and train them so that test-time inference becomes a forward pass or a low-cost correction step. ## 2. Inverse-factor AFINs in directed graphical models The original AFIN construction is defined for directed graphical models with latent variables $x=(x_1,\dots,x_N)$ and observed variables $y=(y_1,\dots,y_M)$. The posterior target is $$ \pi(x) \equiv p(x \mid y) = \frac{p(x,y)}{p(y)}, \qquad \gamma(x) \equiv p(x,y), $$ and the goal is to construct a proposal $q(x \mid y)$ that closely approximates the intractable posterior while remaining tractable to sample and evaluate [1602.06701]. The key structural step is the construction of a stochastic inverse via inverse factorization. One first chooses a valid topological order of the original DAG and defines the Markov blanket $mb(v)$ of a node $v$ as the union of its parents, children, and the parents of its children. The order is then reversed, and inverse parents are defined by intersecting the Markov blanket with later variables in the reversed ordering: $$ (x_i) = mb(x_i)\cap\{x_{i+1},\dots,x_N,y_1,\dots,y_M\}, $$ with the observed-node analogue defined similarly. This yields an inverse conditional $$ \tilde{p}(x \mid y) = \prod_{i=1}^N \tilde{p}\big(x_i \mid (x_i)\big), $$ and the recognition model mirrors this structure as $$ q(x \mid y;\eta) = \prod_{i=1}^N q_i\big(x_i \mid \mathcal{S}_i(y,x_{i+1:N});\eta_i\big). $$ A central proposition in the construction is preservation of local conditional independence: if two inverse variables are conditionally independent given a third in the inverse graph, then the corresponding original variables were also conditionally independent in the original graph. This ensures that the inverse factorization does not introduce spurious independencies. The construction is explicitly designed to preserve dependencies created by explaining-away in head-to-head structures, and when several latents become jointly dependent after conditioning, they are grouped into a joint factor and modeled autoregressively: $$ q_J(x_J \mid \mathcal{S}_J) = \prod_{\ell=1}^{|J|} q_{J,\ell}\big(x_{J,\ell} \mid x_{J,1:\ell-1},\mathcal{S}_J\big). $$ Architecturally, the 2016 AFIN instantiates one conditional neural density estimator per inverse factor. The paper extends MADE into a conditional estimator by injecting inverse parents as conditioning inputs and constructing masks so that conditioning inputs propagate to all outputs, including the first autoregressive dimension. Continuous latents are modeled with mixture-of-Gaussians heads, $$ q_i(x_i \mid \cdot) = \sum_{d=1}^D \alpha_{i,d}\,\mathcal{N}\big(x_i \mid \mu_{i,d},\sigma_{i,d}^2\big), $$ using softplus for $\sigma$ and softmax for $\alpha$, while discrete latents use Bernoulli or categorical heads. Repeated structures, such as per-time-step factors in HMMs or per-group factors in hierarchical models, reuse the same sub-network parameters [1602.06701]. ## 3. Offline training and use as SMC proposals The original AFIN training objective is an inclusive-KL criterion averaged over datasets: $$ \mathcal{J}(\eta) = \mathbb{E}_{p(y)} \Big[D_{KL}\big(p(x \mid y)\,\|\,q(x \mid y;\eta)\big)\Big] = \mathbb{E}_{p(x,y)}\Big[-\log q(x \mid y;\eta)\Big] + \text{const}. $$ Because the expectation is taken under the tractable joint $p(x,y)$, stochastic gradients are available from ancestral samples of the forward model: $$ \nabla_\eta \mathcal{J}(\eta) = \mathbb{E}_{p(x,y)}\Big[-\nabla_\eta \log q(x \mid y;\eta)\Big]. $$ No gradients through the generative model are required; only the recognition network is differentiated. Training is performed offline with synthetic data, using mini-batch stochastic optimization with Adam and a held-out synthetic validation set; when validation worsens or after a fixed number of steps, synthetic training and validation data are regenerated to maintain diversity [1602.06701]. The inclusive-KL choice is not incidental. It encourages mass-covering proposals, reducing the risk that $q$ assigns negligible density where the true posterior has support. In the SMC setting this is important because support mismatch or overly narrow proposals induce weight degeneracy. The proposal learned by an AFIN is then inserted directly into an SMC scheme with sequential targets $\{\pi_n\}$ and unnormalized densities $\{\gamma_n\}$: $$ w_n^k = \frac{\gamma_n(x_{1:n}^k)} {\gamma_{n-1}(x_{1:n-1}^{a_{n-1}^k})\, q_n(x_n^k \mid y, x_{1:n-1}^{a_{n-1}^k})}, \qquad \hat{\pi}_N = \sum_k W_N^k\,\delta_{x_{1:N}^k}. $$ For temporal models, the incremental weight takes the familiar form $$ w_t = w_{t-1}\cdot \frac{p(x_t \mid x_{t-1})\,p(y_t \mid x_{1:t})} {q_t(x_t \mid y_{1:t},x_{1:t-1})}, $$ and resampling, for example when $\mathrm{ESS}