Inference Inversion: Reverse Mapping Concepts

• Inference inversion is a reverse mapping process that reconstructs latent parameters or inputs from observed outputs under structural, probabilistic, or information-theoretic constraints.
• It applies to Bayesian inversion via posterior construction and to graphical models by designing faithful inverse DAGs, ensuring minimal yet accurate dependency representation.
• It also supports neural network recovery, privacy defenses, and scientific inverse problems such as seismic tomography and ultrasonic imaging through both exact and approximate methods.

Inference inversion denotes the construction of a reverse map from observations, outputs, or intermediate representations back to latent causes, parameters, or admissible inputs. In the literature considered here, the term is not used uniformly. In Bayesian settings, it refers to posterior construction by reversing a forward data-generating process, often via Bayes’ rule and its compositional structure (Braithwaite et al., 2023). In amortized inference for graphical models, it denotes the construction of an inverse DAG that specifies the dependency structure of an inference network $q(z\mid x)$ (Webb et al., 2017). In neural-network security and interpretability, it refers to reconstructing inputs from outputs or intermediate features, including exact SAT-based inversion for binarised neural networks, latent-code recovery for GANs, and model inversion attacks in collaborative inference and black-box classification settings (Suhail et al., 2024). In scientific computing, it names probabilistic parameter estimation and uncertainty quantification for inverse problems such as seismic tomography, full waveform inversion, and ultrasonic imaging (Zhao et al., 2023). Despite these differences, the common theme is the reversal of a forward map under structural, probabilistic, or information-theoretic constraints.

1. Terminological scope and formal problem classes

The broadest probabilistic formulation appears in Bayesian inference. Given parameters or hidden states $x$ and observations $y$, Bayes’ rule defines a posterior kernel $K_p^\dagger(y\to x)=P(x\mid y)$ from a prior $p(x)$ and a forward kernel $K(x\to y)=P(y\mid x)$ (Braithwaite et al., 2023). In inverse-problem notation, this is written as

$$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$

with likelihood $p(d\mid\theta)$, prior $p(\theta)$, and evidence $p(d)$ (Zhao et al., 2023).

In probabilistic graphical models, inference inversion is the task of constructing an inverse DAG $x$0 on the same variables $x$1 so that the factorization

$x$2

captures the coarse-grain structure of the amortized inference network $x$3 (Webb et al., 2017). The objective is not exact symbolic inversion of a function but preservation of posterior dependencies needed for accurate amortized inference.

In neural networks, the problem is stated operationally: given a desired output $x$4 or hidden pattern $x$5, determine which inputs $x$6 map to it. For binarised neural networks, exact inversion is performed by encoding the trained network into a Boolean CNF formula and solving that formula under output constraints (Suhail et al., 2024). For pretrained GANs, the inversion problem is to find a latent code $x$7 such that $x$8, equivalently

$x$9

(Lin et al., 2019).

In privacy research, model inversion attacks target outputs or intermediate features. In collaborative inference, a split model computes $y$0, transmits $y$1 to the cloud, and the attacker attempts to reconstruct $y$2 from $y$3 (Liu et al., 1 Jan 2025). In black-box classification, attribute-inference attacks assume the adversary knows non-sensitive attributes and query access to a classifier, and seeks the value of a sensitive attribute (Mehnaz et al., 2020, Mehnaz et al., 2022).

A concise taxonomy from the cited literature is given below.

Setting	Forward object	Inversion target
Bayesian inference	$y$4, $y$5	posterior $y$6, Bayes inverse $y$7
Amortized inference	BN or DAG $y$8	inverse DAG $y$9, structure of $K_p^\dagger(y\to x)=P(x\mid y)$0
Neural-network inversion	trained network $K_p^\dagger(y\to x)=P(x\mid y)$1 or generator $K_p^\dagger(y\to x)=P(x\mid y)$2	inputs $K_p^\dagger(y\to x)=P(x\mid y)$3, hidden-consistent assignments, or latent code $K_p^\dagger(y\to x)=P(x\mid y)$4
Collaborative inference	intermediate feature $K_p^\dagger(y\to x)=P(x\mid y)$5	reconstruction of $K_p^\dagger(y\to x)=P(x\mid y)$6 from $K_p^\dagger(y\to x)=P(x\mid y)$7
Scientific inverse problems	forward simulator $K_p^\dagger(y\to x)=P(x\mid y)$8, $K_p^\dagger(y\to x)=P(x\mid y)$9	posterior over parameters or fields

This suggests that “inference inversion” is best understood as a family of reverse-construction problems rather than a single algorithmic primitive.

2. Bayesian inversion and compositional structure

A categorical treatment makes the reverse map explicit. In a Markov category $p(x)$0, objects $p(x)$1 are spaces of states and morphisms $p(x)$2 are Markov kernels $p(x)$3. Given an input distribution $p(x)$4, the kernel $p(x)$5 yields the forward joint $p(x)$6 and marginal $p(x)$7 (Braithwaite et al., 2023). The associated Bayes inverse is

$p(x)$8

The same work formulates Bayesian inversion as a state-dependent morphism in a fibration. For each $p(x)$9, $K(x\to y)=P(y\mid x)$0 has objects $K(x\to y)=P(y\mid x)$1 and morphisms $K(x\to y)=P(y\mid x)$2 given by state-indexed families of kernels $K(x\to y)=P(y\mid x)$3. A map $K(x\to y)=P(y\mid x)$4 induces a reindexing functor $K(x\to y)=P(y\mid x)$5, and the Grothendieck construction of the opposite indexed category yields the category of Bayesian lenses

$K(x\to y)=P(y\mid x)$6

A morphism $K(x\to y)=P(y\mid x)$7 is a pair $K(x\to y)=P(y\mid x)$8 with forward kernel $K(x\to y)=P(y\mid x)$9 and backward assignment $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$0 (Braithwaite et al., 2023).

The central compositional statement is the chain rule for Bayesian inversion. If $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$1, then for any prior $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$2,

$$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$3

or in kernel notation,

$$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$4

For the concrete two-stage model $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$5 with kernels $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$6, $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$7, and prior $$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$8, the full posterior satisfies

$$p(\theta\mid d)=\frac{p(d\mid\theta)\,p(\theta)}{p(d)},$$9

(Braithwaite et al., 2023).

This compositional view is used to motivate modular inference algorithms. Each component kernel $p(d\mid\theta)$0 carries its own backward arrow $p(d\mid\theta)$1; sequential or parallel compositions inherit inverses by the lens-style chain rule; and the fibration $p(d\mid\theta)$2 makes explicit the prior dependence of inversion (Braithwaite et al., 2023). The stated practical payoff is that, in probabilistic programming or variational inference, one can factor inference into per-component update routines, propagate beliefs backwards along the program’s causal graph, and automatically re-use optimised BayesInversion code for each building-block kernel.

3. Structural inversion in graphical models and probabilistic programs

In amortized inference for Bayesian networks, the inversion problem is structural rather than merely numerical. The inverse DAG should be faithful, meaning it must not introduce conditional independencies that do not hold in the original model: $p(d\mid\theta)$3 It should also be minimal, so that removing any edge violates faithfulness; equivalently, $p(d\mid\theta)$4 is a minimal I-map of $p(d\mid\theta)$5 (Webb et al., 2017).

The NaMI algorithm constructs such a minimally faithful inverse by simulating variable elimination on the moralized undirected graph $p(d\mid\theta)$6. When elimination of a variable $p(d\mid\theta)$7 creates a clique among its neighbors, those neighbors become parents of $p(d\mid\theta)$8 in the inverse DAG. The resulting theorem states that NaMI produces a DAG $p(d\mid\theta)$9 that is natural and a minimal I-map of $p(\theta)$0 (Webb et al., 2017). The paper contrasts this with heuristic inversions. Simply reversing every edge and dropping edges among observed nodes misses explaining-away dependencies; the Stuhlmüller–Paige–Wood heuristic can miss longer-range dependencies; and a fully connected inverse is faithful but non-minimal (Webb et al., 2017).

The empirical results support the structural criterion. In a relaxed Bernoulli VAE on MNIST with 30 latent relaxed Bernoulli units, both mean-field and faithful NaMI inverses used $p(\theta)$1M parameters, but the faithful inverse achieved $p(\theta)$2 versus $p(\theta)$3, $p(\theta)$4 versus $p(\theta)$5, and a variational gap of $p(\theta)$6 versus $p(\theta)$7 after $p(\theta)$8 epochs (Webb et al., 2017). On a binary-tree Gaussian BN of depth $p(\theta)$9, reverse-NaMI obtained $p(d)$0, forward-NaMI $p(d)$1, fully connected $p(d)$2, and the Stuhlmüller heuristic $p(d)$3 (Webb et al., 2017). On a Gaussian mixture model with $p(d)$4, reverse-NaMI achieved average negative log-likelihood under the true posterior $p(d)$5 after $p(d)$6 epochs, compared with $p(d)$7 for the fully connected inverse (Webb et al., 2017).

A related but distinct programmatic formulation appears in sequential software simulators. A simulator with stepwise latent parameters $p(d)$8, internal states $p(d)$9, emissions $x$00, and observed real data $x$01 is cast as a simple sequential probabilistic model with observation likelihood

$x$02

The full joint distribution is

$x$03

(Saeedi et al., 2015). The paper implements four inference strategies—Metropolis-Hastings, sequentialized Metropolis-Hastings, particle Gibbs, and hybrid PGibbs–MH—using under 20 lines of probabilistic code for the model and 4 or fewer lines for each strategy (Saeedi et al., 2015). In the reported geological simulator case study with $x$04 lobes and $x$05, sequential MH achieved the highest median log-score and lowest variance, while PGibbs provided reasonable performance with moderate variance (Saeedi et al., 2015).

Together, these works define a recurring principle: inversion quality depends on preserving the dependency structure of the posterior and on choosing reverse parameterizations that are natural for the original forward process.

4. Exact and approximate inversion of trained neural models

For binarised neural networks, inversion can be exact. A BNN has binary weights and activations, with $x$06, $x$07, and sign activations after affine and batch-normalization layers (Suhail et al., 2024). Each threshold test is encoded as a CNF constraint; output argmax is represented using auxiliary Boolean variables; and the full network becomes

$x$08

Inversion for target label $x$09 then constrains the output variables and solves or samples the resulting SAT instance (Suhail et al., 2024). The paper states that the CNF size is polynomial in the number of neurons $x$10 and maximum fan-in $x$11, with $x$12 and $x$13 using sequential counters. On a $x$14–$x$15–$x$16 BNN trained on $x$17-MNIST for $x$18 epochs, the model achieved $x$19 classification accuracy, the encoding had $x$20 Boolean variables and $x$21 clauses, and all sampled inputs for label “2” were classified back as $x$22 with $x$23 consistency (Suhail et al., 2024). In a smaller $x$24–$x$25–$x$26 BNN on $x$27-MNIST, the CNF was unsatisfiable for label “8,” proving that the network never classifies any input as “8” (Suhail et al., 2024).

GAN inversion is treated differently. InvGAN trains an encoder $x$28 without real data by sampling $x$29 and optimizing semantic-consistency, latent-recovery, and adversarial distribution-matching losses (Lin et al., 2019). The overall objective is

$x$30

with $x$31 and $x$32 in practice (Lin et al., 2019). The paper also states an approximate invertibility theorem: if $x$33 is $x$34-Lipschitz and inversion error is bounded by $x$35 on sampled training latents, then for a fresh $x$36, $x$37 with high probability (Lin et al., 2019). On CIFAR-10, InvGAN with $x$38 gradient-descent steps achieved $x$39, $x$40, $x$41, and classifier accuracy $x$42, compared with direct optimization at $x$43, $x$44, $x$45, and accuracy $x$46 (Lin et al., 2019). The same encoder is then used as a projection-based defense mechanism against adversarial examples, although the paper also emphasizes that it enables a reparameterization white-box attack for evaluation (Lin et al., 2019).

A common misconception is that inversion is necessarily approximate or heuristic. The BNN work shows exact inversion by satisfiability solving (Suhail et al., 2024), whereas the GAN work studies approximate inversion under smoothness and finite-sample assumptions (Lin et al., 2019). The contrast is not contradictory; it reflects different forward-model classes.

5. Privacy-oriented model inversion attacks and defenses

In collaborative inference, intermediate features are a direct inversion surface. The setting is $x$47, with the attacker observing $x$48 and attempting reconstruction (Liu et al., 1 Jan 2025, Liu et al., 18 Jun 2025). A central theoretical result states that the conditional entropy of inputs given intermediate features lower-bounds the minimal achievable reconstruction MSE under any inversion attack. If

$x$49

then

$x$50

(Xia et al., 1 Mar 2025). This establishes $x$51 as a privacy-relevant quantity. A related criterion for collaborative inference identifies mutual information, entropy, and effective information volume as key factors governing model inversion difficulty (Liu et al., 1 Jan 2025).

These theoretical claims are operationalized in two defense lines. Conditional Entropy Maximization introduces stochastic encoding $x$52, models deterministic features by a Gaussian mixture, and defines the surrogate

$x$53

Training minimizes

$x$54

(Xia et al., 1 Mar 2025). Across CIFAR-10, CIFAR-100, TinyImageNet, and FaceScrub, the reported average MSE gains from plugging CEM into obfuscation-based defenses were $x$55, $x$56, $x$57, and $x$58 on training/inference features, with accuracy drops typically $x$59 or negligible and no extra inference-time cost (Xia et al., 1 Mar 2025).

SiftFunnel combines nonlinear and linear correlation constraints, label smoothing, sparsity, and a funnel-shaped edge model with attention. Its loss is

$x$60

(Liu et al., 1 Jan 2025). On CIFAR-10 with a CNN split, the unprotected model had test accuracy $x$61, MLE-MIA MSE $x$62, Gen-MIA MSE $x$63, mutual information $x$64, $x$65, and $x$66; SiftFunnel reported $x$67, $x$68, $x$69, $x$70, $x$71, and $x$72, respectively (Liu et al., 1 Jan 2025). The same work reports that the SiftFunnel edge model has $x$73 versus $x$74 for the baseline, with effective information dropping from $x$75 to $x$76 and single-sample CPU latency increasing from $x$77 ms to $x$78 ms (Liu et al., 1 Jan 2025).

The partition point itself can dominate inversion resistance. The “Golden Partition Zone” work argues against the common belief that increasing model depth can resist MIA, and instead identifies representational transition or decision-level layers as the robust split region (Liu et al., 18 Jun 2025). For IR-152, MSE is reported as $x$79–$x$80 up to Block $x$81, jumps to $x$82–$x$83 at Block $x$84, and grows above $x$85 beyond that; for VGG19, the transition zone around Blocks $x$86–$x$87 yields MSE $x$88–$x$89, while final decision outputs yield MSE $x$90 (Liu et al., 18 Jun 2025). The paper states that partitioning at or just after the representational transition yields on average $x$91–$x$92 higher MSE than shallow splits and a $x$93 stronger resistance margin even with enhanced inversion models (Liu et al., 18 Jun 2025).

Black-box attribute-inference attacks show that inversion risk also arises from confidence vectors and labels. Confidence modeling-based and confidence score-based attacks query a classifier under multiple candidate sensitive-attribute values and infer the sensitive value from correctness patterns and scores (Mehnaz et al., 2020). On the GSS dataset with a decision-tree target, CMMIA achieved G-mean $x$94 and MCC $x$95, CSMIA G-mean $x$96 and MCC $x$97, while the Fredrikson et al. baseline achieved G-mean $x$98 and MCC $x$99 (Mehnaz et al., 2020). A later work adds a label-only attack, LOMIA, and reports that on Adult with a decision-tree target, CSMIA achieved $y$00, G-mean $y$01, and MCC $y$02, while LOMIA achieved $y$03, G-mean $y$04, and MCC $y$05 (Mehnaz et al., 2022). Both works report disparate vulnerability across demographic subgroups (Mehnaz et al., 2020, Mehnaz et al., 2022).

Prediction Purification targets the confidence vector itself by inserting a purifier $y$06 after a fixed classifier $y$07, so that the API exposes $y$08 rather than $y$09 (Yang et al., 2020). Its base objective combines $y$10 reconstruction and label-consistency losses, and can be augmented adversarially against inversion and membership inference (Yang et al., 2020). The reported effects include reducing membership inference accuracy by up to $y$11, increasing model inversion error by a factor of up to $y$12, less than $y$13 test-accuracy drop, and less than $y$14 distortion to confidence scores (Yang et al., 2020).

A plausible implication is that privacy-oriented inference inversion research has shifted from purely attack-driven reconstruction to quantitative control of information in transmitted features, outputs, and partition choices.

6. Inverse problems, variational inference, and posterior reuse

In scientific inverse problems, inference inversion is often synonymous with Bayesian parameter estimation and uncertainty quantification. Boosting Variational Inference approximates the posterior by a finite Gaussian mixture

$y$15

grown one component at a time by maximizing a residual ELBO and updating the new component weight (Zhao et al., 2023). In travel-time tomography, BVI with $y$16 components required $y$17k forward solves, while MH-MCMC required $y$18M; in Love-wave tomography with approximately $y$19 parameters, BVI again used $y$20k evaluations versus $y$21M forward solves for MH-MCMC, with mean and standard-deviation maps agreeing with MCMC and other variational methods at approximately $y$22 of the cost (Zhao et al., 2023). In Marmousi2 full-waveform inversion with $y$23 parameters, BVI used $y$24k gradient solves and produced mean, standard deviation, skewness, and kurtosis maps similar to SVGD and stochastic SVGD, while improving over ADVI’s under-dispersion (Zhao et al., 2023).

Variational Prior Replacement addresses a different reverse operation: changing prior information after an expensive inference has already been solved (Zhao et al., 2024). Starting from

$y$25

the new posterior under prior $y$26 is

$y$27

with $y$28 a $y$29-independent normalization constant (Zhao et al., 2024). The method first approximates the old posterior by $y$30, defines the re-weighted target $y$31, and then solves a second variational projection without further likelihood calls (Zhao et al., 2024). In the reported 2D full-waveform inversion example, the initial PSVI inversion under the uniform prior required $y$32 iterations, $y$33 FWI solves on $y$34 cores in approximately $y$35 days; subsequent VPR updates to smoothed and geological priors each required $y$36 iterations, $y$37 samples per iteration, one core, and approximately $y$38 minutes, with zero new FWI solves (Zhao et al., 2024).

Ultrasonic imaging via SVGD-based full waveform inversion uses a particle approximation $y$39 and Stein updates driven by the posterior gradient and an RBF kernel (Li et al., 13 Jan 2025). In a $y$40 grid linear-array experiment, SVGD achieved the same misfit in approximately $y$41 iterations versus approximately $y$42 for deterministic FWI, with central standard deviation $y$43 km/s (Li et al., 13 Jan 2025). In the ring-array case, the reported relative error inside the region was mean $y$44, max $y$45, and the standard-deviation map highlighted transition zones (Li et al., 13 Jan 2025). In realistic breast tissue, SVGD yielded mean relative error $y$46, max $y$47, and standard deviation $y$48 km/s, compared with mean-field stochastic VI at mean $y$49, max $y$50, and standard deviation up to $y$51 km/s (Li et al., 13 Jan 2025).

A broader optimization-and-inversion framework couples Bayesian optimization with Gaussian-process surrogates and Bayesian inversion (Chiappetta et al., 4 Feb 2026). The GP surrogate uses kernels such as RBF and Matérn-5/2; BO refines the surrogate with acquisition functions such as EI and UCB; and the inversion stage performs MAP or posterior approximation on the surrogate (Chiappetta et al., 4 Feb 2026). The paper reports $y$52 and $y$53–$y$54 until MSE $y$55 in one-dimensional benchmarks, about $y$56 samples for a mixed Gaussian-periodic surface in two dimensions, and about $y$57 for Rosenbrock (Chiappetta et al., 4 Feb 2026). The integrated BO(UCB)+BI(MAP–LS) workflow is reported to provide global surrogate plus MAP plus posterior at negligible cost relative to high-fidelity evaluation (Chiappetta et al., 4 Feb 2026).

These works treat inversion not as single-point reconstruction but as posterior construction, posterior transport under changing priors, or uncertainty-aware field recovery. This suggests a substantive divide between privacy-oriented inversion, which usually aims to suppress or bound recoverability, and scientific inversion, which seeks stable and analytically useful posterior structure.

7. Conceptual tensions, misconceptions, and unresolved directions

Several tensions recur across the literature. One concerns fidelity versus minimality. Inverse graphical structures should preserve true posterior dependencies but avoid superfluous edges; the NaMI results explicitly frame this as faithfulness plus minimality (Webb et al., 2017). Fully connected inverses can be faithful yet non-minimal, with slower learning and larger inference networks (Webb et al., 2017). In privacy settings, the analogous tension is utility versus leakage: defenses such as CEM, SiftFunnel, and Prediction Purification are designed to increase inversion error while maintaining classification accuracy or computing efficiency (Xia et al., 1 Mar 2025, Liu et al., 1 Jan 2025, Yang et al., 2020).

A second tension concerns whether deeper features are necessarily safer. The Golden Partition Zone work explicitly states that it overturns the common belief that increasing model depth can resist MIA, arguing instead for partitioning at representational transition or decision-level layers (Liu et al., 18 Jun 2025). The theoretical language used there centers on changes in $y$58, intra-class mean squared radius $y$59, and feature dimensionality (Liu et al., 18 Jun 2025). By contrast, the ViT-6 experiments are reported not to enter a proper decision region in the same sense and therefore remain vulnerable at every split (Liu et al., 18 Jun 2025). This indicates that depth alone is not a sufficient descriptor of inversion hardness.

A third issue concerns the object being inverted. In one branch of the literature, the inverse is a posterior kernel or posterior distribution (Braithwaite et al., 2023, Zhao et al., 2023). In another, it is a structural object such as an inverse DAG (Webb et al., 2017). In another, it is an adversarial reconstruction map from outputs or features to inputs (Suhail et al., 2024, Liu et al., 1 Jan 2025). A plausible implication is that cross-paper comparisons of “inversion quality” are meaningful only within a fixed inversion object and threat model.

Finally, the literature points toward several explicit future directions without establishing them as solved. In collaborative inference, proposed extensions include richer density models such as normalizing flows for tighter mutual-information bounds, adaptive mixtures, local Lipschitz control, and certified defenses such as differential privacy (Xia et al., 1 Mar 2025). In variational prior replacement, possible extensions include richer variational families such as normalizing flows and boosting VI, and use with Mixture Density Networks (Zhao et al., 2024). In simulator inversion, suggested extensions include automatic tuning of $y$60, block or gradient-informed proposals, reversible-jump proposals, adaptive proposals, and surrogate emulators (Saeedi et al., 2015). These are not presented as settled methodology, but they show that inference inversion remains a moving interface between probabilistic semantics, computational structure, and security constraints.