---
title: Posterior-Seeking Training Objective
url: https://www.emergentmind.com/topics/posterior-seeking-training-objective
type: topic
---

# Posterior-Seeking Training Objective

to=arxiv_search.search  天天送钱彩票json
{"query":"posterior-seeking training objective trajectory-balance active inference posterior collapse flow matching retriever calibration", "max_results": 10}
to=arxiv_search.search  аанацҳауеитjson
{"query":"posterior seeking objective training objective posterior collapse VAE trajectory balance", "max_results": 10}
to=arxiv_search 下载彩神争霸json
{"query":"posterior seeking training objective", "max_results": 5}
to=search_arxiv 一本道高清无码json
{"query":"posterior-seeking training objective", "max_results": 5}
A posterior-seeking training objective is a class of optimization criteria that steers a learned model toward a posterior or posterior-like distribution over latent variables, labels, trajectories, parameters, passages, or actions, rather than toward a single maximizer of reward or likelihood. In the cited literature, this idea appears in discriminative classification, variational generative modeling, Bayesian neural networks, retrieval-augmented generation, flow matching, diffusion language models, robot post-training, inverse problems, and simulation-based inference. The common motivation is to preserve informative uncertainty, incorporate priors or future evidence, mitigate collapse or overconfidence, and avoid degenerate concentration on a narrow subset of valid solutions [2207.09535] [2605.13935] [2605.28516].

## 1. Conceptual scope and contrast with mode-seeking objectives

Posterior-seeking objectives are typically introduced against a specific failure mode of conventional training. In diffusion language models, reward-maximizing post-training can induce “trajectory locking,” in which sampled updates over-concentrate probability mass onto a narrow set of denoising paths and reduce coverage of alternative correct solutions under repeated sampling [2605.13935]. In VAEs, posterior collapse occurs when the encoder posterior falls back onto the prior and the latent representation ceases to carry information about the observation [2207.09535]. In standard VAEs with powerful decoders, the optimizer can drive $\mathrm{KL}(q_\phi(z\mid x)\|p(z)) \to 0$, and $\delta$-VAEs were proposed precisely to forbid that trivial solution by requiring a minimum rate [1901.03416].

A related failure mode appears in probabilistic inference. Neural posterior estimation under limited simulation budgets can become overconfident and unreliable, and distributionally robust training was proposed to replace empirical risk minimization with a worst-case loss over a Wasserstein ambiguity set [2605.28516]. In offline robot post-training, uniform regression on heterogeneous demonstrations can average over recovery behavior, inconsistent operator skill, and weakly informative supervision, motivating sample-level reweighting based on whether a post-action consequence is attributable to the observed transition [2603.16542].

This suggests a broad distinction. A mode-seeking objective typically rewards a small set of high-scoring outcomes or paths, whereas a posterior-seeking objective attempts to preserve or recover a distribution over plausible explanations. The distinction is explicit in trajectory-balance post-training for diffusion language models, which is framed as matching a reward-tilted posterior rather than converging to a single greedy solution [2605.13935].

## 2. Mathematical forms of the posterior target

The posterior object differs by domain, but several recurring forms appear. In trajectory-balance post-training for diffusion language models, the target is a reward-tilted posterior over denoising trajectories,
$$
p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),
$$
anchored to a frozen reference model and normalized by a prompt-dependent partition function $Z(x)$ [2605.13935]. In generative policy learning, POCO defines an implicit posterior over action chunks,
$$
q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),
$$
and then distills this reward-weighted posterior into the policy through an EM procedure [2604.01860].

In Bayesian neural networks, the posterior itself is modified. “Flat Seeking Bayesian Neural Networks” replaces the standard empirical-loss term in the Gibbs posterior with the local worst-case loss inside a ball of radius $\rho$,
$$
q^{SA}(\theta\mid S)\propto p(\theta)\exp\!\bigl[-\lambda \max_{\|\theta'-\theta\|\le\rho} L_S(\theta')\bigr],
$$
yielding a sharpness-aware or flat-seeking posterior [2302.02713]. In Posterior-Augmented Flow Matching, the relevant object is the exact posterior over valid endpoints at an intermediate state,
$$
p_t(z'\mid x_t,c)=\frac{p_t(x_t\mid z')\,p(z'\mid c)}{p_t(x_t\mid c)},
$$
which is then approximated by importance sampling over multiple candidate targets [2605.00825].

Other formulations are variational rather than Gibbs-like. In posterior-guided retrieval, Hindsight introduces a guide retriever $Q_\phi(p\mid x,y)$ and optimizes
$$
\mathcal{L}_{\rm ELBo}(x,y)
= \mathbb{E}_{p\sim Q_{\phi}(p\mid x,y)}\bigl[\log P_{\theta}(y\mid x,p)\bigr]
-\mathrm{KL}\bigl(Q_{\phi}(p\mid x,y)\,\|\,P_{\eta}(p\mid x)\bigr),
$$
so that the retriever and generator are trained in expectation over a posterior over passages conditioned on both input and target output [2110.07752]. In Bayesian posterior inference via conditional flow matching, the target posterior is the Bayesian posterior $\pi(\theta\mid y^*)$, reached by learning a deterministic block-triangular ODE transport in joint data-parameter space [2510.09534].

## 3. Constructing posterior-seeking signals when the posterior is inaccessible

A recurring methodological issue is that the desired posterior is not directly available. Several works therefore construct surrogate posterior signals.

In active-inference-based classifier optimization, each sample is assigned a candidate-label set $\mathbb{L}_j^{(i)}$ computed from class priors $\pi_{j,c}$ and current network posteriors $\hat p_{j,c}^{(i)}$ through a Kelly-criterion selection rule. The resulting loss is the expected free energy of a prospective active inference, combining an uncertainty term with an expected-complexity term, and it can incorporate candidate labels, reference labels, and priors while remaining distribution-based [2306.02447]. The mechanism is posterior-seeking in the sense that it pushes posterior mass toward classes that both have high prior probability and are under-believed by the current network.

In contrastive mitigation of posterior collapse, the posterior-seeking signal is an inference critic. Menon et al. define a critic $f_\psi(x,z)$ and train it on a $K$-way classification problem that matches latent samples $z_i\sim q_\phi(z\mid x_i)$ to their observations. The critic objective
$$
c(x,z)+\log K \le I_q(x;z)
$$
is an InfoNCE-style lower bound on mutual information, so maximizing the critic term counteracts the ELBO’s implicit pressure to reduce $I_q(x;z)$ [2207.09535].

The $\delta$-VAE achieves a similar end structurally rather than contrastively. It preserves the ordinary ELBO but restricts the variational family so that
$$
\mathrm{KL}\bigl(q_\phi(z\mid x)\|p(z)\bigr)\ge \delta,
$$
thereby ensuring nonzero rate by construction and preventing the encoder posterior from collapsing onto the prior [1901.03416].

Posterior calibration offers another surrogate. PosCal training estimates an empirical posterior matrix $Q$ by binning predicted class probabilities and recording empirical frequencies of correctness per class and bin. The training loss augments cross-entropy with a divergence between predicted posterior $\hat y_i$ and empirical posterior estimate $q_i$, and the matrix $Q$ is refreshed incrementally during training [2004.14500]. This is posterior-seeking in the narrower sense of training predicted probabilities toward empirical posterior frequencies rather than performing only post hoc rescaling.

## 4. Bayesian, robust, and calibrated posterior learning

Some posterior-seeking objectives directly target reliability of posterior inference rather than representational usage or retrieval quality.

In Bayesian neural networks, the flat-seeking posterior is motivated by the observation that lower sharpness is associated with better generalization. The sharpness-aware ELBO replaces the expected empirical loss by the expected local worst-case empirical loss, and the paper proves a PAC-Bayes–style upper bound in which controlling the local worst-case empirical loss upper-bounds the true risk [2302.02713]. The resulting variational approximation is therefore not merely a posterior approximation; it is an approximation to a posterior that already encodes flatness.

In simulation-based inference, DRO-NPE replaces the standard negative log-likelihood objective with a worst-case expectation over a Wasserstein ball around the empirical simulator distribution. Under duality and a local quadratic expansion, this yields a practical loss equal to the standard NPE objective plus a gradient-norm regularizer
$$
L_{\mathrm{DRO\text{-}NPE}}(\phi)
=
L_{\mathrm{NPE}}(\phi)
+
\varepsilon\,\Omega(\phi;P_n)
\quad (+\,\mathcal{O}(\varepsilon^2)).
$$
The paper also introduces KL-based notions of miscoverage and miscalibration and proves that controlling the population NPE risk controls these quantities in KL terms [2605.28516].

Conditional flow matching for Bayesian posterior inference occupies a different point in the design space. It learns a block-triangular velocity field on the joint space $Y\times\Theta$ and optimizes the conditional flow-matching regression loss
$$
L_{\mathrm{CFM}}(\hat u)
=
\mathbb{E}_{t\sim \mathrm{Unif}[0,1],\,x_0\sim p_0,\,x_1\sim \pi(y,\theta)}
\bigl[
\|\hat u_t((1-t)x_0+t x_1)-(x_1-x_0)\|^2
\bigr].
$$
By constraining the dynamics, the learned map yields posterior samples, an inverse “vector rank,” and Bayesian credible sets whose contours correspond to level sets of Monge-Kantorovich data depth [2510.09534].

A common misconception is that posterior-seeking must mean exact Bayesian posterior recovery. The cited works show several alternatives: sharpness-aware posterior modification, distributionally robust conservative training, and deterministic transport to a posterior all fall under the same broad logic but are not identical objectives.

## 5. Generative modeling, trajectory learning, and action reweighting

Posterior-seeking objectives are especially prominent when the latent structure is combinatorial or path-valued.

Posterior-Augmented Flow Matching replaces single-target supervision with an expectation over an approximate posterior of valid target completions for a given intermediate state and condition. Instead of regressing to one endpoint $z$, the model regresses to the posterior-average velocity
$$
\mathbb{E}_{z'\sim p_t(\cdot\mid x_t,c)}[v(x_t\mid z')],
$$
approximated by self-normalized importance sampling over multiple candidates [2605.00825]. The key result is that this estimator is unbiased for the original flow-matching objective while reducing gradient variance by a factor determined by the effective sample size.

TraFL makes a parallel move for diffusion language models. It matches the current policy to the full reward-tilted posterior over trajectories through a mean-squared trajectory-balance residual, with a learned prompt-dependent normalization $\log Z_\phi(x)$. Because exact trajectory log-probabilities are unavailable, it uses a sequence-level masked-reconstruction ELBO surrogate on fully denoised completions [2605.13935]. The objective explicitly preserves the reference model’s relative weighting among same-terminal trajectories.

In generative policy learning, POCO uses an EM decomposition. The E-step forms a reward-weighted posterior over temporal action chunks, and the M-step performs weighted supervised learning with a clipped behavioral-cloning loss. The clipping bound $\zeta$ enforces a trust region in function space and is presented as protection against catastrophic “manifold collapse” [2604.01860].

PTR extends posterior-style weighting to reward-free offline robot adaptation. For each training sample, the method places the true future embedding into a pool of mismatched targets, computes an InfoNCE-style identification posterior over candidate indices, forms a posterior-to-uniform ratio score, exponentiates and clips it, and injects the resulting weight into a self-normalized weighted regression loss [2603.16542]. The objective does not require a tractable policy likelihood and is compatible with diffusion and flow-matching action heads.

A boundary case is the variational mode-seeking loss for inverse problems with diffusion models. VML minimizes
$$
\mathrm{KL}\bigl(p(x_0\mid x_t)\,\|\,p(x_0\mid y)\bigr)
$$
at each reverse diffusion step, thereby aligning the diffusion posterior with the measurement posterior [2512.10524]. However, the authors explicitly characterize the resulting behavior as mode-seeking, since reverse KL concentrates on high-density regions. This clarifies that “posterior-seeking” and “posterior-aligned” are not always synonymous with posterior mass coverage.

## 6. Empirical patterns, misconceptions, and open technical questions

Across applications, posterior-seeking objectives are reported to improve either coverage of alternative valid explanations or the reliability of the learned posterior. On Wizard of Wikipedia, Hindsight improves retriever success@10 from $52.8\%$ to $63.9\%$, generator Novel-F1@1 from $17.46$ to $20.78$, and end-to-end Novel-F1@1 from $10.45$ to $11.12$ [2110.07752]. On GLUE, PosCal reports $75.9 \to 78.4$ overall task performance and ECE $0.210 \to 0.176$, while on xSLUE it reports F1 $64.0 \to 64.5$ and ECE $0.272 \to 0.236$ [2004.14500]. In flat-seeking BNNs, the paper reports for CIFAR-100 with WideResNet28$\times$10 that SWAG versus F-SWAG changes accuracy from $82.23$ to $83.57$ and NLL from $0.6078$ to $0.5757$; it also reports largest Hessian eigenvalues dropping by $5$–$15\%$ [2302.02713].

Generative and control settings show analogous patterns. PAFM improves over FM by up to $3.4$ FID50K across SiT-B/2 and SiT-XL/2, across SiT and MMDiT, and on both ImageNet and CC12M [2605.00825]. TraFL is reported as the only evaluated post-training method that improves over the base model in every benchmark-length setting, remains above the base model on Minerva Math, and is the strongest method on every LiveCodeBench difficulty split [2605.13935]. POCO reports evaluations across $7$ simulation benchmarks and $4$ contact-rich real-world tasks and achieves a $96.7\%$ success rate on real-world tasks [2604.01860]. For inverse problems, VML-MAP reports LPIPS reductions of $10$–$30\%$ relative to DDRM, IIGDM and MAPGA, and FID reductions of $20$–$40$ points with fewer total function evaluations than DAPS-4K [2512.10524].

Several technical cautions follow. First, posterior-seeking does not imply a unique objective form: the literature includes ELBO maximization, reverse-KL minimization, expected free energy, Wasserstein-DRO, trajectory-balance residual matching, and clipped weighted regression. Second, posterior-seeking does not always mean exact posterior sampling; some methods seek a reward-tilted posterior, some a sharpness-aware posterior, and some a conservative worst-case posterior surrogate [2605.13935] [2302.02713] [2605.28516]. Third, better calibration or coverage can require extra machinery such as learned normalizers, candidate pools, higher-order autodiff, momentum encoders, or periodic recomputation of empirical posterior statistics. A plausible implication is that posterior-seeking is best understood not as a single algorithmic recipe but as a design principle: preserve and exploit posterior structure whenever purely pointwise, uniform, or mode-seeking training would erase it.

Source: https://www.emergentmind.com/topics/posterior-seeking-training-objective