Posterior-Seeking Training Objective

• Posterior-seeking training objectives are optimization methods that steer models to maintain broad posterior distributions over latent variables, labels, or trajectories rather than converging to a single mode.
• They mitigate issues like posterior collapse and overconfidence by incorporating priors, future evidence, and uncertainty preservation in diverse applications such as VAEs and diffusion models.
• These objectives are applied in Bayesian neural networks, generative modeling, and simulation-based inference, enhancing model calibration, robustness, and performance.

to=arxiv_search.search 天天送钱彩票json {"query":"posterior-seeking training objective trajectory-balance active inference posterior collapse flow matching retriever calibration", "max_results": 10} to=arxiv_search.search аанацҳауеитjson {"query":"posterior seeking objective training objective posterior collapse VAE trajectory balance", "max_results": 10} to=arxiv_search 下载彩神争霸json {"query":"posterior seeking training objective", "max_results": 5} to=search_arxiv 一本道高清无码json {"query":"posterior-seeking training objective", "max_results": 5} A posterior-seeking training objective is a class of optimization criteria that steers a learned model toward a posterior or posterior-like distribution over latent variables, labels, trajectories, parameters, passages, or actions, rather than toward a single maximizer of reward or likelihood. In the cited literature, this idea appears in discriminative classification, variational generative modeling, Bayesian neural networks, retrieval-augmented generation, flow matching, diffusion LLMs, robot post-training, inverse problems, and simulation-based inference. The common motivation is to preserve informative uncertainty, incorporate priors or future evidence, mitigate collapse or overconfidence, and avoid degenerate concentration on a narrow subset of valid solutions (Menon et al., 2022, Ahmadi et al., 13 May 2026, Laplante et al., 27 May 2026).

1. Conceptual scope and contrast with mode-seeking objectives

Posterior-seeking objectives are typically introduced against a specific failure mode of conventional training. In diffusion LLMs, reward-maximizing post-training can induce “trajectory locking,” in which sampled updates over-concentrate probability mass onto a narrow set of denoising paths and reduce coverage of alternative correct solutions under repeated sampling (Ahmadi et al., 13 May 2026). In VAEs, posterior collapse occurs when the encoder posterior falls back onto the prior and the latent representation ceases to carry information about the observation (Menon et al., 2022). In standard VAEs with powerful decoders, the optimizer can drive $\mathrm{KL}(q_\phi(z\mid x)\|p(z)) \to 0$, and $\delta$-VAEs were proposed precisely to forbid that trivial solution by requiring a minimum rate (Razavi et al., 2019).

A related failure mode appears in probabilistic inference. Neural posterior estimation under limited simulation budgets can become overconfident and unreliable, and distributionally robust training was proposed to replace empirical risk minimization with a worst-case loss over a Wasserstein ambiguity set (Laplante et al., 27 May 2026). In offline robot post-training, uniform regression on heterogeneous demonstrations can average over recovery behavior, inconsistent operator skill, and weakly informative supervision, motivating sample-level reweighting based on whether a post-action consequence is attributable to the observed transition (Zhang et al., 17 Mar 2026).

This suggests a broad distinction. A mode-seeking objective typically rewards a small set of high-scoring outcomes or paths, whereas a posterior-seeking objective attempts to preserve or recover a distribution over plausible explanations. The distinction is explicit in trajectory-balance post-training for diffusion LLMs, which is framed as matching a reward-tilted posterior rather than converging to a single greedy solution (Ahmadi et al., 13 May 2026).

2. Mathematical forms of the posterior target

The posterior object differs by domain, but several recurring forms appear. In trajectory-balance post-training for diffusion LLMs, the target is a reward-tilted posterior over denoising trajectories,

$$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$

anchored to a frozen reference model and normalized by a prompt-dependent partition function $Z(x)$ (Ahmadi et al., 13 May 2026). In generative policy learning, POCO defines an implicit posterior over action chunks,

$$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$

and then distills this reward-weighted posterior into the policy through an EM procedure (Chen et al., 2 Apr 2026).

In Bayesian neural networks, the posterior itself is modified. “Flat Seeking Bayesian Neural Networks” replaces the standard empirical-loss term in the Gibbs posterior with the local worst-case loss inside a ball of radius $\rho$,

$$q^{SA}(\theta\mid S)\propto p(\theta)\exp\!\bigl[-\lambda \max_{\|\theta'-\theta\|\le\rho} L_S(\theta')\bigr],$$

yielding a sharpness-aware or flat-seeking posterior (Nguyen et al., 2023). In Posterior-Augmented Flow Matching, the relevant object is the exact posterior over valid endpoints at an intermediate state,

$$p_t(z'\mid x_t,c)=\frac{p_t(x_t\mid z')\,p(z'\mid c)}{p_t(x_t\mid c)},$$

which is then approximated by importance sampling over multiple candidate targets (Stoica et al., 1 May 2026).

Other formulations are variational rather than Gibbs-like. In posterior-guided retrieval, Hindsight introduces a guide retriever $Q_\phi(p\mid x,y)$ and optimizes

$$\mathcal{L}_{\rm ELBo}(x,y) = \mathbb{E}_{p\sim Q_{\phi}(p\mid x,y)}\bigl[\log P_{\theta}(y\mid x,p)\bigr] -\mathrm{KL}\bigl(Q_{\phi}(p\mid x,y)\,\|\,P_{\eta}(p\mid x)\bigr),$$

so that the retriever and generator are trained in expectation over a posterior over passages conditioned on both input and target output (Paranjape et al., 2021). In Bayesian posterior inference via conditional flow matching, the target posterior is the Bayesian posterior $\delta$0, reached by learning a deterministic block-triangular ODE transport in joint data-parameter space (Jeong et al., 10 Oct 2025).

3. Constructing posterior-seeking signals when the posterior is inaccessible

A recurring methodological issue is that the desired posterior is not directly available. Several works therefore construct surrogate posterior signals.

In active-inference-based classifier optimization, each sample is assigned a candidate-label set $\delta$1 computed from class priors $\delta$2 and current network posteriors $\delta$3 through a Kelly-criterion selection rule. The resulting loss is the expected free energy of a prospective active inference, combining an uncertainty term with an expected-complexity term, and it can incorporate candidate labels, reference labels, and priors while remaining distribution-based (Fallah, 2023). The mechanism is posterior-seeking in the sense that it pushes posterior mass toward classes that both have high prior probability and are under-believed by the current network.

In contrastive mitigation of posterior collapse, the posterior-seeking signal is an inference critic. Menon et al. define a critic $\delta$4 and train it on a $\delta$5-way classification problem that matches latent samples $\delta$6 to their observations. The critic objective

$\delta$7

is an InfoNCE-style lower bound on mutual information, so maximizing the critic term counteracts the ELBO’s implicit pressure to reduce $\delta$8 (Menon et al., 2022).

The $\delta$9-VAE achieves a similar end structurally rather than contrastively. It preserves the ordinary ELBO but restricts the variational family so that

$$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$0

thereby ensuring nonzero rate by construction and preventing the encoder posterior from collapsing onto the prior (Razavi et al., 2019).

Posterior calibration offers another surrogate. PosCal training estimates an empirical posterior matrix $$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$1 by binning predicted class probabilities and recording empirical frequencies of correctness per class and bin. The training loss augments cross-entropy with a divergence between predicted posterior $$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$2 and empirical posterior estimate $$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$3, and the matrix $$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$4 is refreshed incrementally during training (Jung et al., 2020). This is posterior-seeking in the narrower sense of training predicted probabilities toward empirical posterior frequencies rather than performing only post hoc rescaling.

4. Bayesian, robust, and calibrated posterior learning

Some posterior-seeking objectives directly target reliability of posterior inference rather than representational usage or retrieval quality.

In Bayesian neural networks, the flat-seeking posterior is motivated by the observation that lower sharpness is associated with better generalization. The sharpness-aware ELBO replaces the expected empirical loss by the expected local worst-case empirical loss, and the paper proves a PAC-Bayes–style upper bound in which controlling the local worst-case empirical loss upper-bounds the true risk (Nguyen et al., 2023). The resulting variational approximation is therefore not merely a posterior approximation; it is an approximation to a posterior that already encodes flatness.

In simulation-based inference, DRO-NPE replaces the standard negative log-likelihood objective with a worst-case expectation over a Wasserstein ball around the empirical simulator distribution. Under duality and a local quadratic expansion, this yields a practical loss equal to the standard NPE objective plus a gradient-norm regularizer

$$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$5

The paper also introduces KL-based notions of miscoverage and miscalibration and proves that controlling the population NPE risk controls these quantities in KL terms (Laplante et al., 27 May 2026).

Conditional flow matching for Bayesian posterior inference occupies a different point in the design space. It learns a block-triangular velocity field on the joint space $$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$6 and optimizes the conditional flow-matching regression loss

$$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$7

By constraining the dynamics, the learned map yields posterior samples, an inverse “vector rank,” and Bayesian credible sets whose contours correspond to level sets of Monge-Kantorovich data depth (Jeong et al., 10 Oct 2025).

A common misconception is that posterior-seeking must mean exact Bayesian posterior recovery. The cited works show several alternatives: sharpness-aware posterior modification, distributionally robust conservative training, and deterministic transport to a posterior all fall under the same broad logic but are not identical objectives.

5. Generative modeling, trajectory learning, and action reweighting

Posterior-seeking objectives are especially prominent when the latent structure is combinatorial or path-valued.

Posterior-Augmented Flow Matching replaces single-target supervision with an expectation over an approximate posterior of valid target completions for a given intermediate state and condition. Instead of regressing to one endpoint $$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$8, the model regresses to the posterior-average velocity

$$p^*(\tau\mid x)=\frac{1}{Z(x)}\,p_{\mathrm{ref}}(\tau\mid x)\,\exp\bigl(\beta\,r(x,y)\bigr),$$9

approximated by self-normalized importance sampling over multiple candidates (Stoica et al., 1 May 2026). The key result is that this estimator is unbiased for the original flow-matching objective while reducing gradient variance by a factor determined by the effective sample size.

TraFL makes a parallel move for diffusion LLMs. It matches the current policy to the full reward-tilted posterior over trajectories through a mean-squared trajectory-balance residual, with a learned prompt-dependent normalization $Z(x)$0. Because exact trajectory log-probabilities are unavailable, it uses a sequence-level masked-reconstruction ELBO surrogate on fully denoised completions (Ahmadi et al., 13 May 2026). The objective explicitly preserves the reference model’s relative weighting among same-terminal trajectories.

In generative policy learning, POCO uses an EM decomposition. The E-step forms a reward-weighted posterior over temporal action chunks, and the M-step performs weighted supervised learning with a clipped behavioral-cloning loss. The clipping bound $Z(x)$1 enforces a trust region in function space and is presented as protection against catastrophic “manifold collapse” (Chen et al., 2 Apr 2026).

PTR extends posterior-style weighting to reward-free offline robot adaptation. For each training sample, the method places the true future embedding into a pool of mismatched targets, computes an InfoNCE-style identification posterior over candidate indices, forms a posterior-to-uniform ratio score, exponentiates and clips it, and injects the resulting weight into a self-normalized weighted regression loss (Zhang et al., 17 Mar 2026). The objective does not require a tractable policy likelihood and is compatible with diffusion and flow-matching action heads.

A boundary case is the variational mode-seeking loss for inverse problems with diffusion models. VML minimizes

$Z(x)$2

at each reverse diffusion step, thereby aligning the diffusion posterior with the measurement posterior (Gutha et al., 11 Dec 2025). However, the authors explicitly characterize the resulting behavior as mode-seeking, since reverse KL concentrates on high-density regions. This clarifies that “posterior-seeking” and “posterior-aligned” are not always synonymous with posterior mass coverage.

6. Empirical patterns, misconceptions, and open technical questions

Across applications, posterior-seeking objectives are reported to improve either coverage of alternative valid explanations or the reliability of the learned posterior. On Wizard of Wikipedia, Hindsight improves retriever success@10 from $Z(x)$3 to $Z(x)$4, generator Novel-F1@1 from $Z(x)$5 to $Z(x)$6, and end-to-end Novel-F1@1 from $Z(x)$7 to $Z(x)$8 (Paranjape et al., 2021). On GLUE, PosCal reports $Z(x)$9 overall task performance and ECE $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$0, while on xSLUE it reports F1 $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$1 and ECE $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$2 (Jung et al., 2020). In flat-seeking BNNs, the paper reports for CIFAR-100 with WideResNet28$$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$310 that SWAG versus F-SWAG changes accuracy from $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$4 to $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$5 and NLL from $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$6 to $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$7; it also reports largest Hessian eigenvalues dropping by $$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$8–$$q(a_{1:T}\mid s_{1:T}) \propto \exp\!\bigl(R(\tau)/\eta\bigr)\, p_\theta(a_{1:T}\mid s_{1:T}),$$9 (Nguyen et al., 2023).

Generative and control settings show analogous patterns. PAFM improves over FM by up to $\rho$0 FID50K across SiT-B/2 and SiT-XL/2, across SiT and MMDiT, and on both ImageNet and CC12M (Stoica et al., 1 May 2026). TraFL is reported as the only evaluated post-training method that improves over the base model in every benchmark-length setting, remains above the base model on Minerva Math, and is the strongest method on every LiveCodeBench difficulty split (Ahmadi et al., 13 May 2026). POCO reports evaluations across $\rho$1 simulation benchmarks and $\rho$2 contact-rich real-world tasks and achieves a $\rho$3 success rate on real-world tasks (Chen et al., 2 Apr 2026). For inverse problems, VML-MAP reports LPIPS reductions of $\rho$4–$\rho$5 relative to DDRM, IIGDM and MAPGA, and FID reductions of $\rho$6–$\rho$7 points with fewer total function evaluations than DAPS-4K (Gutha et al., 11 Dec 2025).

Several technical cautions follow. First, posterior-seeking does not imply a unique objective form: the literature includes ELBO maximization, reverse-KL minimization, expected free energy, Wasserstein-DRO, trajectory-balance residual matching, and clipped weighted regression. Second, posterior-seeking does not always mean exact posterior sampling; some methods seek a reward-tilted posterior, some a sharpness-aware posterior, and some a conservative worst-case posterior surrogate (Ahmadi et al., 13 May 2026, Nguyen et al., 2023, Laplante et al., 27 May 2026). Third, better calibration or coverage can require extra machinery such as learned normalizers, candidate pools, higher-order autodiff, momentum encoders, or periodic recomputation of empirical posterior statistics. A plausible implication is that posterior-seeking is best understood not as a single algorithmic recipe but as a design principle: preserve and exploit posterior structure whenever purely pointwise, uniform, or mode-seeking training would erase it.