Martingale Posterior Paradigm

• Martingale Posterior Paradigm is a framework that uses sequential one‐step-ahead predictive distributions to induce a posterior-like uncertainty instead of relying on a traditional prior–likelihood pair.
• It achieves posterior sampling by forward simulating unseen observations, forming an empirical limiting law that approximates standard Bayesian inference under suitable conditions.
• The paradigm is applicable across diverse models—from neural processes to nonparametric density estimation—while presenting challenges in ensuring predictive completeness and finite-sample calibration.

The Martingale Posterior Paradigm is a framework for posterior-like inference in which the primitive object is not a prior–likelihood pair but a predictive rule, typically a sequence of one-step-ahead predictive distributions from which unseen observations are forward-simulated and then converted into uncertainty on a target functional. In its generic form, one specifies predictive laws $P_0(\cdot)=P(Z_1\in\cdot)$ and $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$, lets these induce a limiting random empirical law $F_\infty$, and defines the martingale posterior as the induced law of $\theta(F_\infty)$ for a functional $\theta:\mathcal P(\mathcal Z)\to\Theta$ such as a risk minimizer $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$ (Ng et al., 29 Oct 2025). In this formulation, posterior sampling is implemented by predictive resampling rather than by Bayes’ formula, and classical Bayes appears as a special case when the predictive rule is the exact Bayesian posterior predictive (Ng et al., 29 Oct 2025).

1. Formal structure and inferential logic

A central formalization treats the martingale posterior as the conditional law induced by repeated forward generation from the predictive rule. If the predictive sequence converges almost surely to a random probability measure $F_\infty$, then posterior inference is defined through

$$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$

with finite computation based on the empirical law $F_N$ of the observed sample plus simulated continuations $Z_{n+1:N}$ (Ng et al., 29 Oct 2025). One posterior draw is therefore obtained by sequentially sampling $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$0 for $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$1, forming $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$2, and computing $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$3; repeating this yields posterior samples without Markov chain Monte Carlo (Ng et al., 29 Oct 2025).

This construction reverses the usual Bayesian order of exposition. Standard Bayes starts from $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$4 and $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$5, then derives the posterior and posterior predictive. The martingale posterior starts from $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$6, the predictive mechanism itself, and induces a posterior-like law through the random completion of the dataset. Several later developments summarize this viewpoint as replacing prior–likelihood elicitation with a predictive rule for forward data generation, or as “predictive Bayes” when the predictive mechanism plays the role ordinarily assigned to the prior–likelihood pair (Zhang et al., 18 May 2026).

The same logic can be expressed directly on parameter space. In the score-driven formulation, one initializes at an estimator $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$7, draws pseudo-data $P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$8, and updates by

$P_i(\cdot)=P(Z_{i+1}\in\cdot\mid Z_{1:i})$9

so that the induced law of the almost sure limit $F_\infty$0 becomes the martingale posterior (Cui et al., 3 Jan 2025). This expresses the paradigm in its most compact form: predictive resampling plus a martingale update yields a posterior-like random limit.

2. Coherence, exchangeability, and asymptotic theory

The usual sufficient condition ensuring existence of the limiting empirical law is that the predictive sequence be a martingale, equivalently conditionally identically distributed (c.i.d.), meaning

$F_\infty$1

for every measurable $F_\infty$2 (Ng et al., 29 Oct 2025). This is the main coherence requirement in the foundational literature: the predictive law must evolve so that its conditional expectation equals the current predictive. In applications built from exchangeable predictive distributions, this condition is inherited from exchangeability, and the resulting future sequence admits a de Finetti-style interpretation (Lee et al., 2023).

A major theoretical development for parametric martingale posteriors is the asymptotic theory based on plug-in predictive densities and natural-gradient stochastic approximation. In the univariate setup,

$F_\infty$3

with $F_\infty$4, yields a martingale $F_\infty$5 because the score has conditional mean zero under the current predictive law (Fong et al., 2024). Two central limit theorems then follow. The first is a predictive CLT for the tail $F_\infty$6, which justifies a hybrid sampler of the form

$F_\infty$7

thereby replacing the long predictive tail by a Gaussian correction (Fong et al., 2024). The second is a Bernstein–von Mises theorem showing that, under regularity and suitable centering, the parametric martingale posterior has the same first-order uncertainty as a regular Bayesian posterior under correct specification (Fong et al., 2024).

Convergence can also be established directly from martingale arguments in nonparametric settings. For log-concave density estimation, the recursive predictive update based on the current log-concave NPMLE generates submartingales for the objective functions $F_\infty$8, from which almost sure convergence of the estimated log-density $F_\infty$9 and asymptotic exchangeability of the generated future sequence are derived (Cui et al., 2024). In the score-function construction, almost sure convergence of the parameter martingale follows from bounded second moments, with asymptotic exchangeability of the pseudo-observations obtained from the convergence of the induced parameter law (Cui et al., 3 Jan 2025).

3. Predictive completeness and the moment hierarchy

A decisive refinement of the paradigm appears in the exchangeable Bernoulli analysis, which separates mean coherence from full predictive specification. If $\theta(F_\infty)$0 is an exchangeable Bernoulli sequence with de Finetti parameter $\theta(F_\infty)$1, then one-step prediction depends only on the first conditional moment: $\theta(F_\infty)$2 However, the $\theta(F_\infty)$3-step all-zero predictive is

$\theta(F_\infty)$4

which depends on all posterior moments up to order $\theta(F_\infty)$5 (Polson et al., 28 Feb 2026). The paper proves a non-identification result: for every $\theta(F_\infty)$6, the map

$\theta(F_\infty)$7

is set-valued rather than single-valued (Polson et al., 28 Feb 2026). In consequence, a martingale posterior that constrains only the first conditional moment of the terminal value determines one-step predictives but does not, in general, determine multi-step predictives.

The same work gives a quantitative version of this limitation. For $\theta(F_\infty)$8,

$\theta(F_\infty)$9

for some $\theta:\mathcal P(\mathcal Z)\to\Theta$0, with $\theta:\mathcal P(\mathcal Z)\to\Theta$1, hence

$\theta:\mathcal P(\mathcal Z)\to\Theta$2

(Polson et al., 28 Feb 2026). Under any strictly proper scoring rule, the plug-in predictive based only on the posterior mean is then strictly dominated by the Bayes predictive whenever the posterior is non-degenerate (Polson et al., 28 Feb 2026).

The corresponding closure theorem states that, for a bounded martingale $\theta:\mathcal P(\mathcal Z)\to\Theta$3 with terminal value $\theta:\mathcal P(\mathcal Z)\to\Theta$4, the following are equivalent: the martingale posterior determines all $\theta:\mathcal P(\mathcal Z)\to\Theta$5-step predictives; the conditional law $\theta:\mathcal P(\mathcal Z)\to\Theta$6 is uniquely determined; and the full conditional moment sequence is uniquely determined (Polson et al., 28 Feb 2026). The positive example in the paper is Hill’s $\theta:\mathcal P(\mathcal Z)\to\Theta$7 rule under the Jeffreys $\theta:\mathcal P(\mathcal Z)\to\Theta$8 prior, where the full conditional law is explicit and therefore predictive completeness holds (Polson et al., 28 Feb 2026). Within the paradigm, this result sharply delineates what a mean-coherent martingale posterior does and does not identify.

4. Historical antecedents and bootstrap connections

A short but influential discussion note places the paradigm in a broader inferential history. Draper and Guo argue that the claim that “the object of interest is fully defined once all the observations have been viewed” is closely related to Fisher consistency and “almost exactly 100 years old,” tracing it back to Fisher (1922) (Draper et al., 2023). This does not reject martingale posteriors, but it recasts one of their motivations as a historical continuation rather than a wholly new principle.

The same note also disputes a sharp separation between Bayesian and frequentist bootstrap methods. Under exchangeability,

$\theta:\mathcal P(\mathcal Z)\to\Theta$9

and with a Dirichlet process prior $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$0, the posterior is

$$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$1

Taking the low-information limit $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$2 yields the posterior $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$3 (Draper et al., 2023). The note then states the theorem that frequentist bootstrap samples of size $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$4 from $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$5 are asymptotically stochastically indistinguishable from stick-breaking samples of the same size from $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$6, and adds the empirical claim that the approximation is good to excellent even for $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$7 as small as $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$8 (Draper et al., 2023). In this reading, the ordinary bootstrap itself acquires a Bayesian nonparametric posterior interpretation, softening the divide between “Bayesian posterior” and “posterior-like uncertainty distribution.”

A related sequential Bayesian analysis sharpens the use of martingale language. For a standard posterior probability process $$\theta(F)=\arg\min_{\vartheta\in\Theta}\int \ell(z,\vartheta)\,dF(z)$$9, one has under the marginal predictive law

$F_\infty$0

while under the true law $F_\infty$1, $F_\infty$2 is a submartingale; under alternative generating parameters, however, the posterior need not be monotone and can oscillate many times in expectation, even in simple Bernoulli models (Hart et al., 2022). This is not itself the martingale posterior construction, but it clarifies that “posterior martingale” and “martingale posterior” are mathematically related yet conceptually distinct.

5. Methodological realizations across model classes

The paradigm has been instantiated in a wide range of model classes by replacing handcrafted predictive rules with model-specific predictive engines or score recursions.

Construction	Core mechanism	Citation
Martingale Posterior Neural Process	Exchangeable predictive generator of pseudo-contexts inside a Neural Process	(Lee et al., 2023)
Score-driven martingale posterior	Predictive resampling plus score updates $F_\infty$3	(Cui et al., 3 Jan 2025)
TabMGP	TabPFN as the predictive rule in a martingale posterior for tabular risk minimizers	(Ng et al., 29 Oct 2025)
Score-based martingale posteriors for DNNs	Parameter-space martingale built from pseudo-data and score/preconditioned score recursions	(Zhumekenov et al., 14 Jun 2026)
Martingale posteriors for discretely observed diffusions	Guided-bridge score surrogates inside a score-based martingale recursion	(Yao et al., 30 Apr 2026)
Federated martingale posterior sampling	One-shot client embeddings and centralized predictive resampling/refitting	(Zhang et al., 18 May 2026)
Martingale predictive model criticism for CGMs	Infinite predictive continuations replacing latent variables in posterior predictive checks	(Jesson et al., 2024)

In Neural Processes, the paradigm appears as a replacement for explicit latent priors by an exchangeable predictive generator $F_\infty$4 over pseudo-future data or representations. The resulting Martingale Posterior Neural Process uses uncertainty in generated pseudo-contexts as the source of function uncertainty and is reported to outperform baselines on various tasks (Lee et al., 2023). In the score-function line of work, the key methodological move is to build the martingale directly on parameter space from predictive draws and score increments; this requires only simulation and gradients, yields independent posterior samples by running multiple martingales in parallel, and establishes almost sure convergence of the induced parameter law (Cui et al., 3 Jan 2025).

TabMGP is a pragmatic shift in emphasis. It plugs TabPFN, a pretrained transformer for tabular posterior predictive approximation, directly into the predictive-resampling mechanism, using empirical resampling for covariates and TabPFN for $F_\infty$5. The paper reports that TabMGP produces credible sets with near-nominal coverage and often outperforms both existing MGP constructions and standard Bayes, while also showing empirically that TabPFN fails both the exact martingale and a.c.i.d. sufficient conditions (Ng et al., 29 Oct 2025). In deep neural networks, score-based martingale posteriors generate a random terminal parameter $F_\infty$6 through score recursions on parameter space; a toy classifier admits comparison with NUTS, while a larger MNIST convolutional model exposes the importance of geometry-aware preconditioning (Zhumekenov et al., 14 Jun 2026).

For low-frequency discretely observed diffusions, the score is inaccessible because the transition density is unavailable in closed form. The diffusion paper replaces the exact transition score by a guided-bridge construction and proves that the practical discretized recursion tracks the continuous-time martingale posterior in the sense

$F_\infty$7

(Yao et al., 30 Apr 2026). In federated learning, the paradigm is adapted by compressing each local dataset into a small set of trainable embeddings, sending those to a server, and then running predictive sampling centrally; the resulting Federated Martingale Posterior is described as one-shot, embarrassingly parallel, and close to the centralized counterpart on MNIST, CIFAR-10, and CIFAR-100 (Zhang et al., 18 May 2026).

A nearby but more specialized extension concerns posterior predictive model criticism for conditional generative models used in in-context learning. There the martingale ingredient is Doob-style convergence of conditional expectations, which supports replacing latent task variables by sufficiently long predictive continuations. Under the idealized assumptions, the paper proves an exact equivalence $F_\infty$8 between a posterior predictive $F_\infty$9-value and its martingale predictive analogue, and then introduces the operational generative predictive $$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$0-value as a finite-$$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$1 approximation using only black-box generation and token log probabilities (Jesson et al., 2024).

6. Limitations, controversies, and open problems

The current literature repeatedly emphasizes that exact martingale or c.i.d. conditions are sufficient but not exhaustive. TabMGP is the clearest example: TabPFN fails both the exact martingale and a.c.i.d. conditions summarized in the paper, yet the induced finite martingale posterior appears empirically stable and concentrates appropriately in a toy regression example (Ng et al., 29 Oct 2025). This suggests that the existing existence theory is too narrow for modern predictive models, but it also means that strong formal guarantees are often absent precisely in the practically most successful cases.

The sharpest conceptual limitation concerns inferential completeness. Mean coherence alone identifies only first-order predictive structure; recovering finite-horizon block predictives requires the full conditional law of the terminal value, equivalently the full conditional moment sequence (Polson et al., 28 Feb 2026). This distinction is not asymptotically fatal when posterior concentration makes the discrepancy $$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$2, but it is decisive in finite samples (Polson et al., 28 Feb 2026).

Computational and geometric issues remain substantial. In score-based martingale posteriors for deep neural networks, the mathematically simple martingale mechanism is empirically highly sensitive to preconditioning: block-diagonal EMA preconditioning is effective in a small network, whereas diagonal Fisher preconditioning in a $$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$3-parameter MNIST convnet leads to severe over-dispersion and miscalibration; the paper concludes that scalable geometry-aware preconditioners such as KFAC or layerwise block approximations are probably necessary (Zhumekenov et al., 14 Jun 2026). In the diffusion setting, the main theorem is an $$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$4 approximation result to an ideal continuous-time martingale posterior, not a theorem of Bayesian exactness, posterior consistency, or frequentist coverage (Yao et al., 30 Apr 2026).

Several application-specific boundaries are also explicit. The CGM model-criticism framework leaves formal finite-$$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$5 analysis of the approximation error between $$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$6 and $$\Pi_\infty\bigl(\theta(F_\infty)\in A\mid z_{1:n}\bigr) = \int \mathbf 1\{\theta(F_\infty)\in A\}\,d\Pi(F_\infty\mid z_{1:n}),$$7 to future work (Jesson et al., 2024). The federated construction provides no formal privacy guarantees and no explicit approximation bound quantifying closeness to centralized martingale posterior sampling as a function of summary size or meta-training mismatch (Zhang et al., 18 May 2026). The log-concave density construction proves convergence and asymptotic exchangeability but not finite-sample calibration or coverage (Cui et al., 2024).

Taken together, these developments portray the Martingale Posterior Paradigm as a spectrum rather than a single algorithmic template. At one end lie constructions with exact martingale structure and explicit asymptotics; at the other are black-box predictive mechanisms that yield strong empirical uncertainty quantification while sitting outside current sufficient-condition theory. Across that spectrum, the common idea is stable: posterior-like uncertainty is induced by coherent enough forward prediction of unseen data, and the limiting random object generated by that predictive mechanism is treated as the posterior analogue.