Likelihood Matching

Updated 3 July 2026

Likelihood Matching is a framework that minimizes discrepancies between target and model likelihoods by matching scores or higher-order derivatives for robust model adaptation.
It is applied across diffusion models, bandit algorithms, graph alignment, and quantum error correction to ensure precise estimation through moment and curvature matching.
The approach enables adaptive corrections and cross-model distillation, thereby improving statistical optimality, mitigating catastrophic forgetting, and enhancing sample quality.

Likelihood Matching is a principle and methodological framework for ensuring statistical, algorithmic, or distributional alignment between models—typically by matching likelihoods, their gradients (scores), or higher derivatives—under various generative, inference, or decision-making paradigms. The term spans several technical domains, including diffusion models, bandit algorithms, empirical likelihood, cross-modal/model distillation, graph matching, and quantum error correction. The unifying theme is the use of likelihood-based objectives to enable precise adaptation, statistical optimality, or efficient transfer of information, often under partial knowledge or with mismatched representations. The following sections provide a comprehensive account of likelihood matching, as substantiated by major results in recent literature.

1. Core Principles of Likelihood Matching

At its core, likelihood matching refers to minimizing the discrepancy between two likelihood expressions—typically that of a target distribution (or process) and a model's (potentially parametrized) likelihood. For models with tractable likelihoods, this amounts to standard maximum likelihood estimation. However, when likelihoods are implicit, inaccessible, or defined only through sequential, conditional, or local structure (as in diffusion or bandit settings), likelihood matching generalizes to objectives that enforce agreement between distributions in terms of their moments, scores, or more generally, their Kullback–Leibler divergences and higher-order differential quantities.

In diffusion models, likelihood matching not only recovers the standard score-matching objective in the first order, but also encompasses the matching of transition moments—modeled via parametric or quasi-likelihood surrogates that approximate the reverse process (Qian et al., 5 Aug 2025). In neural-linear bandits, likelihood matching constructs Bayesian priors to ensure that the predictive distribution under evolving feature representations faithfully continues the model's previous uncertainty, eliminating catastrophic forgetting (Nabati et al., 2021, Zahavy et al., 2019). In empirical likelihood, data-dependent priors are constructed to ensure frequentist coverage properties by matching posterior and sampling distributions to higher order (0805.3203).

2. Likelihood Matching in Diffusion Models and Generative Inference

Diffusion generative models admit multiple likelihood matching paradigms:

A. Pathwise Likelihood Matching via Quasi-Likelihoods

Recent work establishes an exact equivalence between maximizing the marginal likelihood of the target data and maximizing the sum of reverse-path transition likelihoods in time-discretized SDEs. However, the true reverse transition densities are generally intractable. Practical algorithms replace these with Gaussian quasi-likelihoods whose mean and covariance match the first and second conditional moments, parameterized in terms of score and Hessian networks. Likelihood matching then reduces to minimizing the pathwise negative log quasi-likelihood, forcing both scores and Hessians to approximate their true values, thus yielding a consistent, theoretically justified estimator with explicit convergence rates (Qian et al., 5 Aug 2025).

B. High-Order Score-Based Likelihood Matching

For score-based diffusion ODEs, matching only the first-order score is insufficient to guarantee maximal likelihood, since the ODE log-likelihood depends on higher-order derivatives. Bounding the likelihood gap necessitates explicit control over second- and third-order score-matching errors. High-order denoising score matching (DSM) objectives explicitly construct estimators for Hessians and related quantities, ensuring both theoretical control of the likelihood gap and empirical improvement in negative log-likelihood and sample quality (Lu et al., 2022).

C. Likelihood-Curvature Adaptive Corrections

In conditional generative inference (e.g., 3D point cloud reconstruction from images), adaptive likelihood matching algorithms compute step sizes for likelihood updates along the negative gradient direction by explicitly matching local curvature. Forward curvature matching (FCM) estimates the optimal step size using forward automatic differentiation and finite differences, guaranteeing monotonically decreasing loss, faster convergence, and improved fidelity relative to fixed-step schemes (Shin et al., 9 Nov 2025).

3. Likelihood Matching in Conditional Generation and Bandits

A. Conditional Score-Based Generation

Standard conditional sampling in diffusion models relies on decomposing the conditional score into the sum of the marginal score and the gradient of the log-likelihood (i.e., classifier term). However, if the classifier is trained solely via cross-entropy, its gradient can deviate significantly from the true likelihood score, leading to a score-mismatch and degraded generated samples. The Denoising Likelihood Score Matching (DLSM) objective directly aligns the classifier's gradients with the true likelihood score (via a tractable denoising loss), provably removing this mismatch and improving conditional sample quality, class-precision, and recall (Chao et al., 2022).

B. Likelihood Matching under Representation Drift

In neural-linear bandit algorithms, exploration benefits realized via earlier feature representations can be lost if the model's representation changes, leading to catastrophic forgetting. Likelihood matching transfers the sufficient statistics of the Bayesian posterior (mean and covariance) so that the new prior over updated features matches the moment structure (i.e., the predictive mean and variance) of the prior under the old features. This typically involves solving a moment-matching semidefinite program for the covariance and a linear regression for the mean, yielding resilience to forgetting and theoretical regret guarantees equivalent to the non-drifting case (Nabati et al., 2021, Zahavy et al., 2019).

4. Likelihood Matching in Matching, Graph Alignment, and Quantum Decoding

A. Likelihood-Matching in Graph Alignment

For inexact graph matching where one observed graph is a corrupted version of another (under edge-flipping noise and vertex shuffling), the graph matching task is, under the corrupting channel model, precisely the maximum likelihood estimation of the hidden vertex permutation. The objective matches the likelihood by minimizing the number of mismatched edges, and admits rigorous consistency thresholds: for various random graph families, tight necessary and sufficient noise bounds are available for consistent recovery of the underlying permutation (Arroyo et al., 2018).

B. Quantum Error Correction Decoding

In practical QEC decoders such as minimum-weight perfect matching (MWPM), edge weights are ideally calibrated to negative log-likelihoods of error events. Likelihood matching, in this context, denotes the refinement of matching-based decoders by incorporating calibrated edge-weights (from laboratory error probabilities), critical higher-order correlations, or weight-tuning via cross-entropy minimization, which brings their decoding performance close to full maximum-likelihood decoders while maintaining scalability to large quantum systems (Sundaresan et al., 2022).

C. Multi-Label Likelihood Matching in Semantic Graph Matching

In global robot localization, object associations are matched between prior and observed semantic graphs. Preserving a multi-label distribution at each node, likelihood matching fuses observed frequency and detection confidence across overlapping category hypotheses, propagating context via neighbor likelihoods to enhance data association, pose estimation, and scalability to vocabularies spanning thousands of classes (Lee et al., 3 Dec 2025).

5. Likelihood Matching for Cross-Model and Cross-Tokenizer Distillation

In large-LLM distillation, standard techniques usually require matching next-token likelihoods across teacher and student models sharing a tokenizer. Approximate Likelihood Matching (ALM) generalizes distillation to arbitrarily different tokenizations by aligning chunk-level (multi-token) probabilities on shared detokenized substrings and matching their probabilities under a binarized f-divergence, subject to tokenization-bias constraints. The loss is minimized if and only if the student matches the teacher's text-level distribution wherever tokenization permits. This approach enables effective cross-tokenizer and cross-architecture knowledge transfer, performance ensemble constructions, and unlocks distillation across conventional modality and pre/post-processing boundaries (Minixhofer et al., 25 Mar 2025).

6. Likelihood Matching in Empirical, Data-Dependent, and Frequentist/Bayesian Frameworks

Empirical likelihood and related nonparametric approaches may seek Bayesian priors ("probability matching priors") so that resulting posterior quantiles match frequentist coverage up to O( $n^{-1}$ ) or higher. Mukerjee (0805.3203) provides a sharp characterization of when such matching is feasible, showing that only certain forms of empirical-type likelihoods admit a data-dependent prior with frequentist validity to a given expansion order. Explicit forms for the required prior (typically incorporating sample skewness and kurtosis) are constructed, with clear Edgeworth expansions linking the likelihood/prior combination to posterior quantile accuracy.

7. Theoretical and Practical Implications, Limitations, and Extensions

The adoption of likelihood matching frameworks frequently yields statistically efficient, adaptation-robust, and interpretable models, with convergence rates and finite-sample guarantees specified in terms of moment-matching errors, regularity, and sample/sample-path discretization (e.g., $O(d^3\log^{4.5}T / T)$ rates for diffusion model sampling in $d$ dimensions (Qian et al., 5 Aug 2025)). In certain modalities, such as empirical likelihood, data-dependent priors are a necessary ingredient to achieve higher-order frequentist matching, with practical recipes and limitations precisely delineated (0805.3203).

Despite broad efficacy, likelihood matching objectives may introduce algorithmic overhead, additional estimation (Hessian, higher-order differentials), or reliance on side information (covariances, class probabilities, cross-tokenizer alignment statistics). Performance can be limited by model mismatch, finite data, non-invertibility in bandit replay, or tokenization bias in NLP settings. Nonetheless, extensions addressing contraction (ODE stabilization (Li et al., 2 Oct 2025)), cross-modal transfer, and hybrid quantum-classical decoders are active research frontiers.

Principal references: