Per-Event Likelihood-Ratio Estimator

• Per-Event Likelihood-Ratio Estimator is a method that computes a local density ratio for each observation, forming the basis of optimal hypothesis testing and sequential inference.
• It integrates various techniques, including count-based, neural classifier-based, and regularized approaches to address challenges such as rare events and intractable likelihoods.
• This estimator is applied across fields like NLP, forensic genetics, and simulation-based inference, enabling efficient aggregation of evidence and adaptive decision-making.

A per-event likelihood-ratio estimator is an estimator of the likelihood ratio associated with a single observation, feature, or time step. In its simplest form, for two simple hypotheses $H_0$ and $H_1$ with densities $p_0(x)$ and $p_1(x)$, it targets $r(x)=p_1(x)/p_0(x)$ and often also its log form $\ell(x)=\log r(x)$. The same idea appears in several closely related settings: discrete event ranking with empirical counts, neural classifier-based ratio learning, amortized simulation-based inference through likelihood-to-evidence ratios, online non-parametric density-ratio estimation, and sequential decision-making through per-round likelihood-ratio contributions (Rizvi et al., 2023, Hermans et al., 2019, Emmenegger et al., 2023).

1. Formal object and statistical role

For two simple hypotheses, the per-event likelihood ratio is

$$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$

This quantity is fundamental because, by the Neyman–Pearson lemma, the uniformly most powerful test for $H_0$ versus $H_1$ uses the statistic $T=\sum_i \ell(x_i)$ for independent events. The same additive structure makes per-event log-likelihood ratios natural components of profile likelihoods, confidence intervals, and composite scores assembled from multiple features (Rizvi et al., 2023).

The definition generalizes beyond fixed hypotheses. In simulation-based inference with intractable likelihoods, one often defines the likelihood-to-evidence ratio

$H_1$0

so that the per-event likelihood ratio between two parameter values satisfies

$H_1$1

This formulation is central when the simulator can generate $H_1$2 but $H_1$3 cannot be evaluated analytically (Hermans et al., 2019).

In sequential decision-making, an “event” may be a single round $H_1$4 with context or action $H_1$5 and outcome $H_1$6. The per-event contribution is then

$H_1$7

or, in the weighted formulation,

$H_1$8

Cumulative products of these per-event quantities generate anytime-valid confidence sequences through likelihood-ratio processes (Emmenegger et al., 2023).

The term “per-event” therefore does not refer to a single implementation pattern. It refers to a granularity: one estimates a ratio contribution locally, then combines those local contributions across events by multiplication of ratios or addition of log-ratios. This common structure links hypothesis testing, reweighting, ranking, MCMC acceptance ratios, and sequential confidence construction.

2. Count-based estimators and rare-event conservatism

In discrete settings with observed counts, the most direct estimator replaces class-conditional probabilities by empirical frequencies. If $H_1$9 is the count of event $p_0(x)$0 in class $p_0(x)$1 and $p_0(x)$2, then

$p_0(x)$3

In the named-entity context-prediction task of “Conservative Likelihood Ratio Estimator for Infrequent Data Slightly above a Frequency Threshold,” the event is a word bigram immediately to the left of a named entity, the numerator context is “left-of-NE,” and the denominator context is the unconditional corpus distribution. The resulting maximum-likelihood estimator is

$p_0(x)$4

with log form

$p_0(x)$5

The paper emphasizes that low counts make this ratio unstable and heavy-tailed, so rare events tend to be over-ranked (Kikuchi et al., 2022).

A simple baseline imposes a numerator frequency threshold $p_0(x)$6: $p_0(x)$7 In that ranking task, zero means “no contribution” rather than a neutral $p_0(x)$8. This pruning improves efficiency and removes some extreme rare-event estimates, but it leaves discontinuous behavior and still overestimates just above the threshold (Kikuchi et al., 2022).

The conservative alternative proposed in the same work derives a closed-form $p_0(x)$9-regularized least-squares importance-fitting estimator on discrete one-hot bases: $p_1(x)$0 For $p_1(x)$1,

$p_1(x)$2

and

$p_1(x)$3

The correction is largest near the threshold and vanishes as $p_1(x)$4 grows. The estimator is nondecreasing in numerator count, nonincreasing in denominator count, and converges to the MLE as counts become large (Kikuchi et al., 2022).

The same paper reports a concrete large-sparse NLP experiment on the WSJ 1987 corpus with 12,000 sampled articles, split into 10,000 train, 1,000 validation, and 1,000 evaluation articles. Validation over $p_1(x)$5 selected $p_1(x)$6. In rank–recall evaluation, $p_1(x)$7-regularized estimation was best overall, thresholding improved over the naive baseline, and the $p_1(x)$8 conservative estimator slightly outperformed simple thresholding while retaining essentially the same efficiency. Measured averages over 10 runs showed substantial memory and time reductions for thresholding and $p_1(x)$9 relative to the baseline and $r(x)=p_1(x)/p_0(x)$0 estimators (Kikuchi et al., 2022).

Rare-event conservatism also appears in forensic genetics. For a rare haplotype match, the likelihood ratio for event $r(x)=p_1(x)/p_0(x)$1 is

$r(x)=p_1(x)/p_0(x)$2

with $r(x)=p_1(x)/p_0(x)$3 and, in a panmictic model without structure, $r(x)=p_1(x)/p_0(x)$4, so $r(x)=p_1(x)/p_0(x)$5. “Impact of model choice on LR assessment in case of rare haplotype match” reviews plug-in estimators, conservative upper confidence bounds such as the rule of three, a discrete Laplace model, and the nonparametric generalized-Good method. Its central methodological point is that different data reductions and population models define different legitimate likelihood ratios, so it is more sensible to speak of “an LR” rather than “the LR,” and to quantify estimation error explicitly (Cereda, 2015).

3. Classifier-based neural estimators

A broad modern approach estimates per-event likelihood ratios through probabilistic classifiers. If a classifier is trained to distinguish samples from $r(x)=p_1(x)/p_0(x)$6 and $r(x)=p_1(x)/p_0(x)$7 with class priors $r(x)=p_1(x)/p_0(x)$8 and $r(x)=p_1(x)/p_0(x)$9, the Bayes-optimal score is

$\ell(x)=\log r(x)$0

From this,

$\ell(x)=\log r(x)$1

With equal priors, $\ell(x)=\log r(x)$2 and $\ell(x)=\log r(x)$3. “Learning Likelihood Ratios with Neural Network Classifiers” stresses that any strictly monotonic function of $\ell(x)=\log r(x)$4 preserves ranking and optimal decision boundaries, but recovering the numerical likelihood ratio requires calibrated outputs and correct prior adjustment (Rizvi et al., 2023).

That paper develops a general loss-functional view. For a learnable function $\ell(x)=\log r(x)$5 and rescaling functions $\ell(x)=\log r(x)$6,

$\ell(x)=\log r(x)$7

and the extremum satisfies

$\ell(x)=\log r(x)$8

This produces several concrete loss families. BCE and MSE yield the odds mapping $\ell(x)=\log r(x)$9; MLC and SQR target $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$0 directly. The paper studies BCE, MSE, MLC, SQR, and generalized $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$1-MSE and $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$2-SQR families, together with output parameterizations on $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$3 or $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$4 such as sigmoid, Gaussian CDF, arctan-to-$$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$5, ReLU, $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$6, and $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$7 (Rizvi et al., 2023).

The empirical findings are highly specific. For BCE and MSE, sigmoid is best or close in most cases. For MLC and SQR, $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$8 is dramatically better than ReLU or $$r(x)=\frac{p_1(x)}{p_0(x)}, \qquad \ell(x)=\log r(x).$$9. In univariate Gaussian experiments, BCE and MSE outperformed MLC and SQR under naive parameterizations, while with optimized implementations the standard losses became similar and the generalized families often achieved the smallest MAE. In multivariate Gaussian cases, MLC with $H_0$0 was best among the four standard losses in the more complex problems, and $H_0$1-SQR often reduced MAE by $H_0$2–$H_0$3 relative to MLC+$H_0$4, except in the Hyperbola case where gains were limited. In the high-energy physics jet study, BCE and MSE again preferred sigmoid, MLC and SQR preferred $H_0$5, and gains from generalized families were minor because the MAE landscape was comparatively flat (Rizvi et al., 2023).

The implementation details are also part of the method. The paper uses Keras/TensorFlow, Adam, three hidden layers of widths $H_0$6–$H_0$7–$H_0$8, ReLU activations, dropout $H_0$9 after each hidden layer, early stopping with patience $H_1$0, up to $H_1$1 epochs, batch size $H_1$2 of the training set, and ensembles of $H_1$3 independent trainings per configuration. It recommends BCE or MSE with a sigmoid output when the ratio is recovered via odds, and MLC with $H_1$4 when directly regressing the likelihood ratio. Calibration through isotonic regression or Platt scaling is identified as particularly important when BCE or MSE outputs are converted to ratios (Rizvi et al., 2023).

4. Amortized estimators for likelihood-free inference

When the likelihood $H_1$5 is intractable but simulation is available, the per-event object often estimated is not directly $H_1$6, but the likelihood-to-evidence ratio

$H_1$7

where $H_1$8. The key identity

$H_1$9

allows a single amortized estimator to support many pairwise likelihood-ratio computations (Hermans et al., 2019).

“Likelihood-free MCMC with Amortized Approximate Ratio Estimators” learns $T=\sum_i \ell(x_i)$0 by binary classification between dependent pairs $T=\sum_i \ell(x_i)$1 and independent pairs $T=\sum_i \ell(x_i)$2. The Bayes-optimal classifier score is

$T=\sum_i \ell(x_i)$3

and therefore

$T=\sum_i \ell(x_i)$4

If the network outputs logits $T=\sum_i \ell(x_i)$5, then $T=\sum_i \ell(x_i)$6 and $T=\sum_i \ell(x_i)$7. The training objective is binary cross-entropy over mixed batches of joint and product-of-marginals samples, typically implemented by constructing negative examples through shuffling or the “two batches trick” (Hermans et al., 2019).

The principal advantage is amortization. One network is trained once over the prior support of $T=\sum_i \ell(x_i)$8 and the observation space of $T=\sum_i \ell(x_i)$9, then reused for posterior scanning, per-event likelihood-ratio evaluation, and MCMC acceptance probabilities. For a fixed observed event $H_1$00, the Metropolis–Hastings acceptance rule becomes

$H_1$01

The paper recommends log-domain computation, direct use of logits as $H_1$02, and proposal tuning toward acceptance rates around $H_1$03–$H_1$04 (Hermans et al., 2019).

The same estimator supports event-level hypothesis testing and dataset-level aggregation. For a single event,

$H_1$05

and for multiple i.i.d. observations $H_1$06,

$H_1$07

The paper further describes diagnostics and robustness procedures: a ROC/AUC diagnostic based on the identity $H_1$08, simulation-based calibration, posterior predictive checks, effective sample size, $H_1$09-hat, trace plots, autocorrelation, and ensembling with averaged logits for stability (Hermans et al., 2019).

This formulation is closely related to, but distinct from, classifier-based two-sample likelihood-ratio estimation. Here the classifier is conditioned on $H_1$10 and trained against the product of marginals rather than against a fixed alternative class. The paper argues that this avoids the support-mismatch issues that arise with fixed-reference classifiers, provided the simulator covers the prior support adequately (Hermans et al., 2019).

5. Online and sequential formulations

In sequential decision-making, per-event likelihood-ratio estimation is used not only to score observations but also to build time-uniform uncertainty sets. “Likelihood Ratio Confidence Sets for Sequential Decision Making” considers a sequence of contexts or actions $H_1$11 and outcomes $H_1$12, with likelihood family $H_1$13 and estimator sequence $H_1$14 chosen before observing $H_1$15. The per-event contribution is

$H_1$16

or, with adaptive weights $H_1$17,

$H_1$18

The cumulative product

$H_1$19

defines the confidence set

$H_1$20

Under measurability conditions on $H_1$21 and $H_1$22, the reciprocal process is a nonnegative supermartingale, and Ville’s inequality yields anytime-valid coverage $H_1$23 (Emmenegger et al., 2023).

The size of these sets depends on the estimator sequence. The paper therefore links per-event LR estimation to online convex optimization, especially Follow-the-Regularized-Leader: $H_1$24 For GLMs with bounded covariates and strongly convex, smooth log-partition function $H_1$25, nonasymptotic regret bounds imply corresponding controls on likelihood-ratio set size through Bregman divergences. An adaptive reweighting scheme is introduced to counteract large early bias, with

$H_1$26

and a computable upper bound

$H_1$27

The intended effect is to downweight early, poorly learned directions while allowing $H_1$28 as estimation bias decreases (Emmenegger et al., 2023).

A different online perspective appears in “Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization.” There the observed stream consists of i.i.d. pairs $H_1$29, and the target is the relative density ratio

$H_1$30

For $H_1$31, this is $H_1$32; to obtain $H_1$33, one swaps $H_1$34 and $H_1$35 or inverts the estimate when positivity permits. The method works in an RKHS and performs stochastic functional gradient descent on the Pearson-divergence objective using per-step loss

$H_1$36

The functional update is

$H_1$37

with schedules

$H_1$38

The dictionary grows by adding $H_1$39 and $H_1$40 at every step, yielding $H_1$41 per-iteration cost, $H_1$42 cumulative kernel evaluations up to time $H_1$43, and $H_1$44 memory. The paper provides high-probability convergence guarantees in both $H_1$45 and RKHS norm under bounded-kernel and smoothness assumptions (Concha et al., 2023).

These two online lines address different problems. The sequential-decision framework presumes a specified parametric likelihood and uses per-round LR contributions for coverage and control; the OLRE framework estimates density ratios non-parametrically from streaming samples. Their commonality lies in a genuinely per-event update: each new observation modifies the ratio estimator or ratio process through a single local contribution rather than batch recomputation.

6. High-dimensional spectral methods, applications, and caveats

High-dimensional per-event likelihood-ratio estimation is difficult when direct density estimation in the ambient space is poor. “High-Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation” addresses this by expanding the ratio in eigenfunctions of a kernel integral operator associated with the denominator distribution $H_1$46. If $H_1$47, the expansion is

$H_1$48

where $H_1$49 are orthonormal in $H_1$50 for $H_1$51 with density $H_1$52. Truncating at $H_1$53 terms gives

$H_1$54

with $H_1$55 obtained by Nyström extension and $H_1$56 estimated by empirical averages over samples from $H_1$57. The underlying squared-loss objective is LSIF-like, and the method exploits approximate orthogonality so that coefficients reduce to expectations of basis functions under the numerator distribution (Izbicki et al., 2014).

This basis construction adapts to the geometry of the denominator data and is intended to remain effective when observations lie near lower-dimensional manifolds embedded in high ambient dimension. The paper also extends the idea from density ratios to approximate likelihood computation. Defining

$H_1$58

with $H_1$59 often chosen as the marginal under a prior, it uses a tensor-product expansion over basis functions in $H_1$60 and $H_1$61: $H_1$62 This produces per-event approximate likelihoods usable in downstream Bayesian or maximum-likelihood inference without explicit dimension reduction (Izbicki et al., 2014).

The application range of per-event likelihood-ratio estimators is correspondingly broad. In NLP, they can rank bigrams as predictors of named-entity left contexts (Kikuchi et al., 2022). In scientific inference, they provide optimal or near-optimal test statistics, event reweighting, and confidence-interval ingredients (Rizvi et al., 2023). In likelihood-free physics-style workflows, they support MCMC, importance sampling, SMC, nested sampling, Hamiltonian Monte Carlo through $H_1$63, and nuisance-parameter marginalization (Hermans et al., 2019). In sequential learning, they generate anytime-valid confidence sequences for generalized linear bandits, survival analysis, additive-noise models, and RKHS bandits (Emmenegger et al., 2023). In forensics, they quantify evidential weight for rare haplotype matches, but the estimated LR depends strongly on the chosen population model and data reduction (Cereda, 2015).

Several caveats recur across these literatures. First, ranking quality and numeric calibration are distinct: any monotone surrogate preserves rankings, but using a score as a numerical LR requires correct calibration and, in classifier-based methods, correct treatment of training priors (Rizvi et al., 2023). Second, support overlap matters. If the numerator has support outside the denominator, the true ratio may be infinite or undefined; both simulation-based and non-parametric papers explicitly warn about this (Hermans et al., 2019, Concha et al., 2023). Third, rare events are structurally difficult: naive empirical ratios often overestimate them, thresholding can be too discontinuous, and model misspecification may dominate the error budget (Kikuchi et al., 2022, Cereda, 2015). Fourth, different modeling choices may define different legitimate likelihood ratios rather than different estimates of a single uniquely meaningful quantity; the forensic literature makes this point explicitly, and it has a plausible implication for other domains whenever evidence reduction or denominator choice is non-unique (Cereda, 2015).

A common practical pattern nevertheless emerges. One estimates a local ratio or log-ratio for each event, aggregates across events additively or multiplicatively, prunes or regularizes low-information regions, calibrates when numerical ratios are needed, and validates the estimator with task-appropriate diagnostics such as MAE, ROC/AUC-based checks, rank–recall, simulation-based calibration, posterior predictive checks, or time-uniform coverage. Within that pattern, the per-event likelihood-ratio estimator is less a single algorithm than a unifying statistical primitive that supports inference, decision-making, and evidence quantification across discrete, parametric, neural, amortized, online, and high-dimensional settings.