Nonparametric Maximum Multinomial Likelihood

• Nonparametric maximum multinomial likelihood is a family of methods that maximizes multinomial-type likelihoods over infinite-dimensional parameter spaces, providing tractable surrogates for combinatorial exact profile likelihoods.
• The approach employs convex relaxations, isotonic regression, and EM algorithms to efficiently estimate latent mixing distributions and enforce shape constraints like monotonicity.
• It offers robust theoretical guarantees and improved estimation performance in applications such as ROC analysis, rare-symbol estimation, and compound mixture modeling.

Searching arXiv for papers on nonparametric maximum multinomial likelihood, profile likelihood, and related NPMLE formulations. Could you make available the arXiv search tool or confirm its exact interface? I need it to retrieve and verify the most relevant papers before writing the article. Nonparametric maximum multinomial likelihood denotes a family of likelihood-based procedures in which multinomial, profile, or multinomial-type count structures are maximized over infinite-dimensional parameter spaces such as cdfs, mixing distributions, or shape-constrained category probabilities. In the literature represented here, the method appears in several closely related forms: as a convex Kiefer–Wolfowitz NPMLE for Poissonized multinomial counts, as a threshold-indexed multinomial likelihood for compound mixture models, and as an isotonic likelihood under monotone likelihood-ratio constraints for ordinal ROC data (Han et al., 9 Sep 2025). Closely related work on unlabeled histograms makes explicit that these procedures can be viewed as tractable surrogates for exact profile or maximum multinomial likelihood when the latter is combinatorial or unstable (Ma et al., 7 Nov 2025).

1. Core formulations

The common structural feature is that the observed data are reduced to counts or categorical summaries whose likelihood is multinomial or multinomial-type, while the unknown distributional object is left nonparametric. The resulting optimization is finite-dimensional in the observed counts but infinite-dimensional in the parameterization.

Setting	Likelihood object	Nonparametric component
Poissonized multinomial counts	$\sum_{i=1}^k \log f_G(N_i)$	Mixing distribution $G$ on $\mathbb{R}_+$
Two-group/two-season compound mixture	$l_M(\lambda,p,F_1,F_2)$	Cdfs $F_1$ and $F_2$
Ordinal ROC ratings	$\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$ with monotone $w_i=p_i/q_i$	Shape-constrained category probabilities
Unlabeled histograms	$L(\pi;N)=\sum_i \log f_\pi(N_i)$	Mixing distribution $\pi$ on $G$0

In the malaria compound-mixture model, the dry-season sample is modeled as nonmalaria and the wet-season sample as a mixture of nonmalaria and malaria individuals. For each threshold $G$1, the observations are converted into three-category multinomial indicators for zero, positive-and-below-threshold, and above-threshold events. Summing the corresponding multinomial log-likelihood contributions over a trimmed set of observed thresholds yields the maximum multinomial likelihood criterion $G$2 (Tian et al., 21 Jul 2025).

In the ROC setting, ordered ratings from nondiseased and diseased samples produce category counts $G$3 and $G$4, with likelihood

$G$5

Reparameterizing by $G$6 turns the shape restriction into the monotone likelihood-ratio constraint $G$7, so the nonparametric likelihood problem becomes isotonic (Tcheuko et al., 2013).

For large-alphabet multinomial estimation, Poissonization yields independent counts

$G$8

and the estimation target becomes the latent empirical distribution of the Poisson means. The NPMLE then maximizes

$G$9

where $\mathbb{R}_+$0 is the Poisson mixture density (Han et al., 9 Sep 2025).

2. Relation to profile likelihood, empirical Bayes, and mixture modeling

A central development is the reinterpretation of multinomial likelihood through Poisson mixtures. In the Poissonized model, writing $\mathbb{R}_+$1, one may adopt the Robbins empirical Bayes thought experiment that the $\mathbb{R}_+$2 are i.i.d. from an unknown prior $\mathbb{R}_+$3. The marginal law of a count is then

$\mathbb{R}_+$4

and the Bayes estimator of $\mathbb{R}_+$5 under squared error is the posterior mean

$\mathbb{R}_+$6

This places nonparametric maximum multinomial likelihood in a classical Robbins/Kiefer–Wolfowitz empirical Bayes framework: the latent prior is estimated by likelihood, then converted into probability assignments through the posterior-mean map (Han et al., 9 Sep 2025).

The same work emphasizes that this is “maximum multinomial likelihood in disguise.” After relaxing profile-likelihood or permutation constraints, the multinomial and Poisson average likelihoods reduce to the same symmetric objective

$\mathbb{R}_+$7

and the NPMLE arises by replacing the discrete empirical mixing measure $\mathbb{R}_+$8 with an arbitrary mixing distribution $\mathbb{R}_+$9. This yields a convex relaxation of profile maximum likelihood rather than the exact profile optimization (Han et al., 9 Sep 2025).

The unlabeled-histogram formulation makes the same distinction from another angle. For a labeled count vector one could, in principle, maximize the probability of the unlabeled multiset

$l_M(\lambda,p,F_1,F_2)$0

which is the classical profile-likelihood or maximum multinomial-likelihood viewpoint. However, the exact unlabeled likelihood is combinatorial: evaluating it requires a matrix permanent and is $l_M(\lambda,p,F_1,F_2)$1-complete. The NPMLE replaces this by the convex mixture objective

$l_M(\lambda,p,F_1,F_2)$2

thereby preserving the symmetric information in the histogram while avoiding the intractable combinatorics of exact profile maximum likelihood (Ma et al., 7 Nov 2025).

This distinction also clarifies the contrast with Good–Turing. Good–Turing is described as an $l_M(\lambda,p,F_1,F_2)$3-modeling method based on count frequencies $l_M(\lambda,p,F_1,F_2)$4, whereas the Poisson-mixture NPMLE is a $l_M(\lambda,p,F_1,F_2)$5-modeling procedure in Efron’s terminology: it models the latent mixing law and lets the posterior mean induce a smooth, monotone count-to-mean map (Han et al., 9 Sep 2025).

3. Structural constraints and likelihood geometry

Nonparametric maximum multinomial likelihood is rarely a purely unconstrained optimization. Its statistical content comes from structural restrictions that encode the geometry of the underlying problem.

In the compound-mixture malaria model, the relevant structure is twofold: a zero-versus-positive decomposition for nonmalaria individuals, and a malaria-versus-nonmalaria mixture for the wet-season sample. For thresholds $l_M(\lambda,p,F_1,F_2)$6 chosen from the interior quantiles of the positive observations, the multinomial log-likelihood decomposes into binomial-type terms through

$l_M(\lambda,p,F_1,F_2)$7

showing that the criterion is built from thresholdwise likelihood contributions while still respecting the compound mixture structure. The regularity assumptions are $l_M(\lambda,p,F_1,F_2)$8, $l_M(\lambda,p,F_1,F_2)$9, $F_1$0, and absolute continuity with common support for $F_1$1 and $F_1$2 (Tian et al., 21 Jul 2025).

In the ROC model, the governing structure is monotonicity of the categorywise likelihood ratios $F_1$3. The assumption

$F_1$4

is the discrete analogue of an increasing likelihood ratio and is equivalent to convexity of the ROC curve. Whenever adjacent unconstrained ratios violate monotonicity, the NPMLE replaces them by a common pooled value, which is exactly the effect of merging categories. The resulting estimator is not merely constrained; it is the convex hull, or greatest convex minorant, of the empirical ROC (Tcheuko et al., 2013).

For Poisson-mixture NPMLEs, structural information appears through the mixing distribution itself. In the unlabeled-histogram model, the population target is the mixture distribution $F_1$5, and the expected log-likelihood is

$F_1$6

so the estimator is naturally interpreted as a cross-entropy or KL projection onto the mixture family. Under the Poisson model,

$F_1$7

where $F_1$8. This makes the likelihood geometry explicit: the empirical histogram of counts is being matched to a smoothed mixture density (Ma et al., 7 Nov 2025).

4. Optimization and computation

The computational mechanisms differ by application, but they share a profile-likelihood logic: latent or nuisance distributional components are updated nonparametrically, then the structural parameters are updated conditionally.

For the malaria compound-mixture model, the paper derives an EM algorithm. Latent indicators $F_1$9 record whether the thresholdwise contribution of a positive wet-season observation is attributed to the malaria component $F_2$0 or the nonmalaria component $F_2$1. The E-step computes the posterior responsibilities

$F_2$2

and the corresponding upper-tail analogue $F_2$3. The M-step updates $F_2$4 and $F_2$5 in closed form and updates $F_2$6 and $F_2$7 by weighted isotonic regression, solved with the pool-adjacent-violators algorithm. The observed multinomial likelihood is nondecreasing at each EM iteration, is bounded above by $F_2$8, and converges to a stationary point; multiple starting values are therefore recommended (Tian et al., 21 Jul 2025).

For ROC estimation, computation is entirely isotonic. Starting from the unconstrained ratios

$F_2$9

the Pool Adjacent Violator Algorithm repeatedly finds adjacent violations $\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$0, pools the corresponding counts, and replaces them by the pooled ratio

$\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$1

Repeated pooling continues until the slopes are monotone, which simultaneously yields the constrained NPMLE and the convex hull of the empirical ROC (Tcheuko et al., 2013).

For Poisson-mixture NPMLEs, the optimization is convex in the mixing distribution. The fitted $\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$2 in the Poissonized multinomial problem is discrete and has at most $\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$3 atoms, with high probability only $\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$4 support points; the KKT characterization is

$\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$5

In the unlabeled-histogram model, the Poisson NPMLE is likewise discrete, unique, and supported on no more than the number of distinct observed count values, and the first-order optimality condition is

$\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$6

for all $\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$7 (Han et al., 9 Sep 2025, Ma et al., 7 Nov 2025).

5. Statistical guarantees

The theoretical properties of these estimators are heterogeneous, because the targets differ, but several themes recur: oracle comparisons, parametric-rate recovery of finite-dimensional components, and stability gains from shape constraints.

For multinomial distribution estimation under the competitive framework of Orlitsky and Suresh, the Poisson-mixture NPMLE is analyzed through the KL risk

$\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$8

with regret measured relative to the best permutation-invariant oracle. With $\prod_{i=1}^{k} p_i^{m_i} q_i^{n_i}$9 for any $w_i=p_i/q_i$0, the main theorem states that

$w_i=p_i/q_i$1

This matches the lower bound of Orlitsky and Suresh up to logarithmic factors. The same paper proves that modified Good–Turing estimators are strictly suboptimal in this framework, with regret at least

$w_i=p_i/q_i$2

for $w_i=p_i/q_i$3, regardless of the threshold parameter $w_i=p_i/q_i$4 (Han et al., 9 Sep 2025).

For unlabeled histograms, the Poisson NPMLE admits both asymptotic and non-asymptotic guarantees. Theorem 1 states that for fixed $w_i=p_i/q_i$5,

$w_i=p_i/q_i$6

and the same rate holds for $w_i=p_i/q_i$7 for any constant $w_i=p_i/q_i$8. Under a separation assumption on the support of $w_i=p_i/q_i$9, the paper also proves a global $L(\pi;N)=\sum_i \log f_\pi(N_i)$0 rate in the large-alphabet regime and establishes local mass-recovery bounds showing that $L(\pi;N)=\sum_i \log f_\pi(N_i)$1 places nearly the correct amount of mass near the true atoms. For symmetric functional estimation, the localized NPMLE yields the entropy bound

$L(\pi;N)=\sum_i \log f_\pi(N_i)$2

which matches the optimal sample complexity $L(\pi;N)=\sum_i \log f_\pi(N_i)$3 in the large-alphabet regime (Ma et al., 7 Nov 2025).

For the malaria compound-mixture model, the main theorem gives root-$L(\pi;N)=\sum_i \log f_\pi(N_i)$4 consistency for the scalar parameters and $L(\pi;N)=\sum_i \log f_\pi(N_i)$5-type average squared error rates on the threshold grid: $L(\pi;N)=\sum_i \log f_\pi(N_i)$6 and

$L(\pi;N)=\sum_i \log f_\pi(N_i)$7

The proof relies on quadratic lower bounds derived from $L(\pi;N)=\sum_i \log f_\pi(N_i)$8, on likelihood comparisons, and on the use of interior thresholds to avoid boundary degeneracy (Tian et al., 21 Jul 2025).

For ROC estimation, the relevant guarantees are bias–variance properties rather than oracle rates. The convexity-constrained NPMLE yields an AUC that is biased high relative to the ordinary empirical AUC, because taking the convex hull increases area, but it has lower variance. Relative to the underlying continuous AUC, the constrained estimator may be less biased than the empirical estimator when nonconvexity is driven by sampling variability and discretization. Standard unbiased variance estimators remain effective after PAVA pooling, with coverage close to nominal except in extreme settings involving very high AUC, small sample sizes, and many categories (Tcheuko et al., 2013).

6. Applications, empirical behavior, and methodological boundaries

The empirical record is strongly context dependent, but several recurring use-cases emerge. In rare-symbol distribution estimation, the Poisson-mixture NPMLE consistently outperforms Good–Turing and explicit Bayes procedures on synthetic heavy-tailed, uniform, step, Zipf, and Dirichlet examples, as well as on English corpora and U.S. Census data. The paper attributes this to learning a global latent prior over Poisson rates rather than estimating local profile ratios $L(\pi;N)=\sum_i \log f_\pi(N_i)$9, and reports that omission of the unseen-symbol regularizer $\pi$0 has little effect in many experiments even though $\pi$1 is theoretically essential (Han et al., 9 Sep 2025).

For unlabeled histograms and large alphabets, the NPMLE is used as a plug-in engine for entropy, support size, unseen-species estimation, and other symmetric functionals. On the Corbet butterfly dataset, the Poisson mixture fit passes $\pi$2 goodness-of-fit tests at multiple truncation levels while the direct $\pi$3-model is strongly rejected. On Moby Dick and neural spike-train data, the NPMLE-based estimators remain accurate when the observed sample size is much smaller than the effective alphabet size. For unseen-word prediction in Shakespeare’s sonnets, the NPMLE plug-in tracks the true discovery curve more closely than GT, smoothed GT, and PML; for hallucination detection based on semantic entropy, it attains the best or near-best AUCs across several model/dataset combinations while requiring only observed outputs rather than token logits (Ma et al., 7 Nov 2025).

In the malaria application, the method was applied to 144 dry-season observations and 264 wet-season observations from Tanzania. The estimated parameters were

$\pi$4

compared with

$\pi$5

for the binomial plug-in estimator, but with smaller bootstrap standard errors and shorter 95% bootstrap percentile confidence intervals under the multinomial-likelihood method. The confidence bands for $\pi$6 were notably narrower, and the estimated posterior malaria probability

$\pi$7

was increasing in log parasite level (Tian et al., 21 Jul 2025).

In diagnostic ROC analysis, the practical interpretation is equally specific. If the observer is assumed to be proper, the true ROC should be convex, and nonconvex empirical ROC segments are treated as sampling noise. The constrained NPMLE therefore replaces nonconvex portions by straight segments through PAVA pooling, producing a more stable AUC estimator and permitting variance estimation without specialized bootstrap machinery (Tcheuko et al., 2013).

Several methodological boundaries are explicit in this literature. Direct empirical likelihood can be unsuitable: in the malaria compound-mixture model it yields inconsistent estimators of $\pi$8 and $\pi$9 unless the mixture degenerates (Tian et al., 21 Jul 2025). Exact profile maximum likelihood can be computationally prohibitive, because the unlabeled histogram likelihood requires a matrix permanent and is $G$00-complete (Ma et al., 7 Nov 2025). Shape constraints can improve stability but alter bias, as seen in the ROC AUC upward adjustment (Tcheuko et al., 2013). And the classical multinomial MLE remains inadequate for unseen symbols because it is just empirical frequency and assigns zero probability to unseen categories, which is precisely the pathology that the mixture-based NPMLE is designed to avoid (Han et al., 9 Sep 2025).