TPP-Diversity Threshold: Context & Implications

• TPP-Diversity Threshold is a context-dependent mechanism used to regulate diversity in machine learning, instantiated through RPD thresholds, adaptive top-p sampling, or tokens-per-parameter variance.
• In reasoning-path curation, an RPD-based threshold (τ) ensures minimum semantic divergence, leading to measurable gains like a +2.80% increase in pass@16 performance.
• Adaptive decoding and scaling-law applications use tailored thresholds to balance factuality versus diversity and guarantee robust model extrapolation through diverse TPP designs.

Searching arXiv for papers that use or operationalize “TPP-Diversity Threshold” across relevant contexts. “TPP-Diversity Threshold” is not a single standardized construct in the arXiv literature. Recent papers use the phrase, or an operational equivalent, for at least three different thresholding mechanisms: an RPD-based threshold $\tau$ that enforces minimum divergence among reasoning chains, a token-wise adaptive top-$p$ threshold $p_t$ that trades off diversity and factuality during decoding, and a design-level threshold on tokens-per-parameter diversity required for well-conditioned scaling-law estimation. Several of the underlying papers explicitly do not define any native “TPP” quantity, so the term functions less as a canonical definition than as a context-dependent label for a diversity-controlling threshold (Ju et al., 30 Oct 2025, Chang et al., 2024, Kricheli et al., 8 May 2026).

1. Terminological scope

The term has a notably heterogeneous scope. In “Reasoning Path Divergence: A New Metric and Curation Strategy to Unlock LLM Diverse Thinking,” the paper “does not define any ‘TPP’ concept or threshold,” and the expression is operationalized as an RPD-based threshold $\tau$ for selecting reasoning paths that are “sufficiently different” in their intermediate steps (Ju et al., 30 Oct 2025). In “REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy,” the “TPP-Diversity Threshold” is the operational top-$p$ parameter used at each generation step, with REAL making that threshold adaptive through a hallucination-hazard forecast (Chang et al., 2024). In “Tokens-per-Parameter Coverage Is Critical for Robust LLM Scaling Law Extrapolation,” TPP-diversity refers to variation of the tokens-per-parameter ratio $k=D/N$ across training runs, and the paper derives a closed-form threshold for when that diversity is sufficient for well-conditioned estimation (Kricheli et al., 8 May 2026).

Context	Thresholded quantity	Operational meaning
Reasoning-path curation	$\tau \in [0,1]$ on RPD	Minimum semantic divergence between CoTs
Adaptive decoding	$p_t$	Per-token top-$p$ threshold
Scaling-law design	$V_K \ge T_K$	Sufficient TPP variation across runs

This multiplicity matters because the same phrase can denote a semantic-distance gate, a decoding-mass cutoff, or a design-matrix conditioning criterion. A plausible implication is that any technical use of the term should specify the underlying object being thresholded before discussing calibration, guarantees, or downstream effects.

2. RPD-based thresholding for reasoning-path diversity

In the RPD framework, diversity is defined over chains of thought by first summarizing each long CoT into a short list of $p$0–$p$1 logical steps and then comparing those steps in embedding space. Let $p$2 and $p$3 be two solutions, with step summaries $p$4 and $p$5, assuming $p$6, and embeddings $p$7. The paper uses cosine similarity

$p$8

cosine distance

$p$9

nearest-neighbor step matching

$p_t$0

and the asymmetric Reasoning Path Divergence

$p_t$1

The score is low if every step in $p_t$2 is covered semantically by some step in $p_t$3, and high if $p_t$4 contains semantically novel steps relative to $p_t$5 (Ju et al., 30 Oct 2025).

Within this interpretation, the TPP-Diversity Threshold is simply a scalar $p_t$6 such that a pair of solutions is considered “diverse enough” when $p_t$7. The implementation guide specifies max-min diversity with thresholding: greedily add the candidate that maximizes its minimum RPD to the current set, subject to all pairwise distances to the set being at least $p_t$8; if no candidate satisfies the constraint, relax $p_t$9 by $\tau$0. The same guide also defines a problem-level intrinsic diversity score,

$\tau$1

which can itself be thresholded by $\tau$2 for problem selection (Ju et al., 30 Oct 2025).

The surrounding training paradigm is one problem, multiple solutions (1PNS). The reported pipeline uses OpenThought3 with “53,125 math problems; 16 Long-CoT per problem,” filters to “$\tau$3 high-quality problems,” summarizes each solution into “3–5 logical steps,” embeds steps with Qwen3-Embedding-8B, and fine-tunes Qwen3-4B-Base using SFT with “4-bit QLoRA (rank=16, alpha=32),” “12 epochs, BF16, AdamW, cosine LR schedule, LR peak $\tau$4, batch size 16” (Ju et al., 30 Oct 2025). The paper itself “does not specify numeric $\tau$5; selection is greedy by diversity score without a hard threshold,” but the implementation guidance recommends calibrating $\tau$6 from pairwise RPD distributions, notes that “clearly diverse pairs often score around $\tau$7–$\tau$8,” and gives “a practical starting range for $\tau$9” of $p$0–$p$1, with $p$2 and $p$3 as a starting policy (Ju et al., 30 Oct 2025).

Empirically, the diversity-aware curation underlying this thresholded interpretation improves high-$p$4 reasoning performance. Reported gains include “average $p$5 pass@16,” “$p$6 on AIME24,” and a MATH500 Level 5 pass@16 increase from “75.00\%” for “Base (1P1S random)” to “79.29\%” for “RPD (ours)” (Ju et al., 30 Oct 2025). The paper does not report an ablation directly over $p$7, but its results are used to justify thresholded variants for both training-time curation and test-time scaling.

3. Adaptive top-$p$8 thresholding in REAL sampling

In REAL sampling, the TPP-Diversity Threshold is not a set-level semantic distance but the top-$p$9 truncation parameter applied at each generation step. Standard nucleus sampling uses a global threshold $k=D/N$0 to keep the smallest prefix of sorted next-token probabilities whose cumulative mass exceeds $k=D/N$1. REAL replaces that static parameter with an adaptive, per-token threshold $k=D/N$2 that decreases when hallucination hazard is high and increases when hazard is low (Chang et al., 2024).

The key quantities are the current model’s next-token entropy $k=D/N$3, an asymptotic entropy prediction $k=D/N$4, and residual entropy

$k=D/N$5

Operationally, the THF model predicts a smoothed entropy $k=D/N$6 for the generation LLM and an asymptote $k=D/N$7, so the predicted residual entropy is

$k=D/N$8

REAL then maps hazard to the adaptive threshold through

$k=D/N$9

where $\tau \in [0,1]$0 controls the aggressiveness of the diversity–factuality tradeoff (Chang et al., 2024).

The forecasting model is a “tiny decoder-only transformer,” initialized from the smallest model in the family, using “the last 40 tokens” of context to predict the parameters of an entropy-decay curve. The paper uses a square-root MSE objective across model sizes and contexts,

$\tau \in [0,1]$1

and studies fractional-polynomial, exponential, and logistic parameterizations, with “fractional polynomials vs exp/logistic” reported as “comparable” (Chang et al., 2024).

The threshold here is directly tied to decoding behavior. Higher residual entropy produces smaller $\tau \in [0,1]$2, which prunes the tail of the token distribution and prioritizes factuality; lower residual entropy yields larger $\tau \in [0,1]$3, which admits more lexical variety. The paper reports that “REAL sampling based on a 70M THF model can substantially improve the factuality and diversity of 7B LLMs simultaneously,” and that “REAL+CD clearly dominates CD alone” (Chang et al., 2024). Concrete examples include Top-$\tau \in [0,1]$4 $\tau \in [0,1]$5 to REAL $\tau \in [0,1]$6 improving Entail_R from “7.925” to “9.377” while lowering NE_ER from “40.171\%” to “38.850\%,” and CD $\tau \in [0,1]$7 to REAL+CD $\tau \in [0,1]$8 improving Entail_R from “9.380” to “11.394” (Chang et al., 2024).

The calibration parameters are correspondingly different from the RPD setting. The guide states that “$\tau \in [0,1]$9 in the range $p_t$0–$p_t$1 gave good tradeoffs,” that lower $p_t$2 gives “higher factuality, lower diversity,” and that practitioners often clip $p_t$3 to $p_t$4, with $p_t$5 suggested as a starting range for pure REAL (Chang et al., 2024). In this usage, the threshold is token-local, continuous, and hazard-driven rather than pairwise or archive-level.

4. TPP-diversity as a scaling-law conditioning criterion

A third meaning appears in scaling-law methodology. Here TPP means tokens-per-parameter, with ratio

$p_t$6

for model size $p_t$7 and token count $p_t$8. “TPP-diversity” is variation of $p_t$9 across the runs used to fit a scaling law. A single fixed-$p$0 design is “fully collinear,” because all runs lie on the ray $p$1; a multi-$p$2 design is a fan of rays; a genuinely non-collinear design samples a two-dimensional region of the $p$3 plane (Kricheli et al., 8 May 2026).

The paper argues that fixed-TPP fitting is inherently ill-conditioned when the exponents on $p$4 and $p$5 are close. For the Chinchilla-style law

$p$6

the relevant exponent gap is $p$7, with analogous definitions for Kaplan and Droppo–Elibol laws. Under a single TPP ray, the Jacobian columns for the scale coefficients become nearly proportional, producing a Gauss–Newton condition number that scales as

$p$8

The paper states that this makes the scale coefficients “practically unidentifiable,” inflates confidence intervals “by an order of magnitude or more,” and degrades extrapolation “off the training ray” (Kricheli et al., 8 May 2026).

The threshold itself is defined on the variance of transformed TPP ratios. For $p$9 distinct TPP values $V_K \ge T_K$0, with effective exponent $V_K \ge T_K$1, define

$V_K \ge T_K$2

and

$V_K \ge T_K$3

Proposition 2 gives the leading-order criterion

$V_K \ge T_K$4

For $V_K \ge T_K$5, $V_K \ge T_K$6, so the threshold can never hold; the paper therefore treats single-TPP designs as inherently ill-conditioned (Kricheli et al., 8 May 2026).

The two-ray reduction makes the design requirement especially explicit. If $V_K \ge T_K$7 and $V_K \ge T_K$8, then the minimum spread needed to achieve a target condition-number bound is

$V_K \ge T_K$9

The paper’s practical guidance states that “as few as two distinct $p$00’s suffice,” that “endpoint placement maximizes $p$01 at fixed support,” and that for “typical Chinchilla $p$02–$p$03 and $p$04–$p$05,” one typically needs “$p$06–$p$07 for $p$08” (Kricheli et al., 8 May 2026).

This threshold is neither semantic nor decoding-time. It is a property of experiment design. The empirical result attached to it is correspondingly global: “non-collinear designs outperform collinear ones on held-out splits with a $p$09 win rate,” and “NC raises unified holdout $p$10 from $p$11 to $p$12 and cuts RMSE from $p$13 to $p$14” (Kricheli et al., 8 May 2026).

5. Related diversity-threshold formulations

The broader literature contains related thresholding mechanisms that illuminate the term’s usage even when “TPP” is absent. In dialogue generation, “Semantic Diversity in Dialogue with Natural Language Inference” introduces Diversity Threshold Generation, which iteratively resamples the lowest-contributing response in a set until a semantic diversity threshold is met. The paper defines several NLI-based diversity scores, including

$p$15

with class weights $p$16, $p$17, $p$18, and a confidence-weighted version

$p$19

For $p$20 responses, the paper uses the threshold $p$21 with $p$22, corresponding to “at least half of ordered pairs contradict,” and reports an “average $p$23 increase in NLI Diversity” over standard generation procedures (Stasaski et al., 2022).

A different but conceptually adjacent use appears in in-context learning. “Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression” defines pretraining diversity as the number of unique latent regression vectors $p$24 in the pretraining task distribution. The paper identifies a “task diversity threshold” beyond which the pretrained transformer ceases to behave like the Bayes-optimal estimator for the finite pretraining distribution and instead aligns with ridge regression on new tasks. For the base model with $p$25, the threshold is reported to lie “between $p$26 and $p$27 pretraining tasks” (Raventós et al., 2023).

These related formulations do not establish a unified TPP notion. They do, however, show a recurring pattern: diversity thresholds are used either to gate admissible solution sets, to alter search or sampling behavior, or to characterize when a qualitative regime change occurs.

6. Common misconceptions and limitations

A common misconception is that “TPP-Diversity Threshold” names a single transferable scalar. The literature instead attaches the term to different mathematical objects. In the RPD setting, it is a threshold on pairwise divergence between reasoning trajectories; in REAL, it is a token-level probability-mass cutoff; in scaling-law analysis, it is a sufficient condition on the variance of transformed tokens-per-parameter ratios (Ju et al., 30 Oct 2025, Chang et al., 2024, Kricheli et al., 8 May 2026).

Another misconception is that the threshold is always native to the original paper. In the RPD case, the paper “does not define any ‘TPP’ concept or diversity threshold separate from RPD,” and the thresholded formulation is an implementation-ready operationalization layered on top of the paper’s greedy max-min selection procedure (Ju et al., 30 Oct 2025). By contrast, in REAL the adaptive threshold is native to the decoding algorithm itself, and in the scaling-law paper the threshold is a theorem-level object with a necessary-and-sufficient leading-order condition (Chang et al., 2024, Kricheli et al., 8 May 2026).

Failure modes also differ by context. For RPD thresholding, “$p$28 too high” leads to infeasibility and possible loss of coverage, whereas “$p$29 too low” leads to redundancy and weaker test-time-scaling gains (Ju et al., 30 Oct 2025). For REAL, systematic overestimation of residual entropy can “hurt diversity,” while clipping $p$30 and raising $p$31 are suggested mitigations (Chang et al., 2024). For scaling-law design, no amount of sample count repairs a single-ray design because “single-TPP designs are inherently ill-conditioned regardless of sample size” under the stated regime (Kricheli et al., 8 May 2026).

The most defensible encyclopedia-level conclusion is therefore contextual rather than universal: a TPP-Diversity Threshold is best understood as a thresholding device that controls diversity relative to a particular representation—reasoning paths, token distributions, or training-design geometry. Any technical discussion that omits that representation risks conflating fundamentally different objects.