--- title: 'TPP-Diversity Threshold: Context & Implications' url: https://www.emergentmind.com/topics/tpp-diversity-threshold type: topic --- # TPP-Diversity Threshold: Context & Implications Searching arXiv for recent papers that use or operationalize “TPP-Diversity Threshold” across relevant contexts. “TPP-Diversity Threshold” is not a single standardized construct in the arXiv literature. Recent papers use the phrase, or an operational equivalent, for at least three different thresholding mechanisms: an RPD-based threshold $\tau$ that enforces minimum divergence among reasoning chains, a token-wise adaptive top-$p$ threshold $p_t$ that trades off diversity and factuality during decoding, and a design-level threshold on tokens-per-parameter diversity required for well-conditioned scaling-law estimation. Several of the underlying papers explicitly do not define any native “TPP” quantity, so the term functions less as a canonical definition than as a context-dependent label for a diversity-controlling threshold [2510.26122], [2406.07735], [2605.08541]. ## 1. Terminological scope The term has a notably heterogeneous scope. In “Reasoning Path Divergence: A New Metric and Curation Strategy to Unlock LLM Diverse Thinking,” the paper “does not define any ‘TPP’ concept or threshold,” and the expression is operationalized as an RPD-based threshold $\tau$ for selecting reasoning paths that are “sufficiently different” in their intermediate steps [2510.26122]. In “REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy,” the “TPP-Diversity Threshold” is the operational top-$p$ parameter used at each generation step, with REAL making that threshold adaptive through a hallucination-hazard forecast [2406.07735]. In “Tokens-per-Parameter Coverage Is Critical for Robust LLM Scaling Law Extrapolation,” TPP-diversity refers to variation of the tokens-per-parameter ratio $k=D/N$ across training runs, and the paper derives a closed-form threshold for when that diversity is sufficient for well-conditioned estimation [2605.08541]. | Context | Thresholded quantity | Operational meaning | |---|---|---| | Reasoning-path curation | $\tau \in [0,1]$ on RPD | Minimum semantic divergence between CoTs | | Adaptive decoding | $p_t$ | Per-token top-$p$ threshold | | Scaling-law design | $V_K \ge T_K$ | Sufficient TPP variation across runs | This multiplicity matters because the same phrase can denote a semantic-distance gate, a decoding-mass cutoff, or a design-matrix conditioning criterion. A plausible implication is that any technical use of the term should specify the underlying object being thresholded before discussing calibration, guarantees, or downstream effects. ## 2. RPD-based thresholding for reasoning-path diversity In the RPD framework, diversity is defined over chains of thought by first summarizing each long CoT into a short list of $3$–$5$ logical steps and then comparing those steps in embedding space. Let $S_A$ and $S_B$ be two solutions, with step summaries $L_A=\{a_1,\dots,a_m\}$ and $L_B=\{b_1,\dots,b_n\}$, assuming $m \le n$, and embeddings $e_{a_i}, e_{b_j} \in \mathbb{R}^d$. The paper uses cosine similarity $$ s_{ij}=\frac{e_{a_i}\cdot e_{b_j}}{\|e_{a_i}\|\|e_{b_j}\|}, $$ cosine distance $$ \delta_{ij}=1-s_{ij}, $$ nearest-neighbor step matching $$ d_i=\min_{j \in \{1,\dots,n\}} \delta_{ij}, $$ and the asymmetric Reasoning Path Divergence $$ D(S_A,S_B)=\frac{1}{m}\sum_{i=1}^m d_i. $$ The score is low if every step in $A$ is covered semantically by some step in $B$, and high if $A$ contains semantically novel steps relative to $B$ [2510.26122]. Within this interpretation, the TPP-Diversity Threshold is simply a scalar $\tau \in [0,1]$ such that a pair of solutions is considered “diverse enough” when $\mathrm{RPD}(S_i,S_j)\ge \tau$. The implementation guide specifies max-min diversity with thresholding: greedily add the candidate that maximizes its minimum RPD to the current set, subject to all pairwise distances to the set being at least $\tau$; if no candidate satisfies the constraint, relax $\tau$ by $\Delta$. The same guide also defines a problem-level intrinsic diversity score, $$ \mathrm{Score}_{\mathrm{div}}(P)=\frac{2}{k(k-1)}\sum_{i0$ controls the aggressiveness of the diversity–factuality tradeoff [2406.07735]. The forecasting model is a “tiny decoder-only transformer,” initialized from the smallest model in the family, using “the last 40 tokens” of context to predict the parameters of an entropy-decay curve. The paper uses a square-root MSE objective across model sizes and contexts, $$ L=\sqrt{\frac{1}{|B|N}\sum_{c\in B}\sum_{i=1}^N \big(e_c^{\theta_{s_i}}-e_c(s_i)\big)^2}, $$ and studies fractional-polynomial, exponential, and logistic parameterizations, with “fractional polynomials vs exp/logistic” reported as “comparable” [2406.07735]. The threshold here is directly tied to decoding behavior. Higher residual entropy produces smaller $p_t$, which prunes the tail of the token distribution and prioritizes factuality; lower residual entropy yields larger $p_t$, which admits more lexical variety. The paper reports that “REAL sampling based on a 70M THF model can substantially improve the factuality and diversity of 7B LLMs simultaneously,” and that “REAL+CD clearly dominates CD alone” [2406.07735]. Concrete examples include Top-$p$ $(p=0.6)$ to REAL $(T=2.0)$ improving Entail\_R from “7.925” to “9.377” while lowering NE\_ER from “40.171\%” to “38.850\%,” and CD $(\alpha=0.3)$ to REAL+CD $(T=1.5)$ improving Entail\_R from “9.380” to “11.394” [2406.07735]. The calibration parameters are correspondingly different from the RPD setting. The guide states that “$T$ in the range $1.5$–$2.0$ gave good tradeoffs,” that lower $T$ gives “higher factuality, lower diversity,” and that practitioners often clip $p_t$ to $[0.3,0.9]$, with $[0.2,0.95]$ suggested as a starting range for pure REAL [2406.07735]. In this usage, the threshold is token-local, continuous, and hazard-driven rather than pairwise or archive-level. ## 4. TPP-diversity as a scaling-law conditioning criterion A third meaning appears in scaling-law methodology. Here TPP means tokens-per-parameter, with ratio $$ k=\frac{D}{N}, $$ for model size $N$ and token count $D$. “TPP-diversity” is variation of $k$ across the runs used to fit a scaling law. A single fixed-$k$ design is “fully collinear,” because all runs lie on the ray $D=kN$; a multi-$k$ design is a fan of rays; a genuinely non-collinear design samples a two-dimensional region of the $(N,D)$ plane [2605.08541]. The paper argues that fixed-TPP fitting is inherently ill-conditioned when the exponents on $N$ and $D$ are close. For the Chinchilla-style law $$ L(N,D)=AN^{-\alpha}+BD^{-\beta}+E, $$ the relevant exponent gap is $\epsilon=|\alpha-\beta|$, with analogous definitions for Kaplan and Droppo–Elibol laws. Under a single TPP ray, the Jacobian columns for the scale coefficients become nearly proportional, producing a Gauss–Newton condition number that scales as $$ \kappa(J^T J)=O(\epsilon^{-2}). $$ The paper states that this makes the scale coefficients “practically unidentifiable,” inflates confidence intervals “by an order of magnitude or more,” and degrades extrapolation “off the training ray” [2605.08541]. The threshold itself is defined on the variance of transformed TPP ratios. For $K$ distinct TPP values $k_1,\dots,k_K$, with effective exponent $\beta_{\mathrm{eff}}$, define $$ V_K:=\mathrm{Var}(\{k_\ell^{\beta_{\mathrm{eff}}}\}_{\ell=1}^K) $$ and $$ T_K:=\frac{\left(K+\sum_{\ell=1}^K k_\ell^{2\beta_{\mathrm{eff}}}\right)^2}{K^2 K_{\mathrm{target}}}. $$ Proposition 2 gives the leading-order criterion $$ \kappa(J^T J)\le K_{\mathrm{target}} \quad \text{if and only if} \quad V_K \ge T_K. $$ For $K=1$, $V_1=0$, so the threshold can never hold; the paper therefore treats single-TPP designs as inherently ill-conditioned [2605.08541]. The two-ray reduction makes the design requirement especially explicit. If $k_1