True-Label Rank (TLR): An Overview

• True-Label Rank (TLR) is a metric that quantifies the position of the ground-truth label within a predicted or Bayes-optimal label ordering, enhancing traditional 0–1 accuracy.
• TLR formulations differ across settings—multiclass, score-based, and continual learning—by varying normalization, tie-breaking rules, and ranking conventions.
• Empirical results demonstrate that TLR-based methods improve top-k accuracy, calibration, and continual learning performance through refined ranking and loss adjustments.

Searching arXiv for the cited papers and topic to ground the article in current literature. True-Label Rank (TLR) denotes the position assigned to the ground-truth label, or to the ground-truth relevant labels, within a predicted or Bayes-optimal ordering of labels. Across recent arXiv literature, the construct appears in several closely related forms: as the rank of the true class in multiclass posterior orderings, as a position-sensitive statistic for ordered multi-label supervision, as a rank event underlying top-$k$ conformal prediction, and as a normalized softmax-derived signal for continual-learning dynamics (Clémençon et al., 2020, Dari et al., 2022, Shi et al., 2024, Anwar et al., 23 Jun 2026). The common thread is that TLR refines 0–1 correctness by measuring how far the correct label information has fallen in the ordered label list; the principal variations concern whether lower or higher values are preferable, whether ties are counted with $>$ or $\ge$, and whether the setting involves a single true class or an ordered set of relevant labels.

1. Core definitions and variants

In multiclass label ranking, the most direct definition is the position of the true label $Y$ in the decreasing ordering of the posterior vector $\eta(X)$, where $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$. If $\pi(X)$ orders labels by decreasing posterior probability, then

$$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$

Conditional on $X$, the distribution of TLR is

$$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$

and

$>$0

This formulation makes TLR a random variable induced by the Bayes ranking itself (Clémençon et al., 2020).

In score-based prediction systems, TLR is often computed from model scores rather than from the unknown posterior vector. For a score function $>$1, RC3P defines the true-label rank as

$>$2

and equivalently uses

$>$3

Under this convention, the true label is in the top-$>$4 predictions if and only if $>$5 (Shi et al., 2024).

In continual learning, TLR is defined from softmax probabilities with a strict inequality and then normalized to $>$6. For sample $>$7,

$>$8

and the per-sample normalized score is

$>$9

In this convention, larger values are better: $\ge$0 when the true class is top-1 and $\ge$1 when it is last (Anwar et al., 23 Jun 2026).

In ranked multi-label learning, the relevant object is not a single true class but an ordered true-label set $\ge$2. The predicted rank of a label $\ge$3 is

$\ge$4

and a mean TLR over the true labels is

$\ge$5

A top-weighted refinement is also given: $\ge$6 with decreasing $\ge$7 and normalization $\ge$8 (Dari et al., 2022).

The main definitional variants therefore differ along three axes: single-label versus multi-label ground truth, raw rank versus normalized score, and tie handling. A plausible implication is that comparisons of “TLR” across papers must first align these conventions.

Setting	TLR object	Preferred direction
Multiclass Bayes ranking	Rank of $\ge$9 in posterior ordering $Y$0	Lower is better
RC3P	Rank of true class under score ordering with $Y$1 ties	Lower is better
Continual learning	Normalized transform of true-class rank	Higher is better
Ranked multi-label	Average predicted position of true labels	Lower is better

2. Statistical role in multiclass label ranking

In the multiclass framework of label ranking, the Bayes-optimal ranking sorts labels in decreasing posterior probability, and TLR is the position of the realized label $Y$2 in that ordering (Clémençon et al., 2020). This immediately links TLR to top-$Y$3 performance: $Y$4 For $Y$5,

$Y$6

where $Y$7 is the Bayes misclassification risk (Clémençon et al., 2020). Thus, top-1 accuracy is the mass of the TLR-$Y$8 event, whereas top-$Y$9 accuracy is the mass of the TLR-$\eta(X)$0 event.

The paper "A Multiclass Classification Approach to Label Ranking" (Clémençon et al., 2020) shows that label ranking can be viewed as a partially observed ranking median regression problem. Under a Bradley–Terry–Luce–Plackett construction with preference vector $\eta(X)$1, only the top label $\eta(X)$2 is observed during training, while the full permutation is latent. Pairwise conditional probabilities become

$\eta(X)$3

and under strict stochastic transitivity the Bayes ranking is characterized by Copeland scores. This is important for TLR because any method that recovers the Bayes ranking also recovers the optimal TLR distribution.

The One-Versus-One (OVO) reduction trains a binary classifier for each pair of labels and aggregates pairwise predictions by Copeland score: $\eta(X)$4 Under the paper’s Mammen–Tsybakov-style noise condition, finite VC complexity, and minimal mass assumption, the recovered ranking $\eta(X)$5 equals the Bayes ranking with overwhelming probability for large $\eta(X)$6 (Clémençon et al., 2020). Proposition 2 in that paper further states that the top-$\eta(X)$7 risk

$\eta(X)$8

is minimized by the Bayes ranking $\eta(X)$9, which is equivalent to saying that the optimal ranking minimizes the probability of the event $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$0.

Empirically, OVO aggregation improved top-$\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$1 results on MNIST and Fashion-MNIST. On MNIST, Top-1 improved from $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$2 for logistic regression to $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$3 for OVO, and Top-5 improved from $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$4 to $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$5; on Fashion-MNIST, Top-1 improved from $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$6 to $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$7, while Top-5 remained $\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$8 for both methods (Clémençon et al., 2020). Since top-$\eta_y(x)=\mathbb{P}\{Y=y\mid X=x\}$9 accuracy is $\pi(X)$0, these improvements are direct improvements in the TLR distribution.

A common misconception is that TLR is merely another name for accuracy. The multiclass theory shows otherwise: accuracy corresponds only to the indicator $\pi(X)$1, while the full TLR distribution encodes how close incorrect predictions are to being correct.

3. Ordered multi-label supervision and RLSEP

In ranked multi-label learning, TLR becomes a position-sensitive statistic over the set of relevant labels, rather than a scalar tied to a single true class. The paper "RLSEP: Learning Label Ranks for Multi-label Classification" (Dari et al., 2022) addresses the setting in which the ground truth provides an ordered true-label set, represented either as a rank vector $\pi(X)$2 or as a partial order $\pi(X)$3 over the subset of relevant labels $\pi(X)$4. The order is ordinal: $\pi(X)$5 means that label $\pi(X)$6 should be ranked above label $\pi(X)$7, while $\pi(X)$8 has no quantitative meaning beyond ordering (Dari et al., 2022).

The paper does not use the term “True-Label Rank (TLR)” explicitly, but it builds a loss that directly optimizes the ranking of ground-truth labels by exploiting their intra-positive order (Dari et al., 2022). In that spirit, mean TLR measures the average predicted position of the true labels: $\pi(X)$9 A lower value is better because it places the true labels nearer the top of the ranking.

The paper’s central methodological contribution is the Ranked Log-Sum-Exp Pairwise loss, RLSEP: $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$0 where $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$1 is a random subset of $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$2 ordered pairs used for negative sampling, and

$$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$3

The full objective is

$$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$4

When all positives have rank $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$5 and all negatives rank $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$6, RLSEP reduces to LSEP (Dari et al., 2022).

This construction operationalizes TLR through inversion control. Defining

$$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$7

the label ranking loss is $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$8, and TLR is described as monotonically related to $$\mathrm{TLR}(X,Y)=r \quad \text{such that} \quad \pi_r(X)=Y.$$9: fewer inversions imply lower TLR and stronger alignment with the ordered ground truth (Dari et al., 2022). The loss penalizes every pairwise violation $X$0 whenever $X$1, including violations within the positive set, which standard unranked pairwise label ranking does not enforce.

The paper reports strong empirical gains on both synthetic and real ranked data. On Ranked MNIST (full), RLSEP (R) achieved ordered-pair Precision/Recall/F1/Accuracy of approximately $X$2, exact-match $X$3–$X$4, and $X$5. CE (R) and LSEP (R) were much lower on exact-match and mAP, with $X$6–$X$7–$X$8 and $X$9–$$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$0 (Dari et al., 2022). On ARC, RLSEP (R) improved exact-match from $$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$1 for CE and $$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$2 for LSEP to $$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$3, and improved mAP from $$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$4 to $$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$5 (Dari et al., 2022). The paper interprets these outcomes as substantially better alignment with true-label order, which corresponds to lower TLR in the formalization given in the details block.

Calibration findings are also stated explicitly. On a controlled 4-digit set, RLSEP-trained scores increased with importance and exhibited distinct score distributions per true rank, which the paper describes as direct evidence of TLR-oriented calibration (Dari et al., 2022). This suggests that TLR is not only an evaluation statistic but also a useful lens on score geometry.

A further connection is drawn to rank-sensitive metrics. The paper states that lower TLR aligns with higher ranked Accuracy/F1/$$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$6–$$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$7 and higher mAP, and that lower TLR typically increases DCG/NDCG, even though the reported evaluation emphasizes pairwise metrics and mAP (Dari et al., 2022).

4. TLR as a calibration signal in conformal prediction

In class-conditional conformal prediction, TLR enters as the event structure underlying top-$$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$8 inclusion. The paper "Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration" (Shi et al., 2024) defines the true-label rank from classifier scores and uses the class-conditional tail probability

$$\mathbb{P}\{\mathrm{TLR}(X,Y)=r\mid X\}=\eta_{\pi_r(X)}(X),$$9

as the class-wise top-$>$00 error. This identifies TLR as the quantity that determines whether the true class lies within the top-$>$01 region for class $>$02 (Shi et al., 2024).

RC3P builds on class-conditional conformal prediction by adding a rank gate to the usual score-based gate. Standard CCP uses class-specific quantiles of a nonconformity score $>$03 and outputs

$>$04

RC3P introduces per-class choices $>$05 and $>$06 satisfying

$>$07

and then defines

$>$08

The recommended choice is

$>$09

The resulting prediction set therefore depends simultaneously on a score gate and a TLR gate (Shi et al., 2024).

The key coverage guarantee is class-wise: $>$10 The proof sketch given in the paper is that the score gate provides at least $>$11 inclusion probability within class $>$12, while the rank gate removes at most $>$13 mass, so coverage remains at least $>$14 (Shi et al., 2024). Here TLR is the mechanism that quantifies how much mass the rank gate can safely remove.

The efficiency claim is also explicitly TLR-dependent. If

$>$15

for all $>$16, then RC3P has no larger expected prediction-set size than CCP (Shi et al., 2024). The paper introduces the per-class condition number

$>$17

and states that $>$18 for all $>$19 suffices for RC3P to dominate CCP in expected size (Shi et al., 2024).

Empirically, the paper reports that RC3P achieves class-wise coverage and a $>$20 reduction in prediction set sizes on average, with additional reductions such as $>$21 average APSS reduction over the best baseline across four imbalanced datasets and approximately $>$22 APSS reduction on balanced tasks with APS while maintaining $>$23 (Shi et al., 2024). The abstract also reports $>$24 reduction excluding CIFAR-10 (Shi et al., 2024).

A common misunderstanding is to treat TLR in conformal prediction as merely an internal rank used by APS or RAPS. The RC3P construction goes further: APS and RAPS already depend on rank inside the nonconformity score, but RC3P adds a separate per-class rank gate calibrated by class-wise TLR tails (Shi et al., 2024).

5. Continuous TLR metrics in continual learning

In continual learning, TLR is used not as a static accuracy refinement but as a continuous forgetting signal. The paper "The Gentle Collapse: Distributional Metrics for Continual Learning" (Anwar et al., 23 Jun 2026) states that accuracy degradation records only whether forgetting occurred or not, saturates at the extremes, and collapses discretely at task boundaries. TLR and related softmax-derived metrics are introduced to characterize forgetting continuously (Anwar et al., 23 Jun 2026).

The linear normalized TLR score is

$>$25

and the dataset- or class-level statistic is its sample mean. The paper also defines Log-TLR: $>$26 and Reciprocal-TLR: $>$27 All are normalized to $>$28, with larger values better (Anwar et al., 23 Jun 2026).

The paper’s central claim about TLR is comparative informativeness. At $>$29 accuracy on CIFAR-100, the Confusion Margin spans an interquartile range of $>$30 across classes that accuracy treats identically; TLR also remains discriminative, with a median of approximately $>$31, while CTL and NKL collapse near $>$32 (Anwar et al., 23 Jun 2026). This establishes that TLR retains structure in regimes where binary correctness is fully saturated.

For trend estimation, the paper computes an ordinary least squares slope over a trailing window for class-level metric trajectories: $>$33 The claim is that continuous signals such as Log-TLR yield stable slopes for short windows, whereas accuracy, being step-like, gives high-variance estimates. In the window-sweep analysis at $>$34 epochs, accuracy-trend degraded to $>$35 with standard deviation $>$36, while log-TLR achieved $>$37 with standard deviation $>$38 (Anwar et al., 23 Jun 2026).

This TLR signal is then used operationally in two ways. First, per-sample metric scores are used as inverse-proportional loss weights: $>$39 Second, class-level replay sampling probabilities are based on a softmax over slopes: $>$40 Both uses are explicitly described as exploiting the continuity of TLR-like signals (Anwar et al., 23 Jun 2026).

The reported gains are concrete. On CIFAR-100, Log-TLR-weighted loss yielded $>$41 accuracy and $>$42 forgetting versus the experience-replay baseline’s $>$43 accuracy and $>$44 forgetting, corresponding to approximately $>$45 percentage points less forgetting (Anwar et al., 23 Jun 2026). On TinyImageNet, log-TLR trend sampling achieved $>$46 accuracy and $>$47 forgetting versus $>$48 and $>$49 for the baseline, corresponding to approximately $>$50 percentage points less forgetting (Anwar et al., 23 Jun 2026).

The continual-learning usage introduces a notable reinterpretation. Here TLR is not only a ranking statistic but also a smooth proxy for latent representational degradation. This suggests that TLR can serve as a control signal whenever binary correctness is too coarse for optimization or monitoring.

6. Relations to other ranking losses, estimators, and limitations

Several papers place TLR in relation to more established ranking losses. In ranked multi-label learning, inversion counts

$>$51

yield a pairwise label ranking loss $>$52, and the details block states that this is directly related to Kendall-style inversion counts and to

$>$53

Lower TLR is said to align with higher ranked Accuracy/F1/$>$54–$>$55, higher mAP, and typically higher DCG/NDCG (Dari et al., 2022). This makes TLR a bridge between rank-sensitive evaluation and pairwise surrogate training.

In the nonparametric label-ranking framework of "Label Ranking through Nonparametric Regression" (Fotakis et al., 2021), TLR is not defined explicitly, but the details block derives a compatible notion as the predicted position of the true top label. There, if $>$56 denotes the noiseless or median-optimal ranking and $>$57 its top label, then for a predicted ranking $>$58,

$>$59

The same details block derives lower bounds

$>$60

and therefore

$>$61

This suggests a general interpretation: global ranking losses can upper-bound expected TLR, while pairwise errors involving the top label can identify the part of the loss most directly responsible for TLR degradation (Fotakis et al., 2021).

Empirical TLR estimation also differs by setting. In multiclass label ranking, given a predicted permutation $>$62, the empirical statistic is

$>$63

with summaries

$>$64

(Clémençon et al., 2020). In continual learning, the per-sample rank can be computed in $>$65 without sorting by counting how many probabilities exceed the true-class probability (Anwar et al., 23 Jun 2026). In ranked multi-label learning, TLR-like summaries can be aggregated over all relevant labels and optionally top-weighted (Dari et al., 2022).

The literature also makes several caveats explicit. First, tie handling is not uniform. RC3P uses $>$66, so ties count against the true class (Shi et al., 2024); the continual-learning paper uses strict $>$67, so ties do not increase rank and the rule becomes a dense-ranking convention (Anwar et al., 23 Jun 2026); the RLSEP formalization assumes any fixed tie-breaking when induced rankings are needed (Dari et al., 2022). Second, TLR may be insensitive to confidence changes that preserve order, a caveat stated for rank-only metrics in continual learning (Anwar et al., 23 Jun 2026). Third, richer TLR supervision can require richer annotations: RLSEP requires ordered labels rather than only binary relevance, and the paper notes that acquisition cost may therefore be higher (Dari et al., 2022).

A final misconception is that TLR is a single universally standardized metric. The papers instead support a family resemblance: all TLR formulations quantify the location of ground-truth label information within an ordering, but the operational meaning depends on whether the objective is Bayes label ranking, ranked multi-label prediction, conformal set construction, or continual-learning diagnostics.

Taken together, the recent arXiv literature presents TLR as a unifying rank-based primitive. In multiclass label ranking, it is the canonical random variable underlying top-$>$68 accuracy and Bayes-optimal orderings (Clémençon et al., 2020). In ranked multi-label learning, it becomes a statistic over ordered true-label sets that RLSEP reduces by minimizing pairwise violations (Dari et al., 2022). In class-conditional conformal prediction, its tail probabilities determine per-class rank gates that preserve coverage while reducing set size (Shi et al., 2024). In continual learning, normalized TLR variants provide continuous, actionable signals that remain informative when accuracy saturates (Anwar et al., 23 Jun 2026).