---
title: Internal Label Shift Problem
url: https://www.emergentmind.com/topics/internal-label-shift-problem
type: topic
---

# Internal Label Shift Problem

The **internal label shift problem** is not a single universally standardized problem class in the arXiv literature. In its narrowest and most classical sense, it reduces to **label shift** or **prior shift**, where the class priors change across environments while class-conditional feature distributions remain invariant. In more recent work, the phrase has also been used for structurally different phenomena: hidden-layer distribution changes induced by prior changes in deep networks, internal-node reweighting in hierarchical recommendation systems, and target-aware aggregation in federated settings. A precise reading therefore requires identifying which object is treated as the “label,” which conditional distribution is assumed invariant, and whether the shift is purely marginal or coupled with covariate or class-conditional change [2003.10443], [2308.08810], [2607.10915].

## 1. Terminological status and definitional landscape

The term is non-uniform across papers. One source is explicit that the paper “never uses the term ‘internal label shift’” and that “the only shift scenario analyzed is the classical one above,” namely
$$
P(X\mid Y)=Q(X\mid Y)\quad\text{but}\quad P(Y)\neq Q(Y),
$$
with source prior \(\pi_P=P(Y=1)\), target prior \(\pi_Q=Q(Y=1)\), and shared class-conditionals \(G_0=g_0(x)\), \(G_1=g_1(x)\) [2003.10443]. By contrast, later work adopts the phrase for settings that are not merely synonymous with classical prior shift.

A compact way to organize the terminology is the following.

| Usage of “internal label shift” | Invariant object | Representative paper |
|---|---|---|
| Classical prior shift | \(p_s(x\mid y)=p_t(x\mid y)\) | [2003.10443] |
| TTA with changing target priors | target-time adaptation conditioned on estimated \(p_t(y)\) | [2308.08810] |
| Hidden-activation shift induced by priors | \(P(h_k\mid y)\) mixed by changed \(p(c)\) | [2402.04958] |
| Hierarchical internal-node shift | \(p(i,u\mid c)\) fixed while \(p(c)\to \hat p(c)\) | [2607.10915] |
| Federated target-aware aggregation | \(S_i(X\mid Y)=T(X\mid Y)\) across clients and target | [2411.03799] |

This suggests that “internal” is used in at least three different ways. In some papers it is effectively a renaming of standard label shift; in others it refers to **internal representations** of a network; in still others it refers to **internal labels** in a hierarchy, such as item attributes above item identities. The resulting ambiguity is itself part of the topic.

## 2. Classical label shift as the canonical foundation

The canonical formulation assumes
$$
p_s(x\mid y)=p_t(x\mid y)\quad\text{but}\quad p_s(y)\neq p_t(y),
$$
or equivalently an importance ratio
$$
w(y)=\frac{p_t(y)}{p_s(y)}.
$$
Under this assumption, target risk can be rewritten as a reweighted source risk, and many estimation and correction procedures reduce to estimating the target marginal \(p_t(y)\) or the weight vector \(w\) from unlabeled target data [1802.03916], [2305.19123].

Several foundational consequences recur across the literature. First, identifiability requires some form of class distinguishability. In BBSE this appears as invertibility of the predictor’s confusion matrix; in MLLS it appears as invertibility of a soft confusion matrix or strict linear independence of the class-conditional induced distributions; in mixture-model treatments it appears as linear independence of the class-conditional measures \(\{Q_i^*\}\) [1802.03916], [2003.07554], [2502.03174]. Second, correction is usually performed either by importance-weighted ERM or by posterior reweighting,
$$
\hat q(y\mid x)\propto \hat p(y\mid x)\,\hat w(y),
$$
followed by normalization [1802.03916].

The nonparametric analysis in “Minimax optimal approaches to the label shift problem in non-parametric settings” isolates a further distinction between the **unsupervised** case, where only unlabeled target samples are available, and the case with a **small number of labeled target examples**. The paper attributes the difference in difficulty to the availability of target-domain data for estimating class-conditional distributions in the latter setting, and shows that a **class proportion estimation approach is minimax rate-optimal in the unsupervised setting** [2003.10443]. This places classical label shift at the base of the broader internal-label-shift vocabulary.

## 3. Estimation, correction, and robustness under prior shift

A large part of the internal-label-shift literature, when stripped to its statistical core, concerns estimation of \(p_t(y)\) without target labels. BBSE exploits a black-box classifier \(f\) and the linear relation
$$
q_f=C\,q,
$$
where \(C_{ij}=P(f(X)=i\mid Y=j)\), \(q_f(i)=Q(f(X)=i)\), and \(q(j)=Q(Y=j)\). With plug-in estimates \(\hat C\) and \(\hat q_f\), one forms
$$
\hat q=\hat C^{-1}\hat q_f.
$$
The method is consistent under the label-shift assumption, support inclusion \(q(y)>0\implies p(y)>0\), and invertibility of \(C\) [1802.03916].

“A Unified View of Label Shift Estimation” places BBSE and maximum-likelihood estimation within a common distribution-matching perspective. It shows that **population MLLS** is consistent when the predictor is **canonically calibrated** and the soft confusion matrix is invertible, and that BBSE can be recovered from MLLS under a coarser confusion-matrix-style calibration. The paper attributes BBSE’s statistical inefficiency to information loss induced by coarse calibration [2003.07554]. ELSA then reformulates estimation as a semiparametric moment-matching problem and constructs a weight estimator by solving a \((k-1)\times(k-1)\) linear system, with \(\sqrt n\)-consistency and asymptotic normality, while explicitly aiming to avoid post-prediction calibration [2305.19123].

Robustness has also become a central theme. “Robust Label Shift Quantification” treats unlabeled target covariates as a finite-mixture sample
$$
P^*(dx)=\sum_{i=1}^k \beta_i^* Q_i^*(dx)
$$
and studies the MLE
$$
\hat\beta_{\rm MLE}\in\arg\max_{\beta\in\Delta^{k-1}}
\sum_{j=1}^n\log\Bigl(\sum_{i=1}^k \beta_i q_i(x_j)\Bigr).
$$
A key result is that the proposed robust estimators coincide with the MLE, and the paper gives deviation bounds that recover the optimal \(n^{-1/2}\) rate up to logarithmic factors in the well-specified case, together with robustness to contamination and outliers [2502.03174].

The same weighting principle also supports downstream procedures. Under label shift, calibration-error estimation without target labels can be made consistent by importance-reweighting labeled source samples within a bin-based estimator [2312.08586]. Distribution-free uncertainty quantification can likewise be restored by reweighting conformal calibration scores and binning-based post-hoc calibration by \(w(y)=q(y)/p(y)\), with exact or asymptotic coverage guarantees depending on whether the true or estimated weights are used [2103.03323].

## 4. Internal label shift in test-time adaptation and internal representations

In test-time adaptation, the phrase is used more specifically for settings in which **covariate shift and label shift co-occur** and standard entropy-based TTA becomes unstable. “Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts” formulates a labeled source dataset
\(\mathcal D_s=\{(x,y)\sim p_s(x,y)\}\), an unlabeled target stream \(\mathcal D_t=\{x\sim p_t(x)\}\), and distinguishes covariate shift, label shift, and their joint occurrence. The report states: “We refer to this setting as internal label shift in TTA” [2308.08810].

The failure mode is tied to entropy minimization,
$$
\mathcal L_{\rm ent}=-\sum_{x\sim p_t(x)} f(x)^\top \log f(x).
$$
If the source model is biased toward majority classes, entropy minimization can reinforce that bias when \(p_t(y)\) is shifted. The proposed remedy is a lightweight adapter \(G_\phi(\pi)\) that maps a candidate label prior \(\pi\in\Delta^{C-1}\) to feature-affine adjustments \(\gamma_h,\beta_h\) and classifier deltas \(\Delta W,\Delta b\), producing logits
$$
\hat y=(\gamma_h\odot h+\beta_h)\cdot (W+\Delta W)+(b+\Delta b).
$$
At test time, the target prior is estimated by an EMA of batch-mean predictions,
$$
\hat h_t=\alpha\cdot \bar y_t+(1-\alpha)\cdot \hat h_{t-1},
$$
initialized at the uniform prior \(u\), and adaptation updates only BN affine parameters or IABN parameters [2308.08810]. Reported results include a gain of **+6.1% on CIFAR-10-C backward** for Adapter+TENT over source performance, **+9.8% on CIFAR-10-C** and **+5.6% on CIFAR-100-C** for IABN+Adapter, and **72.97% versus 48.68%** on VisDA-C for TENT+Adapter relative to TENT alone [2308.08810].

A different but related formulation appears in “Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation.” There, **Internal Label Shift** is defined as the discrepancy between source and target hidden-layer activation laws
$$
P^s(h_k)=\sum_{c=1}^C p^s(c)\,P(h_k\mid y=c),
\qquad
P^t(h_k)=\sum_{c=1}^C p^t(c)\,P(h_k\mid y=c),
$$
caused purely by a change in the label prior \(p(c)\) [2402.04958]. The consequence is that BN statistics \((\mu_k,\sigma_k)\) become prior-dependent. CSN scores each channel by the squared \(2\)-Wasserstein discrepancy
$$
W_{k,f}(c)^2=(\mu_{k,f}^s-\mu_{k,f}^c)^2+(\sigma_{k,f}^s-\sigma_{k,f}^c)^2,
$$
aggregates these scores using a class prior estimate, freezes label-sensitive channels with a depth-dependent mask, and uses hybrid BN statistics
$$
\mu_{k,f}^{hyb}=m_{k,f}\mu_{k,f}^t+(1-m_{k,f})\mu_{k,f}^s,
\qquad
\sigma_{k,f}^{hyb}=m_{k,f}\sigma_{k,f}^t+(1-m_{k,f})\sigma_{k,f}^s.
$$
Empirically, under one-class extreme imbalance, **TTN loses ≈70 pp accuracy** while **Hybrid-TTN loses only ≈8 pp** on CIFAR-10-C; on ImageNet-1K-C under single-class adaptation, **TTN/TENT collapse by >70 pp** whereas **Hybrid-TTN collapses by <4 pp** [2402.04958].

These formulations retain the intuition of prior shift but relocate its effect to **adaptation dynamics** or **internal feature statistics**, rather than only to the external label marginal.

## 5. Hierarchical and distributed formulations

The strongest literal use of “internal” occurs in hierarchical recommendation. “Normative Alignment of Recommender Systems via Internal Label Shift” treats users \(U\), items \(I\), and item-level attributes \(C\), with a base recommender \(p_0(i\mid u)\) and an item-to-attribute map \(p(c\mid i)\). Under the joint model
$$
p(c,i,u)=p(c\mid i)\cdot p_0(i\mid u)\cdot p(u),
$$
the original attribute marginal is
$$
p(c)=\sum_{u\in U}\sum_{i\in I} p(c\mid i)\,p_0(i\mid u)\,p(u).
$$
Internal label shift assumes that the conditional “leaf” distributions \(p(i,u\mid c)\) remain unchanged while the marginal over the **internal labels** \(c\) is moved from \(p(c)\) to a stakeholder-defined target \(\hat p(c)\) [2607.10915].

The resulting aligned recommender is
$$
\tilde p(i\mid u)=\frac{1}{Z(u)}\,p_0(i\mid u)\,w(i),
\qquad
w(i)=\sum_{c\in C} p(c\mid i)\,\bigl[\hat p(c)/p(c)\bigr],
$$
with
$$
Z(u)=\sum_{j\in I} p_0(j\mid u)\,w(j).
$$
In the common one-category-per-item case, this reduces to per-item reweighting by \(\hat p(a(i))/p(a(i))\). NAILS then mixes the shifted and original recommenders through
$$
p_\lambda(i\mid u)=\lambda\cdot \tilde p(i\mid u)+(1-\lambda)\cdot p_0(i\mid u),
\qquad \lambda\in[0,1],
$$
and applies the procedure entirely at inference time, without retraining [2607.10915]. On EB-NeRD, **with \(\lambda\approx0.7\)** toward the editorial target, NAILS-det reduces **KL@10 from 2.060 to 0.574** and improves **coverage from 88.4 %→92.3 %**; under the uniform target, **KL@10 from 3.802→0.574** at \(\lambda\approx0.3\), with **coverage ≈85 %** [2607.10915].

Federated learning introduces yet another structure. In FedPALS, client distributions \(S_i\) and target distribution \(T\) satisfy the pure label-shift assumption
$$
S_i(X\mid Y)=S_j(X\mid Y)=T(X\mid Y),
$$
while label marginals differ across clients and target [2411.03799]. Standard FedAvg uses data-size weights \(n_i/\sum_j n_j\), which become biased for the target gradient under label shift. FedPALS instead computes
$$
\alpha^\lambda=\arg\min_{\alpha\in\Delta^{M-1}}
\Bigl\|T(Y)-\sum_{i=1}^M \alpha_i S_i(Y)\Bigr\|_2^2
+\lambda\sum_{i=1}^M \alpha_i^2/n_i,
$$
and aggregates by
$$
\theta_{t+1}=\sum_{i=1}^M \alpha_i^\lambda \theta_{i,t}.
$$
When \(T(Y)\) lies in the convex hull of \(\{S_i(Y)\}\), the \(\lambda=0\) solution yields an unbiased single-step SGD update for the target objective; as \(\lambda\to\infty\), the method recovers FedAvg [2411.03799]. Reported gains include **65.6 ± 10.1%** for FedPALS versus **44.0 ± 8.4%** for FedAvg on CIFAR-10 with 10 clients and \(C=3\) labels/client, and **86.0 ± 2.9%** versus **73.4 ± 1.6%** on PACS [2411.03799].

## 6. Generalizations, monitoring, and recurring controversies

A recurrent source of confusion is whether internal label shift is **pure prior shift** or a broader **joint-shift** phenomenon. “Optimal Transport for Conditional Domain Matching and Label Shift” explicitly assumes both
\(P_S(y)\neq P_T(y)\) and \(P_S(x\mid y)\neq P_T(x\mid y)\), and uses a weighted source loss plus a Wasserstein penalty
$$
\frac{1}{n_s}\sum_{i=1}^{n_s} w(y_i^s)\,\ell(h(g(x_i^s)),y_i^s)
+\lambda\,W_c(\tilde P_S^g,P_T^g)+\Omega(h,g)
$$
to align a reweighted source marginal with the target marginal in latent space, with target priors estimated by a mixture-plus-OT procedure [2006.08161]. The paper’s summary explicitly describes this as resolving the “internal” label shift where **both the priors and the class-conditionals have changed**. This is materially different from classical label shift.

A second generalization is **Sparse Joint Shift**, where the density ratio \(w^*(x,y)=q(x,y)/p(x,y)\) depends only on a small subset of coordinates \(x_I\) and the label \(y\), with classical label shift recovered as the special case \(m=0\), \(I=\varnothing\). Under SJS, SEES estimates both performance change and the shifted feature subset from labeled source and unlabeled target data, and empirically improves over BBSE and KLIEP on several benchmarks [2209.08436]. This suggests that some uses of “internal” in the literature function as shorthand for **structured latent or joint shift**, not just label-prior mismatch.

Monitoring and decision-theoretic responses also extend the classical problem. Sequential changepoint detection under label shift reduces the multivariate problem to one-dimensional classifier scores \(s(x)\approx P_\infty(Y=1\mid X=x)\), with a CUSUM-type statistic on the score density ratio [2009.08592]. Distributionally robust optimization instead dispenses with target samples and trains against worst-case label distributions inside a KL ball around the source marginal,
$$
\mathcal W=\{w\in\Delta^L:\;D_{kl}(w\|P_S(Y))\le r\},
$$
yielding a saddle-point problem in model parameters and adversarial label weights [2010.12230]. Active learning under label shift introduces a **medial distribution**
$$
q(y)\propto p_s(y)^\alpha p_t(y)^{1-\alpha},
$$
balancing subsampling bias and importance-weighting variance [2007.08479]. In high-dimensional binary classification, label shift can even produce a **phase transition**: in sufficiently overparameterized regimes, an imbalanced training set can outperform a balanced down-sampled counterpart, while strong regularization removes that phase transition [2212.00700].

The central controversy, therefore, is definitional rather than merely algorithmic. Some papers reserve the underlying assumption
\(p_s(x\mid y)=p_t(x\mid y)\), \(p_s(y)\neq p_t(y)\)
for the term; others apply “internal label shift” to hidden-layer shifts, internal-node marginals, client-target marginal mismatch, or joint label-plus-class-conditional shift. A plausible implication is that the term currently functions less as a settled statistical primitive than as a family resemblance around **marginal reweighting of semantically meaningful latent or observable labels while preserving a lower-level conditional structure**.

Source: https://www.emergentmind.com/topics/internal-label-shift-problem