---
title: ListCascade in Bandit PAC Learning
url: https://www.emergentmind.com/topics/listcascade
type: topic
---

# ListCascade in Bandit PAC Learning

ListCascade is an algorithmic principle and concrete algorithm for multiclass PAC learning with bandit feedback in the realizable setting. It was introduced as the central mechanism for obtaining the upper bound in “PAC Learning with Bandit Feedback: Sharp Sample Complexity in the Realizable Setting,” where it reduces bandit multiclass learning to a sequence of list-learning problems by maintaining lists of plausible labels, shrinking those lists epoch by epoch, and terminating when each list becomes a singleton classifier [2605.25678].

## 1. Formal setting and basic object

ListCascade is defined for multiclass PAC learning with bandit feedback in the realizable case. The instance space is \(\mathcal{X}\), the label space is \(\mathcal{Y}=[K]=\{1,\dots,K\}\), and the concept class is \(\mathcal{H}\subseteq \mathcal{Y}^{\mathcal{X}}\). The data distribution \(\mathcal{D}\) on \(\mathcal{X}\times\mathcal{Y}\) is assumed realizable by \(\mathcal{H}\): there exists \(h^\star\in\mathcal{H}\) such that \(\mathcal{L}_{\mathcal{D}}(h^\star)=0\). Training examples are i.i.d., but the learner does not observe labels directly. Instead, on round \(i\) it receives an unlabeled instance \(x_i\), predicts \(\hat y_i\), and observes only the bandit feedback bit
\[
f_i=\mathds{1}\{\hat y_i=y_i\}.
\]
The input to ListCascade is an unlabeled sample \(S=\{x_1,\dots,x_m\}\subseteq\mathcal{X}\), together with \(\epsilon,\delta\). Its output is a multiclass classifier \(h_S:\mathcal{X}\to\mathcal{Y}\) [2605.25678].

The algorithmic premise is that bandit feedback can occasionally be converted into full-information labeled examples. If the learner predicts from a restricted list of plausible labels and the bandit bit is \(1\), then the predicted label must equal the true label. ListCascade exploits this by maintaining a list predictor \(\mu_t\) and using bandit interaction as a filter that produces labeled samples only when the current randomized prediction is correct.

## 2. Epoch structure and cascade mechanism

ListCascade proceeds in epochs \(t=1,2,\dots,\lfloor\log K\rfloor\). It begins with the maximal list predictor
\[
\mu_0(x)=\mathcal{Y}\qquad\text{for all }x\in\mathcal{X},
\]
so every label is initially plausible. In epoch \(t\), the learner uses the previous list predictor \(\mu_{t-1}\) to explore: for each incoming instance \(x_i\) assigned to that epoch, it samples a prediction uniformly from the current list,
\[
\hat y_i \sim \operatorname{Unif}(\mu_{t-1}(x_i)).
\]
If \(y_i=\hat y_i\), the learner records \((x_i,y_i)\) in a labeled sample \(S_t\); otherwise the example is discarded. The new list-size target is
\[
L_t=\left\lceil \frac{K}{2^{t+1}}\right\rceil.
\]

The new predictor \(\mu_t\) is built from \(S_t\) via an \(L_t\)-list learner applied to multiple prefixes of \(S_t\), followed by majority vote on lists. With \(n_t=|S_t|\), the construction is
\[
\mu_t=\operatorname{Maj}\!\Big(\mu^{L_t}_{S_{t,\le \lceil n_t/4\rceil}},\dots,\mu^{L_t}_{S_{t,\le n_t}}\Big),
\]
where \(\mu^{L_t}_{S_{t,\le n}}\) denotes the one-inclusion \(L_t\)-list learner trained on the first \(n\) examples of \(S_t\). After the last epoch \(T=\lfloor\log K\rfloor\), \(\mu_T(x)\) is a singleton list for all \(x\), and the final classifier \(h_S(x)\) is defined as its unique element [2605.25678].

The cascade in the name refers to the geometric contraction of list sizes:
\[
K \to K/2 \to K/4 \to \dots \to 1.
\]
The exploration overhead in epoch \(t\) scales with the current list size rather than with the full label count. The expected success probability of converting a bandit round into a labeled example is at least approximately
\[
\Pr[\hat y_i=y_i]\approx \frac{1}{|\mu_{t-1}(x_i)|},
\]
up to a factor accounting for the current error of \(\mu_{t-1}\). The epoch schedule used in the analysis is
\[
m_t-m_{t-1}
=
O\!\left(
\frac{
L_{t-1}\left(
d_{DS}^{\lceil L_{t-1}/2\rceil}(\log K)^2+\log(1/\delta)
\right)\log K
}{\epsilon}
\right),
\]
where \(d_{DS}^{\lceil L_{t-1}/2\rceil}=\lceil L_{t-1}/2\rceil\text{-}\operatorname{DS}(\mathcal{H})\). An inductive invariant states that with high probability
\[
\mathcal{L}_{\mathcal{D}}(\mu_t)\le \frac{t\epsilon}{\log K},
\]
so after at most \(\log K\) epochs the final error is at most \(\epsilon\).

## 3. Reduction to list learning

The reduction hinges on list learning. A list predictor of size \(L\) is a function
\[
\mu:\mathcal{X}\to \{Y\subseteq\mathcal{Y}:|Y|\le L\},
\]
with error
\[
\mathcal{L}_{\mathcal{D}}(\mu)=\Pr_{(x,y)\sim\mathcal{D}}[y\notin \mu(x)].
\]
List learning is easier than ordinary multiclass learning because correctness requires only that the true label appear somewhere in the output list.

ListCascade uses the one-inclusion list algorithm \(\mathcal{A}_{\mathcal{H}}^L\), attributed in the paper to Charikar–Pabbaraju and Hanneke–Meng–Moran–Shaeiri, together with a majority-vote operator on lists. Given list predictors \(\mu_1,\dots,\mu_n\) of size \(L\),
\[
\mu(x)=\operatorname{Maj}(\mu_1,\dots,\mu_n)(x)
:=
\{y\in\mathcal{Y}: |\{i:y\in\mu_i(x)\}|\ge n/2\}.
\]
The resulting list has size at most \(2L-1\), padded to exactly \(2L-1\) if needed. This converts expected-error guarantees into PAC guarantees [2605.25678].

The analytical bridge consists of several lemmas. If \(\mathcal{H}\subseteq[K]^n\) has \(\lceil L/2\rceil\)-DS dimension \(d\), then
\[
\operatorname{E}_L(\mathcal{H}) \le 6\log K\cdot d.
\]
For realizable \(\mathcal{D}\) and \(n>0\), the one-inclusion \(L\)-list learner satisfies the leave-one-out guarantee
\[
\Pr_{(S,(x,y))\sim\mathcal{D}^{n+1}}\big[y\notin \mu_S^L(x)\big]
\le
\frac{6\log K}{n+1}.
\]
From this, the paper obtains a PAC bound: with probability at least \(1-\delta\), there exists a list predictor \(\mu\) of size \(2L-1\) such that
\[
\mathcal{L}_{\mathcal{D}}(\mu)
\le
9.64\left(
\frac{6\log K}{n}
+
\frac{1}{n}\log\frac{2}{\delta}
\right).
\]

In ListCascade, bandit interaction is used to produce labeled samples \(S_t\) that are conditionally i.i.d. from a distribution restricted to instances where the current list predictor contains the true label. The list learner is then run on \(S_t\), and majority vote over multiple prefixes yields a high-probability bound. This reduction is the algorithmic core of ListCascade.

## 4. Bandit \(\mathrm{DS}\) dimension, pseudo-boxes, and sample complexity

The upper bound achieved by ListCascade is expressed in terms of a new combinatorial parameter, the bandit \(\mathrm{DS}\) dimension. Its definition uses generalized structures called pseudo-boxes. For functions \(f,g\in\mathscr{Y}^d\), \(f\) and \(g\) are \(i\)-neighbors if they differ only at coordinate \(i\):
\[
f_i\neq g_i
\quad\text{and}\quad
f_j=g_j \text{ for all } j\neq i.
\]
An \(N_1\times\cdots\times N_d\) pseudo-box is informally a finite set \(T\subseteq\mathcal{Y}^d\) such that, for each \(f\in T\) and each coordinate \(i\), there are at least \(N_i-1\) distinct \(i\)-neighbors of \(f\) in \(T\), together realizing \(N_i\) distinct labels at coordinate \(i\). Pseudo-cubes arise as the special case \(N_i=2\) for all \(i\) [2605.25678].

Given \(\mathcal{H}\subseteq\mathcal{Y}^{\mathcal{X}}\), a sequence \(S=(x_1,\dots,x_d)\in\mathcal{X}^d\) and multiplicity vector \(N\in\mathbb{N}^d\) are BDS-shattered by \(\mathcal{H}\) if there exists a finite \(F\subseteq\mathcal{H}\) such that for every \(f\in F|_S\) and each coordinate \(i\), \(f\) has at least \(N_i\) distinct \(i\)-neighbors in \(F|_S\). The bandit \(\mathrm{DS}\) dimension is
\[
\operatorname{BDS}(\mathcal{H})
=
\sup\left\{
\sum_{i=1}^{d'} N_i
:
\exists S\in\mathcal{X}^{d'},\, N\in\mathbb{N}^{d'}
\text{ BDS-shattered by }\mathcal{H}
\right\}.
\]
The relevant aggregate is the total number of neighbors across coordinates, \(\sum_i N_i\), rather than the number of coordinates alone.

ListCascade achieves the paper’s main upper bound:
\[
\mathbf{m}_{\mathcal{H}}^{B}(\epsilon,\delta)
=
O\!\left(
\frac{d_{DS}^{B}(\log K)^3 + K\log K\log(1/\delta)}
{\epsilon}
\right),
\]
where \(d_{DS}^{B}=\operatorname{BDS}(\mathcal{H})\). The matching lower bound is
\[
\mathbf{m}_{\mathcal{H}}^{B}(\epsilon,\delta)
=
\Omega\!\left(
\frac{d_{DS}^{B}+\log(1/\delta)}
{\epsilon}
\right),
\]
for \(\epsilon\in(0,0.1)\), \(\delta\in(0,1)\). Hence
\[
\mathbf{m}_{\mathcal{H}}^{B}(\epsilon,\delta)
\in
\widetilde{\Theta}\!\left(
\frac{\operatorname{BDS}(\mathcal{H})}{\epsilon}
\right).
\]

The per-epoch cost of ListCascade is of the form
\[
\tilde O\!\left(\frac{L\cdot \operatorname{DS}_L(\mathcal{H})}{\epsilon}\right),
\]
and the bridge to the global complexity measure is
\[
\operatorname{BDS}(\mathcal{H})
\ge
\max_{1\le L\le K-1} L\cdot \operatorname{DS}_L(\mathcal{H}).
\]
This is the formal reason the algorithm’s geometric list contraction matches the combinatorial structure of the problem.

## 5. Interpretation and comparison with prior bandit PAC reductions

Bandit feedback is harder than full-information PAC learning because the learner observes only whether its prediction is correct. A naive exploration strategy that guesses uniformly from all \(K\) labels reveals the true label only with probability \(1/K\), yielding a multiplicative \(K\) overhead. Earlier work by Daniely et al. gave a lower bound \(\tilde{\Omega}(\operatorname{ND}(\mathcal{H})/\epsilon)\) and an upper bound \(\tilde{O}(K\operatorname{ND}(\mathcal{H})/\epsilon)\), leaving a multiplicative gap of \(K\) up to logarithmic factors [2605.25678].

ListCascade avoids that global \(K\) factor by never exploring over all labels after the initial stage. Instead, it maintains a current list of plausible labels for each instance and explores uniformly only within that list. In epoch \(t\), the exploration overhead is roughly \(L_{t-1}\), not \(K\), while list learning requires about \(\tilde{O}(\operatorname{DS}_{\lceil L_{t-1}/2\rceil}(\mathcal{H})/\epsilon)\) labeled examples. Summed over logarithmically many epochs, the overall complexity is controlled by the bandit \(\mathrm{DS}\) dimension rather than by \(K\operatorname{ND}(\mathcal{H})\).

The paper presents this as a conceptually clean reduction from bandit PAC learning to list learning and explicitly states that ListCascade “may be of independent interest.” A plausible implication is that the algorithm’s significance is not confined to the proof of the upper bound: it isolates a reusable principle in which sparse bandit feedback is converted into full-information examples through progressively shrinking candidate sets.

## 6. Terminological scope and relation to other cascade literatures

Within the provided literature, “ListCascade” in the strict sense refers to the list-learning reduction introduced for realizable multiclass PAC learning with bandit feedback [2605.25678]. Other recent works use the word “cascade” for distinct architectures and problem classes: online model cascades for stream inference [2402.04513], LLM cascades with early abstention [2502.09054], streaming proxy–oracle cascades for semantic SQL [2604.00660], response-level speculative decoding for API serving [2606.22840], and end-to-end training of multi-stage top-\(k\) ranking systems [2503.09492]. In ranking and bandit theory, “cascade” also denotes sequential examination models for ranked lists [2202.01562], diverse online learning to rank from partial-click feedback [1811.00911], non-stationary cascade ranking [1905.12370], and cost-aware cascading bandits [1805.08638].

These usages are structurally related only at a high level: each involves staged filtering, sequential examination, or deferred computation. ListCascade itself is narrower. It is a specific reduction from bandit multiclass PAC learning to list learning, with geometric list contraction, majority vote on lists, and sample complexity characterized by pseudo-boxes and the bandit \(\mathrm{DS}\) dimension. This suggests that the term is best reserved for the algorithmic principle of [2605.25678], rather than used as a generic synonym for cascade architectures.

Source: https://www.emergentmind.com/topics/listcascade