PAC Battling Bandits: Plackett-Luce Model

• The paper introduces the PAC Battling Bandits framework under the Plackett–Luce model, establishing optimal sample complexity bounds for both Winner Information and Top-m Ranking feedback.
• It exploits the IIA property to convert rank-ordered outcomes into consistent pairwise estimates, enabling robust elimination and adaptive selection strategies.
• The study shows that richer feedback (top-m rankings) leads to a proportional m-fold improvement in learning efficiency, offering practical benefits for online decision-making.

The Probably Approximately Correct (PAC) Battling Bandits framework under the Plackett–Luce (PL) model addresses subsetwise preference learning in online settings. At each round, a learner chooses a subset of $k$ arms out of $n$, and receives stochastic feedback indicating information about the relative quality of those arms, for example, the winner or a partial ranking. The objective is to efficiently identify an arm whose performance is close to optimal with high confidence, according to $\epsilon,\delta$–PAC criteria. This model generalizes the classic dueling bandits scenario ($k=2$ sets), and exploits the information structure of the PL model, which connects subsetwise observations to underlying latent scores. The principal research advances in this domain characterize the sample complexity limits under different feedback regimes and design algorithms that optimally leverage the feedback granularity.

1. The Plackett–Luce Model and Subsetwise Preference Learning

The PL model parameterizes each arm $i \in [n]$ by a positive score $\theta_i>0$. For any non-empty subset $S \subseteq [n]$, the probability that $i$ is the winner (top-ranked) in $S$ is given by

$$\Pr(i \mid S) = \frac{\theta_i}{\sum_{j \in S} \theta_j}.$$

A full ranking $n$0 of $n$1 is sampled recursively: at each position $n$2, select an item from the remaining pool with probability proportional to its $n$3, remove it, and repeat. The PL model uniquely satisfies the independence of irrelevant alternatives (IIA), so for any $n$4, the odds ratio $n$5 equals $n$6. This property underpins the consistent extraction of pairwise preferences from subsetwise and rank-ordered feedback.

2. PAC Identification Objective and Feedback Models

A sequential algorithm aims to identify an $n$7-optimal arm, typically assumed without loss of generality to be arm 1, i.e., $n$8. An arm $n$9 is $\epsilon,\delta$0–optimal if the pairwise win probability

$\epsilon,\delta$1

An algorithm is $\epsilon,\delta$2–PAC if it stops in finite time and returns an $\epsilon,\delta$3–optimal arm with probability at least $\epsilon,\delta$4. The learning protocol is defined by the choice of subsets of size $\epsilon,\delta$5 and two main types of feedback per subset:

• Winner Information (WI): Observe only the winner in the subset.
• Top-$\epsilon,\delta$6 Ranking (TR): Observe the first $\epsilon,\delta$7 items in the full PL ranking of the subset, with $\epsilon,\delta$8.

As special cases, $\epsilon,\delta$9 reduces to WI, and $k=2$0 corresponds to full ranking feedback.

3. Information-Theoretic Lower Bounds

The fundamental limits of PAC identification mirror the amount of information provided per round. Under WI, no matter the subset size $k=2$1, the minimax sample complexity lower bound is

$k=2$2

for some PL instance. This shows that increasing the batch size under WI provides no asymptotic gain over the classic dueling bandit ($k=2$3) setting. The proof proceeds by constructing adversarially hard PL instances and applying a change-of-measure argument based on KL-divergence, showing that distinguishing between nearly optimal arms requires this many samples (Saha et al., 2018).

With TR feedback of length $k=2$4, the information per round increases proportionally, yielding a lower bound

$k=2$5

and a corresponding $k=2$6 multiplicative advantage. When $k=2$7, this reaches

$k=2$8

Thus, richer feedback can dramatically accelerate learning by packing more pairwise comparison information in each round.

4. Algorithmic Frameworks and Upper Bounds

Several PAC identification algorithms match the above lower bounds, both in the worst case and adaptively to instance hardness:

• Trace-the-Best initializes with a reference arm, partitions the rest into $k=2$9-sized batches, battles each batch against the reference arm for $i \in [n]$0 rounds using WI, and updates the reference if a batch contains an empirically superior arm. The sample complexity is

$i \in [n]$1

for WI (Saha et al., 2018).

• Divide-and-Conquer repeatedly partitions the survivor set into $i \in [n]$2-sized groups, performs in-group battles, and retains empirical batch winners, iteratively shrinking the survivor set with progressively smaller tolerances. This also achieves the $i \in [n]$3 bound for WI (Saha et al., 2018).
• Halve-the-Pool allows arbitrary subset sizes up to $i \in [n]$4 and improves constants/log factors by median elimination.

For TR feedback ($i \in [n]$5), these same structural ideas generalize: each rank-ordered output is "rank-broken" into $i \in [n]$6 pairwise tallies per round, and the confidence intervals scale as $i \in [n]$7. Both Trace-the-Best and Divide-and-Conquer achieve

$i \in [n]$8

for length-$i \in [n]$9 TR feedback (Saha et al., 2018).

5. Instance-Optimality and Adaptive Sample Complexity

Subsequent advances show that sample complexity can adapt to the actual hardness of the PL instance, encoded via arm-specific gaps $\theta_i>0$0. The PAC-Wrapper framework executes a sequence of phases, each phase finding an approximately best item among current survivors using a PAC subroutine, and eliminating arms that appear statistically worse. For winner-only feedback, the instance-optimal sample complexity is

$\theta_i>0$1

where $\theta_i>0$2 is the sum of the $\theta_i>0$3 largest PL scores (Saha et al., 2019). Under TR feedback, this is improved by $\theta_i>0$4: $\theta_i>0$5 Lower bounds confirm the optimality of these rates. This approach ensures that when all gaps $\theta_i>0$6 are large, the algorithm stops much earlier than in worst-case settings, but never exceeds the minimax guarantee.

6. Fixed-Budget Identification and Error Exponents

In the fixed-budget setting, where the learner is given a total play budget $\theta_i>0$7, the fundamental objective becomes minimizing the misidentification probability $\theta_i>0$8. The lower bound on achievable $\theta_i>0$9 for any symmetric algorithm is

$S \subseteq [n]$0

where

$S \subseteq [n]$1

Uniform-Allocation algorithms that allocate batches evenly and iterate elimination based on empirical wins attain (up to logarithmic factors) the optimal exponential decay in $S \subseteq [n]$2, with error controlled as

$S \subseteq [n]$3

and sufficient budget

$S \subseteq [n]$4

(Saha et al., 2019).

7. Practical Implications and Further Observations

Allowing winner-only feedback from larger subsets ($S \subseteq [n]$5) does not improve the sample complexity over the dueling bandit setting; thus, subset size only matters under richer feedback regimes. In contrast, collecting top-$S \subseteq [n]$6 rankings yields an $S \subseteq [n]$7-fold speedup, and the efficiency gain is optimal in both worst- and instance-dependent settings. Rank-breaking exploits the IIA structure of PL, converting rank information into consistent pairwise empirical estimates and enabling sharper confidence bounds. Recent numerical experiments confirm that instance-optimal algorithms can adaptively stop much earlier on "easy" PL instances and scale gracefully with problem difficulty and feedback granularity (Saha et al., 2018, Saha et al., 2019).