Contextual Slate GLM Bandits with Limited Adaptivity

Abstract: We investigate the contextual slate bandit problem with generalized linear rewards under limited adaptivity. At each round, the learner is presented with $N$ sets of items, where each item is represented by a $d$-dimensional feature vector. The learner then constructs a slate by selecting one item per set; the resulting slate yields a scalar reward sampled from a Generalized Linear Model (GLM). We propose algorithms under two limited-adaptivity settings: (a) Batched and (b) Rarely-Switching. For the batched setting, we introduce B-SlateGLinCB, which partitions the time horizon into $\mathcal{O}(\log\log T)$ batches such that each batch's policy relies only on data from previous batches. For the rarely-switching setting, we propose RS-SlateGLinCB, which adaptively performs only $\mathcal{O}(Nd\log T)$ parameter updates. Under a diversity assumption on the item sequences, we prove that B-SlateGLinCB and RS-SlateGLinCB achieve regret bounds of $\mathcal{O}(Nd^{3/2}\sqrt{T})$ and $\mathcal{O}(Nd\sqrt{T})$, respectively. Notably, both bounds are independent of the non-linearity parameter $κ$ that is typically found to scale the regret of GLM bandit algorithms. Our algorithms are computationally efficient, requiring only $\text{poly}(N)$ time per round despite $2^{Ω(N)}$ possible slates. Simulations show our algorithms outperform existing baselines with limited adaptivity and remain competitive with Slate-GLM-OFU, a fully adaptive state-of-the-art algorithm. Notably, a slightly modified B-SlateGLinCB empirically matches this baseline. Finally, we demonstrate strong performance in a practical in-context example selection task for LLMs.

• The paper introduces novel batched and rarely-switching algorithms for contextual slate GLM bandits that achieve sublinear regret with constrained adaptivity.
• It leverages slot-level G-optimal design and GLM-MLE based confidence bounds to ensure efficient computation and regret guarantees independent of the nonlinearity parameter.
• Empirical results on synthetic and real-world prompt tuning demonstrate that the methods match fully adaptive algorithms with significantly fewer policy updates.

Contextual Slate GLM Bandits with Limited Adaptivity: A Technical Essay

Problem Overview

The paper "Contextual Slate GLM Bandits with Limited Adaptivity" (2606.31449) undertakes the formalization and algorithmic development of the contextual slate bandit problem with generalized linear reward models (GLMs) under constraints on adaptivity. At each round, the agent selects a slate—an ordered sequence defined by choosing one item per slot from $N$ candidate sets—receiving scalar feedback governed by an unknown GLM. The core technical challenge is to provably and efficiently learn in this exponentially large combinatorial action space while limiting the number and frequency of adaptive policy updates, which is critical for large-scale online systems involving parallelization or where minimizing update cost is necessary.

Limited Adaptivity Model

The work departs from standard bandit and contextual slate bandit settings by considering two practical regimes:

• Batched setting (B-SlateGLinCB): The time horizon is partitioned into $M = O(\log\log T)$ batches. Policy updates and learning are only allowed between batches, greatly reducing adaptivity and facilitating parallelized decisions within batches.
• Rarely-Switching setting (RS-SlateGLinCB): The agent adaptively decides when to update the model, but policy changes are restricted to $O(Nd\log T)$ updates, a polylogarithmic number in the time horizon.

In both cases, the feedback is bandit-level (the total reward of the slate) and the reward mapping is a GLM, subsuming logistic, Poisson, and others, controlled by monotonic, $L_\mu$-Lipschitz link functions.

Algorithmic Design and Regret Guarantees

The paper’s main technical claims are the proposals of B-SlateGLinCB and RS-SlateGLinCB, both of which are polynomial in $N$ per round and avoid explicit enumeration of $2^{\Omega(N)}$ possible slates. Policy selection leverages confidence bounds based on GLM-MLE and slot-level design matrices, as well as elimination techniques and optimal design strategies.

B-SlateGLinCB (Batched)

Each batch proceeds via a warm-up phase with slot-level G-optimal design sampling, followed by elimination rounds where items are pruned based on confidence intervals computed from prior estimated parameters. Items are retained if their upper confidence bound (UCB) exceeds the maximal lower confidence bound (LCB) in the candidate set, computed recursively across earlier batches. After each batch, updated GLM-MLE parameters and scaled design matrices are computed. The normalization/scaling procedures ensure that confidence sets for GLM parameters are tight without explicit dependence on the instance nonlinearity parameter $\kappa$.

Theoretical guarantee: Under a diversity assumption on slot sequence features, the expected regret after $T$ rounds is

$R(T) = \tilde{O} \left(N d^{3/2} \sqrt{T} \right)$

entirely independent of $\kappa$—a parameter that in much prior work induces exponential regret dependence—thereby establishing a strong theoretical improvement. See Section 4 for details and proof structure, as well as direct comparison with prior art.

RS-SlateGLinCB (Rarely-Switching)

In this regime, the algorithm maintains slot-level design matrices and only updates the GLM-MLE parameters when the determinant of an aggregated design matrix increases by a constant factor, amortizing updates over episodes of progressively informative data. Proposed elimination and selection processes are similar to B-SlateGLinCB, but updates and corresponding designs are strictly limited as per determinant conditions.

Theoretical guarantee: Regret of

$M = O(\log\log T)$0

with only $M = O(\log\log T)$1 policy updates—matching the regret of fully adaptive methods such as Slate-GLM-OFU but with drastically reduced adaptivity.

Additionally, both methods guarantee that computation per round only scales polynomially in $M = O(\log\log T)$2 and $M = O(\log\log T)$3.

Empirical Results

Synthetic Settings

Extensive synthetic experiments are reported for two representative setups ($M = O(\log\log T)$4 and $M = O(\log\log T)$5), demonstrating that both the batched and rarely-switching algorithms yield strongly sublinear regret in $M = O(\log\log T)$6, outperform all baseline limited-adaptivity algorithms (including RS-GLinCB, RS-MNL, and SoftBatch), and are highly competitive with the unconstrained fully adaptive algorithm, Slate-GLM-OFU.

Figure 1: Batch-mode experiment showing cumulative regret for B-SlateGLinCB and RS-SlateGLinCB versus several baselines, highlighting the empirical improvement from the theoretical guarantees.

The gap between the regret of RS-SlateGLinCB and fully adaptive Slate-GLM-OFU is negligible, and a slightly modified batched algorithm (B-SlateGLinCB+) empirically closes the gap completely.

Real-World: Prompt Tuning

The algorithms are applied to exemplar selection for prompt tuning in LLMs, specifically on a binary classification (SST-2 sentiment) task with RoBERTa-large and Nomic-Embed-Text-v1.5. The slate bandit agent adaptively selects $M = O(\log\log T)$7 exemplars from candidate pools per slot for thousands of queries, seeking to maximize classification accuracy with only $M = O(\log\log T)$8 parameter updates. Compared to both a zero-shot and random-exemplar baseline, as well as the unconstrained adaptive method, B-SlateGLinCB+ rapidly achieves near-identical average accuracy to the state-of-the-art, demonstrating its practical value for in-context RL over long horizons.

Figure 2: Exemplar selection for prompt tuning: Average cumulative accuracy for B-SlateGLinCB+ matches fully adaptive benchmarks with limited adaptivity.

Structural and Analytical Advances

A technical strength of this work is replacing prior exponential $M = O(\log\log T)$9 dependence with diversity-based assumptions on slot feature distributions, enabling polylogarithmic adaptive schedules. The use of slot-level G-optimal/Distributional-Optimal designs and appropriate (block-diagonal) scaled design matrices is non-trivial in the slate setting and forms a crucial analytical innovation. The framework directly subsumes and strengthens prior advances for contextual combinatorial/GLM/MNL bandits under adaptivity constraints.

It is also shown (theoretically and via experiment) that, despite adaptation restrictions, the algorithms empirically retain optimal behavior across large-scale combinatorial slate problems, including when $O(Nd\log T)$0 and $O(Nd\log T)$1 (items per slot) are large.

Implications and Future Directions

Practically, these results indicate it is possible to achieve nearly minimal regret and strong empirical performance for GLM-parameterized slate bandits relevant to ad selection, website optimization, and in-context learning problems, with adaptivity constraints necessary for parallel or deployment environments. The capacity to realize per-round efficiency at scale and without dependence on $O(Nd\log T)$2 removes key barriers to real-world application.

Theoretically, the work fully bridges gaps between adversarial robustness, bandit combinatorics, and nonlinear reward learning under adaptivity constraints. Open problems include extending these results to semi-bandit or non-monotonic GLMs, deriving regret lower bounds that are tight in all parameters, and precisely characterizing how other exploration heuristics or hybrid elimination strategies affect regret and implementation overhead. The empirical success of B-SlateGLinCB+ motivates further formal study of batch elimination heuristics.

Conclusion

This paper delivers rigorous algorithms for contextual slate GLM bandits under stringent adaptivity constraints, achieving regret bounds that both improve on or match fully adaptive algorithms, but with a fraction of the parameter updates. The results are validated both mathematically—by eliminating $O(Nd\log T)$3 dependence and operating in polynomial time for combinatorial slates—and empirically, by demonstrating performance competitive with state-of-the-art fully adaptive methods on large real and synthetic problems. The framework lays substantive groundwork for bandit optimization under operational constraints ubiquitous in real-world large-scale AI deployments.