Time-Constrained Slate Recommendation

• Time-constrained slate recommendation is a framework that models the trade-off between item relevance and evaluation cost under explicit user time constraints.
• It formalizes the task as a budget-aware MDP and employs reinforcement learning alongside contextual slate bandit methods to optimize user engagement.
• Empirical studies show that RL methods can improve play rate and effective slate size compared to traditional bandit approaches in resource-constrained environments.

Time-constrained slate recommendation is the study and development of algorithms for generating ordered sets (or “slates”) of items to recommend to users subject to explicit limits on the user's time or attention budget. Unlike classical slate or listwise recommendation, the time-constrained variant explicitly models the trade-off between utility (e.g., relevance, engagement) and per-item evaluation costs (e.g., seconds spent inspecting an item), reflecting the finite time available for user interaction. This problem setting appears in user-facing systems such as mobile e-commerce platforms, where scroll-based recommendation slates must account for both the likelihood of user engagement and the evaluation overhead of each presented item (Chakrabarty et al., 13 Dec 2025).

1. Formal Problem Definition

The time-constrained slate recommendation task is modeled as a finite-horizon Markov Decision Process (MDP) with a resource budget. A state at timestep $t$ is $s_t=(u_t, q_t)$, where $u_t \geq 0$ denotes remaining user time budget, and $q_t$ is the prefix of items already examined or recommended in the current slate. The action $a_t$ is the next slate of $K$ items to recommend, $a_t \in I^K$, where $I$ is the universe of items.

Transition dynamics account for examination probabilities and cost consumption: let $\sigma_i$ be the predicted relevance of item $i$, and $\beta_k(a_t)$ the probability that slot $k$ is examined. A single-click per slate model is assumed: at most one item is clicked per slate, and the evaluation cost $c_{i^k_t}$ for item $i^k_t$ in slot $k$ is deducted from the budget if clicked. If no click occurs, cost is zero. The next state $s_{t+1}$ depends on the consumed cost and budget update $u_{t+1} = u_t - c_{i^k_t} 1_{k>0}$, with the slate prefix reset.

The reward combines the indicator of a click and a penalty for time cost: $$r(s_t, a_t, s_{t+1}) = 1_{e_t>0} - \lambda c_{i^{e_t}_t}$$ or, using click/examination probabilities, $$r(s_t,a_t) = \sum_{k=1}^K \beta_k(a_t)(1 - \lambda c_{i^k_t})$$ for a scalar penalty parameter $\lambda > 0$. The total cost across an episode is constrained to not exceed user budget $B$: $\sum_t c(a_t, e_t) \leq B$.

The policy optimization objective is thus: $$\max_\pi \; \mathbb{E}_\pi \left[ \sum_{t=0}^{T} \gamma^t \left( r(s_{t},a_{t}) - \lambda c(a_{t},e_{t}) \right)\right] \quad \text{s.t.} \quad \sum_{t=0}^{T} c(a_{t},e_{t}) \leq B$$ with $\gamma$ a discount factor, and where the Lagrangian relaxation facilitates policy gradient or value-based approaches (Chakrabarty et al., 13 Dec 2025).

2. Algorithmic Approaches

Value-based RL Methods

The budget-aware MDP structure supports both on-policy and off-policy reinforcement learning controllers. SARSA (on-policy) and Q-learning (off-policy) are applied, where Q-value estimates condition on remaining budget and partial slate context. Updates follow standard rules:

• SARSA:

$$Q(s_t, a_t) \leftarrow Q(s_t, a_t) + \alpha \left[ r_t + \gamma Q(s_{t+1}, a_{t+1}) - Q(s_t, a_t) \right]$$

• Q-learning:

$$Q(s_t, a_t) \leftarrow Q(s_t, a_t) + \alpha \left[ r_t + \gamma \max_{a'} Q(s_{t+1}, a') - Q(s_t, a_t) \right]$$

Q-values are function-approximated using XGBoost regression trees over state-action pairs, and slates are constructed slot-by-slot to avoid exponential blow-up in action space.

Contextual Slate Bandit Methods

In related slate bandit settings without explicit budgets but with combinatorial slate selection, algorithms must address the exponential size of the slate space. The Logistic Contextual Slate Bandit framework introduces models where, at every round, an agent chooses a slate (ordered tuple) of $N$ items and receives a binary outcome (reward) generated by a logistic model $\mu(\cdot)$ over candidate feature vectors (Goyal et al., 16 Jun 2025).

• Slate-GLM-OFU (Optimism in the Face of Uncertainty) and Slate-GLM-TS (Thompson Sampling) split decision-making into slot-wise (local) item optimization and global parameter (joint) estimation, exploiting per-slot diversity to obtain computational tractability and near-optimal regret.
• Diversity assumptions requiring slot-specific feature independence underpin the theoretical analysis showing that slot-local exploration bonuses suffice to match global optimality up to lower-order terms.

Approach	Budget-aware?	Reward Model	Action Selection	Complexity	Regret Bound
SARSA/Q-learning (Chakrabarty et al., 13 Dec 2025)	Yes	MDP (with cost)	Q-value, slot-by-slot	$\mathcal{O}(|I| \cdot K)$	Empirical (Play rate, etc.)
Slate-GLM-OFU (Goyal et al., 16 Jun 2025)	No	Logistic Bandit	Slot-wise optimism	$N^{O(1)} \operatorname{poly}(\log T)$	$\widetilde O(dN\sqrt{T})$
Slate-GLM-TS (Goyal et al., 16 Jun 2025)	No	Logistic Bandit	Slot-wise TS	$N^{O(1)} \operatorname{poly}(\log T)$	$O(d^{3/2}N^{3/2}\sqrt{T})$

3. Simulation and Empirical Evaluation

Simulation uses Alibaba’s Personalized Re-ranking dataset, which contains 150 users and approximately 143,000 unique items, with slates of size $K=30$. Per-item evaluation costs are drawn uniformly from $[0, 100]$ seconds, and initial user budgets $u_0$ are log-normally distributed ($100 \leq u_0 \leq 500$).

User choice within the slate is simulated by transforming predicted relevance scores into examination probabilities, sampling the click/no-click event, and updating the budget and observed reward accordingly. Items surpassing the remaining budget are dynamically excluded. Learning uses experience replay and $\epsilon$-greedy action selection ($\epsilon=0.1$).

Play Rate (average clicks per slate) and Effective Slate Size (number of items examined before budget exhaustion) are the primary metrics. RL-based controllers are compared to a contextual bandit baseline ($\gamma \approx 0$) under variable budgets:

• For low budgets ($u_0 \leq 200$s), RL methods increase play rate by up to 8% and effective slate size by up to 15% over the bandit approach.
• Off-policy Q-learning dominates in play rate; SARSA matches effective slate size on larger budgets as episode length grows, reducing variance.
• RL methods also reduce premature abandonment relative to purely myopic reranking (Chakrabarty et al., 13 Dec 2025).

Empirical validation of contextual slate bandit methods on synthetic and real tasks (prompt tuning for LLMs) demonstrates both computational efficiency (sub-millisecond per round) and strong regret minimization, supporting practical application in large-scale deployments (Goyal et al., 16 Jun 2025).

4. Theoretical Guarantees and Computational Complexity

For budget-aware MDPs, no explicit regret bounds are reported; evaluation is empirical. In contextual slate bandits, slot-wise diversity ensures that cumulative regret is $\widetilde O(dN\sqrt{T})$ for Slate-GLM-OFU, and $O(d^{3/2}N^{3/2}\sqrt{T})$ for Slate-GLM-TS under non-contextual fixed-arm conditions (Goyal et al., 16 Jun 2025). Both achieve polynomial (rather than exponential) runtime in the number of slate slots $N$, as action selection is decomposed across slots and parameter updates are globally coordinated. This factorization is justified by a “multiplicative equivalence” between slot-local and global second-moment (Fisher) matrices under sufficient diversity.

5. Trade-Offs, Limitations, and Extensions

Time-constrained slate recommendation fundamentally involves a relevance–cost trade-off: as $\lambda$ (the cost penalty) increases, optimal policies shift focus from high-relevance but costly items toward lower-cost recommendations. For smaller budgets, optimal policies prioritize items with a high utility-to-cost ratio; for larger budgets, policies can afford to include items of higher relevance despite higher cost.

Limitations identified include:

• Single-click reward model, while actual users may click multiple items or perform graded feedback (e.g., purchases).
• Cost prior is simplified; real systems must consider item/position-dependent cost and model uncertainty or context-dependence in evaluation time.
• Budgets are sampled exogenously; in most applications, user attention budgets are latent and must be inferred jointly with preferences, motivating partially observable MDP (POMDP) generalizations (Chakrabarty et al., 13 Dec 2025).

Future work directions include:

• Estimating feature-conditional, slot- and position-dependent costs.
• Applying policy-gradient and actor-critic methods (e.g., PPO) for direct constrained optimization.
• Extending to multi-click, multi-objective rewards and to models where user time budgets are not observed but inferred from behavior (Chakrabarty et al., 13 Dec 2025).

6. Broader Context and Practical Impact

Practical instantiations of time-constrained slate recommendation span interactive recommender systems, mobile commerce, and content curation platforms. The confluence of MDP modeling, cost-sensitive optimization, and scalable combinatorial bandit algorithms offers a principled approach to balancing exploitation of learned preferences and cognizance of user resource constraints. Contextual bandit formulations and slot-wise tractable factorization support deployment at web-scale with minimal computational overhead (Goyal et al., 16 Jun 2025).

In summary, time-constrained slate recommendation provides a rigorous foundation for next-generation engagement optimization, combining RL, constrained optimization, and contextual bandit advances to reflect realistic user interaction budgets in ordered recommendation tasks (Chakrabarty et al., 13 Dec 2025, Goyal et al., 16 Jun 2025).