Instance-Adaptive Regret Bounds

• Instance-adaptive regret bounds are performance metrics that adapt to observed losses and constraints, offering sharper guarantees than worst-case bounds.
• They employ techniques like log-barrier mirror descent and safe decision spaces to optimize decisions under hard constraints in online settings.
• The bounds decompose into bandit and safety complexities, quantifying both the statistical difficulty of the loss sequence and the cost of constraint enforcement.

Instance-adaptive regret bounds, also referred to as instance-dependent, data-dependent, or problem-dependent regret guarantees, represent a leading paradigm in online learning and sequential decision-making. These bounds precisely quantify algorithmic performance as a function of the observed (or adversarially determined) data sequence, the comparator or optimal decision for the instance, and any additional structure such as constraints, loss geometry, or instance hardness. This stands in contrast to classical uniform (worst-case) guarantees that upper bound regret solely in terms of the length of the horizon or ambient problem dimension. The increasingly centrality of instance-adaptive analysis is motivated by its capacity to explain and exploit observed performance improvements in real-world, high-dimensional, large-scale, or nonstationary environments across machine learning, control, and game theory.

1. Formalization and Motivation

Instance-adaptive (or data-dependent) regret bounds guarantee that the performance gap between the learner and the best comparator is not only sublinear in worst-case settings but much tighter when measured on the sequence of observed data. The canonical form is

$$R_T(\ell_{1:T}) = \sum_{t=1}^T \ell_t(x_t) - \sum_{t=1}^T \ell_t(x^*) \leq \widetilde{\mathcal{O}}(\sqrt{\sum_{t=1}^T \ell_t^\top x^*}),$$

where $x^*$ is the optimal static or dynamic decision subject to constraints, and $\ell_t$ is the loss vector observed at round $t$. The key property is that, unlike $\widetilde{\mathcal{O}}(\sqrt{T})$ guarantees, this bound automatically improves for "easy" instances—where the optimal loss is much less than $T$—and is never worse in the hardest instances.

This approach is essential for aligning theoretical guarantees with empirical outcomes, especially in scenarios with benign losses, rare constraint activations, highly structured features, or in the presence of safety or fairness requirements.

2. Algorithmic Design in Constrained MABs

The instance-adaptive regret analysis in the context of constrained multi-armed bandits (MABs) with adversarial losses and stochastic constraints hinges on several algorithmic innovations (Genalti et al., 26 May 2025):

• Online Mirror Descent with Log-Barrier Regularization: The learner uses mirror descent updates with a log-barrier regularizer to maintain the stochastic decision vector in a safely truncated simplex. This facilitates fine-grained adaptivity to cumulative loss while inherently maintaining positivity of arm probabilities.
• Safe Decision Spaces and Confidence-Guided Feasibility: To handle constraints—especially hard constraints that must be satisfied at every round—a safe set is constructed using empirical constraint cost estimates, combined with confidence intervals accounting for sampling uncertainty. The feasible set is defined as

$$\mathcal{S}_t = \left\{ x \in \Delta_K: \forall i\,\, \hat{g}_{t,i}^\top x \leq \alpha_i - \beta_t \right\},$$

where $\hat{g}_{t,i}$ is the average constraint cost estimator, and $\beta_t$ is a confidence width.

• Convex Combination with Strictly Feasible Solutions: For hard-constraint settings, feasibility may not be guaranteed by the mirror descent step alone. A convex combination with a known strictly feasible solution $x^\diamond$ is computed, according to a data-dependent mixture coefficient. This ensures constraints are satisfied with high probability at each round.
• Adaptive Learning Rate Scaling: Learning rates are dynamically increased when arm probabilities approach boundaries, a technique critical for attaining data-dependent regret similar to unconstrained, small-loss settings.

3. Structure of the Instance-Dependent Regret Bound

The primary data-dependent regret bound for constrained MABs decomposes as follows: $$R_T(\ell_{1:T}) \leq \widetilde{\mathcal{O}}\left( \frac{K}{\rho} \sqrt{\sum_{t=1}^T (\ell_t^\top (x^\diamond - x^*))^2} + \frac{1}{\rho} \sqrt{K \sum_{t=1}^T \ell_t^\top x^* \log(1/\delta)} \right),$$ where:

• $x^*$ is the optimal feasible solution (the "benchmark").
• $x^\diamond$ is any strictly feasible strategy satisfying $g_i^\top x^\diamond < \alpha_i - \rho$ for all $i$, with margin $\rho > 0$.
• The bandit complexity term, $$\widetilde{\mathcal{O}}((1/\rho)\sqrt{K \sum_{t=1}^T \ell_t^\top x^*})$$, captures the true instance difficulty and is tight when $L^* = \sum_{t} \ell_t^\top x^*$ is small.
• The safety complexity term, $$\widetilde{\mathcal{O}}((K/\rho)\sqrt{\sum_t (\ell_t^\top(x^\diamond-x^*))^2})$$, quantifies the additional regret paid due to constraint satisfaction, scaling with the squared cost of "moving away" from $x^*$ to $x^\diamond$.

These two terms are individually necessary: the lower bounds constructed in (Genalti et al., 26 May 2025) (via KL-divergence-based instance construction) match both terms up to constants, showing that neither is an artifact of analysis.

4. Theoretical Foundations and Lower Bounds

The theoretical foundations rest on supermartingale concentration for cost and constraint estimates, together with instance-specific "small-loss ball" constructions. For any family of loss sequences with (i) average optimal loss $1/T\sum_t \ell_t^\top x^* \leq \omega$ and (ii) constraint-induced squared deviation no more than $\Delta^2$,

$$\mathcal{B}_{\omega,\Delta,T} = \bigg\{ \ell_{1:T}: \frac{1}{T} \sum_t \ell_t^\top x^* \leq \omega,\,\, \frac{1}{T} \sum_t (\ell_t^\top(x^\diamond-x^*))^2 \leq \Delta^2 \bigg\},$$

any randomized algorithm incurs expected regret at least $\Omega(\sqrt{\omega T} + (\Delta/\rho)\sqrt{T})$ on some member of this class. This matches the upper bounds up to logarithmic terms.

Thus, the bandit complexity and safety complexity quantify the statistically unavoidable limits of instance-adaptive performance in constrained online learning.

5. Comparison with the Classical Literature

In classical unconstrained adversarial bandits, algorithms obtain regret at rate $\widetilde{\mathcal{O}}(\sqrt{K T})$ (or instance adaptivity via small-loss bounds $\widetilde{\mathcal{O}}(\sqrt{K L^*})$). Constraint handling in the adversarial regime traditionally paid at least a $\sqrt{T}$ penalty even for easy loss instances, as feasibility demands could dominate learning dynamics.

The approach and results of (Genalti et al., 26 May 2025) unify these two regimes:

• If $x^\diamond$ is as good as $x^*$ (i.e., safe and optimal coincide), the safety complexity vanishes and full instance-adaptivity is possible.
• When the safety margin $\rho$ is small, or $x^\diamond$ is costly, the unavoidable safety complexity is separately revealed in the bound.

This explicit decomposition into bandit and safety complexities—provably tight for both hard and soft constraint variants—represents a conceptual advance relative to prior constrained learning theory.

6. Algorithmic Implications, Extensions, and Applications

The proposed techniques—combining log-barrier mirror descent, safety-truncated domains, adaptive regularization, and convex combinations—enable instance-adaptive performance guarantees in practically relevant settings requiring strict or soft safety (resource, budget, risk, or regulatory) constraints. The algorithmic structure is modular and may be extended:

• To settings with more general convex (or time-varying) constraints,
• To contextual bandits with side-information,
• To online combinatorial optimization,
• Or as a meta-algorithm for controlling constraint violation in multi-objective optimization and real-time systems.

Moreover, the log-barrier regularization and safety-aware projection techniques bridge recent advances in unconstrained small-loss and adaptive regret analysis with the domain of safe exploration and online decision-making under hard constraints.

7. Future Directions and Open Problems

Several directions are evident:

• Tightening the logarithmic factors in both upper and lower bounds for various regimes.
• Extension to more complex or non-linear constraints or to bandit learning with resource allocation, scheduling, or queueing constraints.
• High-probability, rather than expectation-based, instance-adaptive regret bounds for constrained problems.
• Combining the data-dependent regret framework with path-length or variation-based metrics to further refine adaptivity in dynamically evolving environments.

A plausible implication is that similar instance-adaptive analyses could be performed for a broader class of constrained online learning frameworks, including those under stochastic nonstationarity, partial feedback, or multitask architectures.

---

In summary, instance-adaptive (data-dependent) regret bounds in constrained MABs provide two-component performance guarantees that reflect both the statistical learning complexity of the observed loss sequence and the intrinsic difficulty of enforcing hard constraints. The log-barrier OMD strategies, safe space construction, and novel lower bounds in (Genalti et al., 26 May 2025) establish these terms as unavoidable and sharp in adversarial regimes, marking a significant advance in the theory and practice of safe online learning.