Simple Projection-Free Algorithm for Contextual Recommendation with Logarithmic Regret and Robustness

Abstract: Contextual recommendation is a variant of contextual linear bandits in which the learner observes an (optimal) action rather than a reward scalar. Recently, Sakaue et al. (2025) developed an efficient Online Newton Step (ONS) approach with an $O(d\log T)$ regret bound, where $d$ is the dimension of the action space and $T$ is the time horizon. In this paper, we present a simple algorithm that is more efficient than the ONS-based method while achieving the same regret guarantee. Our core idea is to exploit the improperness inherent in contextual recommendation, leading to an update rule akin to the second-order perceptron from online classification. This removes the Mahalanobis projection step required by ONS, which is often a major computational bottleneck. More importantly, the same algorithm remains robust to possibly suboptimal action feedback, whereas the prior ONS-based method required running multiple ONS learners with different learning rates for this extension. We describe how our method works in general Hilbert spaces (e.g., via kernelization), where eliminating Mahalanobis projections becomes even more beneficial.

• The paper introduces CoRectron, a projection-free algorithm that attains O(d log T) regret while circumventing expensive Mahalanobis projections.
• It leverages scale invariance to eliminate norm constraints by using a second-order update reminiscent of the Perceptron, ensuring computational efficiency.
• The algorithm shows robust performance under suboptimal action feedback and extends to nonparametric kernelized settings with proven logarithmic regret bounds.

Simple Projection-Free Algorithm for Contextual Recommendation with Logarithmic Regret and Robustness

Problem Formulation and Motivation

The contextual recommendation problem abstracts action-only feedback regimes encountered in inverse optimization, inverse reinforcement learning, and revealed preference learning. In this setting, at each round $t$ the learner observes a context-encoded feasible action set $X_t$, predicts a utility vector $\hat{w}_t$ (optionally in a Hilbert space $V$), outputs $$\hat{x}_t = \arg\max_{x \in X_t} \langle\hat{w}_t, x\rangle$$, and then receives as feedback a user/chosen action $x_t \in X_t$ that is optimal (or suboptimal) for a fixed, unknown preference $u \in V$. The learner's performance is measured by cumulative regret,

$$R_T(u) = \sum_{t=1}^T \langle u, x_t - \hat{x}_t \rangle~,$$

which captures the aggregate utility shortfall of the learner’s recommendations versus the true user preference.

Prior approaches based on Online Newton Step (ONS) algorithms achieve the state-of-the-art $O(d \log T)$ regret, but such methods require expensive Mahalanobis projections at each round, which dominate the computational cost in high dimensional or nonparametric (kernelized) settings. This paper's main contribution is a projection-free algorithm—CoRectron—that matches the logarithmic regret guarantees of ONS-based strategies but with strictly lower computational overhead and enhanced robustness to suboptimal action feedback.

Core Algorithm: CoRectron

The key technical innovation is recognizing and leveraging the improperness induced by scale invariance in the actual contextual recommendation objective. Since actions are recommended as maximizers over $\langle \hat{w}_t, \cdot\rangle$ and the regret is evaluated on actions rather than the actual utility-vector predictions, the scale of $\hat{w}_t$ is irrelevant—only its direction matters. Therefore, unlike classical OCO-based designs, no norm constraint or projection onto a ball is required for the prediction vectors.

CoRectron builds on this insight. It maintains the cumulative residual $\zeta_t = \sum_{s=1}^t g_s$, where $g_t = \hat{x}_t - x_t$, and a second-order preconditioner $A_t = \lambda I + \sum_{s=1}^t g_s \otimes g_s$. At each round, it computes

$\hat{w}_t = -A_{t-1}^{-1} \zeta_{t-1}~,$

selects $\hat{x}_t$, observes feedback $x_t$, updates $g_t,A_t,\zeta_t$, and proceeds. Notably, no Mahalanobis projection is needed during the update. This mechanism is strongly reminiscent of the second-order Perceptron from online classification, but the analysis in this setting yields strictly data-dependent and scale-free bounds.

Theoretical Guarantees

The analysis employs a novel use of a sign condition—arising from the optimality of $\hat{x}_t$ under $\hat{w}_t$—to control the growth of a log-determinant (elliptical) potential of cumulative residuals. The main regret guarantee is:

• For any $u \in V$,

$$R_T(u) \leq \|u\|_{A_T} \sqrt{\log\det(I_T + \lambda^{-1} K_T)}$$

where $K_T$ is the Gram matrix of residuals. Under standard boundedness assumptions and optimal-action feedback, this yields

$R_T(u) = O\left(d \log T\right)$

when $V = \mathbb{R}^d$.

• If the observed $x_t$ can be suboptimal with respect to $u$ (i.e., action feedback exhibits cumulative suboptimality $\Delta_T(u)$), the regret bound degrades smoothly, with

$$R_T(u) = O\left(d \log T + \sqrt{d \log T\, \Delta_T(u)}\,\right)~.$$

This level of robustness is significant: prior efficient ONS-based methods needed a computationally expensive MetaGrad-type ensemble—running $O(\log T)$ adaptive ONS learners—to attain any form of suboptimality-robust regret.

The analysis is flexible, encompassing actions selected in general Hilbert spaces, e.g., the kernelized contextual recommendation model. This extends existing results, which primarily focus on finite-dimensional linear settings, to encompass nonparametric function spaces.

Computational Efficiency

Unlike ONS-based algorithms, which require $O(d^\omega)$ arithmetic (with current $\omega \simeq 2.3714$) for Mahalanobis projections at each step—or $O(t^3)$ in the kernelized variant—CoRectron’s per-iteration complexity is $O(d^2)$ in finite dimensions (dominated by rank-one updates) and $O(t^2 + t\,\tau_{Kv})$ in kernelized settings (with an $n \times n$ action base and kernel-vector product cost $\tau_{Kv}$). This leads to substantial empirical runtime improvements, especially in large-scale or kernelized scenarios. The provided empirical results indicate that CoRectron achieves lower cumulative regret and runtime than ONS-type algorithms, maintaining strong performance across a broad range of hyperparameter choices.

Implications and Directions

Practical Implications:

The algorithm’s capacity to avoid projections without sacrificing regret rate is a nontrivial computational benefit. In high-dimensional and nonparametric contextual bandits/recommendation systems where linear optimization over the feasible action set is efficient but projections are not (e.g., dashboards, resource allocation, portfolio selection), CoRectron's projection-freeness allows real-time operation and scaling to larger function classes (including RKHSes) than previously feasible.

Theoretical Implications:

This work reveals that improperness (arbitrary predictor scale) can be structurally exploited to sidestep bottlenecks in log-regret contextual recommendation. The analytic technique—using the sign structure of residuals in potential-based regret bounds—is applicable to other improper-learning settings and more broadly to online decision problems with scale invariance. The results sharpen our understanding of the role that Perceptron-like updates can play beyond online classification, with the derived logarithmic-regret guarantees holding in substantially general models.

Future Directions:

Avenues for further research include (i) approximation strategies or sketching for kernelized variants to reduce per-iteration cost below quadratic with respect to time, (ii) extensions to richer feedback models (beyond linear preferences or stochastic feedback), and (iii) broader application to other inverse decision-making tasks with action-only supervision. Additionally, understanding further improper-learning regimes where structure can obviate the cost of constraining online iterates is a compelling theoretical direction.

Conclusion

CoRectron presents a significant advance in contextual recommendation, attaining the theoretically optimal logarithmic regret in a practical, projection-free, and computationally efficient manner. Its robustness to feedback imperfections and applicability in kernelized settings widen the range of deployable inverse optimization algorithms for complex decision environments. The analysis also yields conceptual clarity on the power of improper, Perceptron-like algorithms in online contextual learning. Future research will further refine the interplay between computational and statistical efficiency for action-only feedback models.

Reference:

"Simple Projection-Free Algorithm for Contextual Recommendation with Logarithmic Regret and Robustness" (2603.20826)