Tight Regret Bound for Online Inverse Linear Optimization via Multiscale Matrix Weights

Abstract: We study online inverse linear optimization with a fixed unknown linear utility: in each round, an environment presents a compact action set, the learner recommends an action from it, and the environment returns an action that maximizes the utility over the same set. When the utility vector and the actions lie in the $d$-dimensional Euclidean unit ball, we give a randomized algorithm whose regret---the cumulative utility shortfall relative to optimal actions---is $O(\sqrt d)$ in expectation for every time horizon, without knowledge of the horizon. The dependence on $d$ is optimal up to a constant factor by the known $Ω(\sqrt d)$ lower bound for horizons $T\ge d$. Our algorithm maintains matrix multiplicative weights on polynomial feature spaces at geometrically spaced scales. It selects a recommendation distribution by solving a linear program and updates its score matrices by comparing the available actions with the feedback action. With rational oracle outputs and feedback actions, an implementation computable relative to a linear-optimization oracle preserves the $O(\sqrt d)$ regret bound. Whether the same rate is attainable with running time polynomial in the dimension, horizon, and input length remains open.

• The paper establishes a tight horizon-independent expected regret bound of $2^{21}ackslash sqrt{d}$ for online inverse linear optimization, eliminating $O(d)$ horizons.
• It introduces a multiscale matrix multiplicative-weights construction using polynomial feature spaces to enable effective learning without prior knowledge of the horizon.
• The method handles arbitrary compact action sets and adapts to an adaptive context, demonstrating pairwise comparisons across geometrically separated accuracy scales.

Problem formulation and principal result

The paper studies online inverse linear optimization under an unusually weak feedback model. An unknown utility vector $u \in B_2^d$ is fixed throughout the interaction. At round $t$, an adaptive environment reveals a nonempty compact action set $Z_t \subseteq B_2^d$. The learner publishes a probability distribution $P_t$ over $Z_t$, samples and reveals a recommendation $A_t \sim P_t$, and then observes an action $Y_t$ selected from the best-response set

$BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$

The learner observes the optimal action but never observes the utility value, the utility vector, or the comparison direction $Y_t-A_t$ before making its recommendation. The instantaneous regret is

$$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$

The main theorem establishes a horizon-independent expected regret bound

$t$0

for every horizon $t$1, every $t$2, and every adaptive environment, without requiring prior knowledge of $t$3 (2609.26978). The result improves the previously stated $t$4 horizon-independent guarantees for arbitrary compact action sets to $t$5, and it matches a lower bound of $t$6 up to the universal constant $t$7. Thus, under Euclidean normalization, the minimax expected regret is characterized in its dependence on dimension for all $t$8.

The lower bound is elementary but structurally important. For the first $t$9 rounds, the environment presents $Z_t \subseteq B_2^d$0 and chooses the utility vector $Z_t \subseteq B_2^d$1, where the $Z_t \subseteq B_2^d$2 are independent random signs. Before the learner selects its action in round $Z_t \subseteq B_2^d$3, the sign $Z_t \subseteq B_2^d$4 remains unknown, so the expected utility of its recommendation is zero, whereas the optimal action has utility $Z_t \subseteq B_2^d$5. The cumulative expected regret over the first $Z_t \subseteq B_2^d$6 rounds is therefore $Z_t \subseteq B_2^d$7. The construction has unique optimal actions and uses only two-point action sets, showing that the lower bound does not depend on ties, continuous action sets, or computational complications.

Multiscale comparison through polynomial feature spaces

The central technical device is a collection of comparison matrices indexed by geometrically spaced scales. For each scale $Z_t \subseteq B_2^d$8, the algorithm uses

$Z_t \subseteq B_2^d$9

where $P_t$0 is the dimension of the polynomial feature space containing all monomials of degree at most $P_t$1.

The utility vector is embedded at scale $P_t$2 into a normalized polynomial feature vector $P_t$3. The normalization is chosen so that $P_t$4 has unit Euclidean norm and its coordinates reproduce truncated exponential moments of $P_t$5. For any unit direction $P_t$6, the construction defines a symmetric operator $P_t$7 on the feature space. The operator is obtained by rotating a fixed degree-swapping operator associated with $P_t$8; algebraically, it can also be represented using lowering operators and a polynomial projector onto the degree-one eigenspace.

For actions $P_t$9, the comparison matrix is

$Z_t$0

with $Z_t$1. It satisfies

• $Z_t$2;
• $Z_t$3;
• the quadratic form $Z_t$4 has the sign of $Z_t$5;
• the corresponding quadratic form for $Z_t$6 is nonnegative.

The construction is designed so that the first and second quadratic forms are simultaneously controlled. More specifically, if $Z_t$7, then, up to a factor $Z_t$8,

$Z_t$9

and

$A_t \sim P_t$0

where

$A_t \sim P_t$1

A single scale cannot uniformly control the ratio between the first-order signal $A_t \sim P_t$2 and the second-order correction $A_t \sim P_t$3: that ratio diverges as the utility gap approaches zero. The multiscale construction resolves this by summing the terms with weights $A_t \sim P_t$4. For every $A_t \sim P_t$5 and truncation level $A_t \sim P_t$6,

$A_t \sim P_t$7

while

$A_t \sim P_t$8

The lower bound incurs only a cutoff error of order $A_t \sim P_t$9. This dyadic aggregation is the mechanism that allows a fixed learning rate to handle both large and arbitrarily small positive utility differences.

Matrix multiplicative weights and the recommendation distribution

At each active scale, the algorithm maintains a symmetric score matrix $Y_t$0. It forms the density matrix

$Y_t$1

For a finite action set $Y_t$2, the learner constructs the skew-symmetric comparison matrix

$Y_t$3

where $Y_t$4. The learner then solves a linear feasibility problem for a probability vector $Y_t$5 satisfying

$Y_t$6

This balance condition exists for every skew-symmetric matrix by the minimax theorem. Its role is stronger than merely selecting a low-value action under the current score: it ensures that, for every possible feedback action $Y_t$7, the expected trace contribution of the first-order update is nonpositive,

$Y_t$8

Consequently, the environment may break ties after observing the sampled recommendation without invalidating the potential argument.

After observing $Y_t$9, the score matrices are updated using the entire recommendation distribution rather than only the realized sample:

$BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$0

with fixed $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$1. Averaging the update over $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$2 is essential: it makes the balance constraint apply to the actual update even when the feedback action depends on the sampled recommendation.

The algorithm is improper. Its recommendation distribution generally does not correspond to maximizing a single utility estimate fixed before seeing $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$3. This distinction is important in comparison with proper algorithms based on online Newton steps or variable-metric updates. The improved $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$4 dependence is obtained by allowing the learner to exploit the current action set through a distribution over actions and pairwise comparisons.

Potential analysis and the origin of the $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$5 rate

The analysis uses the matrix potential

$BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$6

which is nonnegative for every symmetric $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$7 and unit vector $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$8. At initialization, $BR_{Z_t}(u)=\arg\max_{y\in Z_t}\langle u,y\rangle.$9.

For a symmetric matrix $Y_t-A_t$0 with $Y_t-A_t$1, the scalar inequality

$Y_t-A_t$2

for sufficiently small fixed $Y_t-A_t$3 yields the matrix inequality

$Y_t-A_t$4

Combined with the Golden–Thompson inequality, this gives

$Y_t-A_t$5

Convexity of the log-trace exponential extends the inequality to the distribution-weighted update. The balance condition eliminates the aggregate trace term, so the potential decrease controls the utility difference between the feedback action and the mean recommendation.

The comparison inequalities imply, for an optimal feedback action $Y_t-A_t$6,

$Y_t-A_t$7

where $Y_t-A_t$8 and $Y_t-A_t$9 is the weighted increase in the linear part of the score matrices evaluated at $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$0. Summing over time telescopes the potentials across all scales:

$$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$1

The crucial dimensional estimate is

$$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$2

This bound follows from the scale choice $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$3. When $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$4, the feature-space dimension behaves differently from when $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$5; the transition occurs around $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$6, equivalently $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$7. The geometric weights $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$8 make the contributions on both sides of this transition summable, with the largest aggregate contribution occurring near the balance scale. This is the precise source of the $$\Delta_u(Z_t,A_t) = \max_{y\in Z_t}\langle u,y\rangle-\langle u,A_t\rangle = \langle u,Y_t-A_t\rangle.$$9 dependence.

The cutoff schedule $t$00 gives $t$01, so the approximation errors are summable over an arbitrarily long interaction. The resulting finite-action bound is

$t$02

and hence the stated $t$03 expected-regret guarantee after translating from the mean recommendation to the sampled recommendation.

Extension from finite to compact action sets

The finite-action analysis does not directly apply to arbitrary compact sets because the optimal feedback action may not belong to the finite list used to define the recommendation distribution. The paper addresses this by constructing a finite list whose convex hull approximates the convex hull of the action set.

At round $t$04, the learner chooses a list $t$05 such that every point in $t$06 lies within $t$07 of $t$08. This list is generated through linear-optimization oracle calls over a finite rational grid of directions. After observing $t$09, the learner finds coefficients $t$10 such that

$t$11

approximates $t$12 within $t$13 in Euclidean norm. The score update is then averaged over both the recommendation distribution $t$14 and the feedback representation $t$15.

The approximation introduces two errors. First, replacing $t$16 by $t$17 costs at most $t$18 in utility. Second, the represented feedback action may be suboptimal relative to $t$19, so the negative-part term in the pairwise comparison inequality no longer vanishes exactly. The paper bounds the aggregate effect by

$t$20

Choosing $t$21 makes this error summable. The compact-action analysis therefore preserves the same $t$22 rate and the same explicit constant.

This treatment is significant because it avoids finite action-set, polyhedral, margin, lattice, or uniqueness assumptions. The action sets may be arbitrary nonempty compact subsets of the Euclidean unit ball, and ties among optimal actions may be resolved adaptively after the recommendation is observed.

Oracle implementation and computational guarantees

The paper also gives a computable implementation relative to a linear-optimization oracle. The oracle accepts rational query vectors and rational accuracy parameters and returns rational approximate maximizers. Under the additional assumption that feedback actions have rational coordinates, every round terminates for every realization of the learner's randomness and preserves the regret bound.

The implementation has three main components. First, it constructs the finite action list by querying a rational grid of directions. With

$t$23

the number of oracle calls in round $t$24 is

$t$25

For a known upper horizon $t$26, this is $t$27 per round. Second, it approximates comparison matrices using rational interval arithmetic and computes matrix exponentials through finite Taylor expansions. Third, it solves rational linear programs to obtain the recommendation probabilities and the convex representation of the observed feedback action.

The paper carefully propagates approximation errors through the matrix potential argument. The balance violation, feedback approximation, and score-update error are all set to $t$28, yielding a finite total perturbation. This proves termination and preservation of the $t$29 regret bound.

However, the computational guarantee is not a polynomial-time result in the dimension. The oracle-call complexity is exponential in $t$30, and the remaining arithmetic complexity is not bounded by a polynomial in the dimension, horizon, and input length. The paper therefore separates statistical optimality from computational efficiency: it establishes the optimal dimension dependence but does not provide a polynomial-time implementation with the same regret rate.

Limitations and open questions

The explicit constant $t$31 is not optimized and is substantially larger than the information-theoretic scale suggested by the lower bound. The result is also an expected-regret guarantee; the analysis obtains a pathwise bound for the mean recommendations and then uses the sampling identity to control expected regret, but it does not state a high-probability regret theorem.

The computational construction relies on rational oracle outputs and rational feedback actions. The abstract regret theorem itself applies to all compact action sets and does not require rationality, but the computability theorem is conditional on this representation model. Moreover, the finite action list requires $t$32 oracle calls, so the implementation does not resolve whether the $t$33 rate can be attained with running time polynomial in dimension, horizon, and input length.

Finally, the feedback model assumes exact optimality. The paper explicitly leaves open whether the $t$34 dependence survives suboptimal or corrupted feedback. This is nontrivial because the proof uses exact optimality both to eliminate the negative-part term in the pairwise comparison inequality and to identify the regret with the utility gap between the feedback action and the learner's mean recommendation.

Conclusion

The paper establishes a tight $t$35 dependence for expected regret in online inverse linear optimization with arbitrary compact action sets, adaptive contexts, exact optimal-action feedback, and no horizon knowledge (2609.26978). Its main contribution is a multiscale matrix multiplicative-weights construction whose polynomial feature spaces encode action comparisons across geometrically separated accuracy scales. The dyadic aggregation controls both first- and second-order terms, while the weighted feature-space dimensions sum to $t$36. The result closes the dimension-dependence gap between prior $t$37 upper bounds and the $t$38 lower bound, while leaving polynomial-time implementation and robustness to suboptimal feedback unresolved.