Information-Regret Analysis

• Information-Regret Analysis is the study of how regret is affected by limited, delayed, or asymmetric information in learning systems such as bandits, reinforcement learning, control, and filtering.
• It leverages information-theoretic tools like entropy, mutual information, and divergence measures to derive regret bounds and analyze information-reward tradeoffs.
• The framework spans diverse applications including causal bandits, partial monitoring, robust estimation, and imperfect-information games, highlighting the practical impact of feedback restrictions.

Information-regret analysis is the study of how regret depends on information: what the learner observes, what internal update signals it can represent accurately, what side information it receives, and what informational advantage is granted to the comparator. Across online learning, bandits, reinforcement learning, control, filtering, collaborative filtering, and imperfect-information games, the central theme is consistent: regret can be characterized as the cost of limited, delayed, approximate, or asymmetric information, and this cost can often be bounded through entropy, mutual information, KL divergence, Bregman divergence, local counterfactual structure, or explicit forecast and estimation error terms (Russo et al., 2014, Lattimore et al., 2019, D'Orazio et al., 2019, Sabag et al., 2021, Zhang et al., 2024).

1. Definitions and scope

In the information-theoretic bandit setting of Thompson sampling, the defining object is the per-round tradeoff between expected regret and information gain about the optimal action $A^*$. Russo and Van Roy define the information ratio as

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$

and show that if $\Gamma_t \le \bar\Gamma$ almost surely for all $t$, then

$$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$

This formulation makes regret an explicit function of the information gained about the identity of the optimal action (Russo et al., 2014).

A broader formulation appears in partial monitoring, where the relevant information object need not be mutual information itself. Lattimore and György generalize the Russo–Van Roy machinery from mutual information and entropy to expected Bregman divergence of posterior martingales. Their master theorem states that if

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$

then

$$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$

Here information-regret analysis becomes a generic posterior-martingale argument rather than an entropy-only argument (Lattimore et al., 2019).

A different but related usage appears in robust estimation and control, where regret compares a feasible decision rule to an informationally stronger benchmark. In regret-optimal filtering, the benchmark is the optimal non-causal estimator

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$

and regret is

$\regret = \min_{\text{causal}\ \mathcal K } \left\|T_\mathcal{K}^\ast T_\mathcal{K} - T_{\mathcal{K}_0}^\ast T_{ \mathcal{K}_0}\right\|.$

In that setting, information-regret analysis measures the price of causality, or more precisely the price of lacking future observations (Sabag et al., 2021).

These formulations are not identical, but they share a common structure. This suggests that information-regret analysis is best understood as a family of analyses in which the regret benchmark, the available feedback, and the information state of the learner are all explicit parts of the theorem statement.

2. Information-theoretic regret-information tradeoffs

The clearest information-regret template is the entropy-and-information-gain decomposition used for Thompson sampling. The cumulative expected information acquired about $A^*$ is bounded by the initial uncertainty: $$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$0 which yields the regret bound

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$1

The entropy term is significant because it makes the bound depend on prior uncertainty about which action is optimal rather than only on worst-case problem size such as $$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$2 (Russo et al., 2014).

The same philosophy is extended to minimax partial monitoring by combining a generalized information-regret inequality with a minimax/Bayes equivalence theorem. For finite-action partial monitoring,

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$3

so prior-uniform Bayesian bounds imply minimax regret bounds. This reduction is then paired with one-step inequalities of the form

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$4

yielding minimax rates for locally observable and globally observable partial monitoring, adversarial bandits, and cops and robbers (Lattimore et al., 2019).

A more explicit “bits versus reward” formulation appears in “On Bits and Bandits: Quantifying the Regret-Information Trade-off” (Shufaro et al., 2024). There, the paper introduces Bayesian regret lower bounds that depend on the information the agent accumulates. Under an information constraint $$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$5, the paper derives for finite decision spaces lower bounds of the form

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$6

and for linear bandits

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$7

It also proves information-dependent upper bounds for constrained-observation Thompson sampling, such as

$$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$8

for $$\Gamma_t := \frac{ \mathbb E_t\!\left[R(Y_{t,A^*})-R(Y_{t,A_t})\right]^2 }{ I_t\!\left(A^*;(A_t,Y_{t,A_t})\right) },$$9-armed MAB and

$\Gamma_t \le \bar\Gamma$0

for linear MAB. In that treatment, information is literally measured in bits, and regret is the price paid when those bits are unavailable or limited (Shufaro et al., 2024).

A related reinforcement-learning formulation appears in information-directed reinforcement learning. The generic decomposition is

$\Gamma_t \le \bar\Gamma$1

where the information ratio is determined by how efficiently a policy converts information into value improvement, and the cumulative information term depends on the chosen learning target. The paper’s main conceptual point is that changing the learning target—from the whole environment to a compressed surrogate—can strictly improve the resulting Bayesian regret bound (Hao et al., 2022).

3. Feedback-limited and partially observed settings

In online combinatorial optimization, information-regret analysis appears as the dependence of minimax regret on the feedback model. Audibert, Bubeck, and Lugosi study three assumptions for the feedback the decision maker receives: full information, semi-bandit, and bandit. The corresponding rates summarized in the paper are:

• full information: $\Gamma_t \le \bar\Gamma$2,
• semi-bandit: $\Gamma_t \le \bar\Gamma$3,
• bandit: lower bound $\Gamma_t \le \bar\Gamma$4, best known upper bound $\Gamma_t \le \bar\Gamma$5. This is a direct statement that weaker feedback increases regret, and the increase is tied to the amount and granularity of recoverable coordinate-level information per round (Audibert et al., 2012).

First-order Bayesian regret analysis of Thompson sampling refines this perspective further. The paper introduces the scale-sensitive information ratio

$\Gamma_t \le \bar\Gamma$6

together with the bound

$\Gamma_t \le \bar\Gamma$7

This replaces $\Gamma_t \le \bar\Gamma$8-type bounds by first-order bounds in $\Gamma_t \le \bar\Gamma$9, and it does so by preserving the loss scale round by round (Bubeck et al., 2019).

The same paper replaces entropy over combinatorial actions by coordinate entropy,

$t$0

which yields

$t$1

For $t$2-sparse actions,

$t$3

leading to

$t$4

The technical message is that action entropy is too coarse in combinatorial settings, while coordinate entropy matches the linear loss structure (Bubeck et al., 2019).

Continuous dueling bandits provide another information-limited setting: instead of values, gradients, or noisy losses, the learner gets a single noisy comparison bit,

$t$5

Under strong convexity and smoothness assumptions, the paper proves

$t$6

Although the paper is not framed in information-ratio language, it is explicitly relevant to information-regret analysis because it shows how much regret can still be controlled when full value feedback is compressed down to one-bit noisy pairwise comparison feedback (Kumagai, 2017).

These results, taken together, support a broad but precise conclusion: feedback restriction changes regret not only quantitatively but structurally. Semi-bandit feedback preserves coordinate-level observability; bandit feedback compresses all selected-component losses into one scalar; dueling feedback compresses feedback to a noisy comparison bit. The regret bounds mirror those distinct information channels (Audibert et al., 2012, Kumagai, 2017, Bubeck et al., 2019).

4. Imperfect-information games and local information sets

In extensive-form games with imperfect information, information-regret analysis is local to information sets. Counterfactual regret minimization studies two-player zero-sum imperfect-information extensive-form games with extensive-form representation

$t$7

and the key local object is the instantaneous regret

$t$8

with cumulative counterfactual regret

$t$9

The standard bridge from local to global regret is

$$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$0

and if $$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$1, then the average strategy profile is a $$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$2-Nash equilibrium (Li et al., 2020).

Lazy-CFR sharpens this picture by arguing that global regret should not be bounded by naively summing independent local regrets over all information sets. The paper reconstructs the exact decomposition

$$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$3

and derives for vanilla CFR the bound

$$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$4

with Hedge, compared with the older

$$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$5

style bound. The structural quantity

$$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$6

captures an effective reach-weighted number of information sets, and the paper’s central claim is that it is impossible that immediate regrets are all very large simultaneously (Zhou et al., 2018).

Approximate local information adds another layer. “Bounds for Approximate Regret-Matching Algorithms” studies regret minimization when the learner acts on estimates $$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$7 rather than the true cumulative regret $$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$8. Its core theorem states that approximate $$\mathbb E[\mathrm{Regret}(T,\pi^{\rm TS})] \le \sqrt{\bar\Gamma\, H(A^*)\, T}.$$9-regret matching satisfies the $$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$0-Blackwell condition with

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$1

where

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$2

The master bound is

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$3

The paper’s explicit interpretation is that approximation remains safe when it preserves the decision-relevant transformed regret signal (D'Orazio et al., 2019).

ECFR modifies local information-regret dynamics directly by exponentially reweighting instantaneous regrets: $$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$4 and cumulative ECFR regret is updated as

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$5

The paper claims that the weighted average strategy profile is a

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$6

-Nash equilibrium, although the same source also notes that the mathematics in the paper is not fully clean and that the claimed super-exponential decay is unusual (Li et al., 2020).

This literature uses “information-regret” in a game-theoretic sense: regret is decomposed over information sets, driven by counterfactual reach, and affected by the fidelity of local regret signals and update rules.

5. Side information, prediction, and asymmetric information structures

In several literatures, information-regret analysis studies not partial feedback alone but the value of exogenous side information. In stochastic causal bandits, the key gain comes from estimating a smaller set of conditional reward components indexed by $$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$7, rather than one mean reward per intervention. The expected reward under intervention $$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$8 decomposes as

$$E_t[\Delta_t]\le \alpha+\sqrt{\beta\,E_t[D_F(M_{t+1},M_t)]},$$9

and this yields regret $$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$0 for C-UCB and C-TS, versus $$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$1 without such structure. The paper explicitly states that this is not an information-theoretic regret framework based on mutual information or information gain; it is a structured cumulative regret analysis in which causal knowledge reduces effective estimation complexity from arm count to parent-state count (Lu et al., 2019).

In non-stationary MDPs with forecasts, the role of information is even more direct. The learner receives look-ahead predictions of rewards and transitions, and the regret against the dynamic clairvoyant oracle is

$$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$2

For Model Predictive Dynamic Programming, the paper proves

$$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$3

and in the error-free case,

$$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$4

The paper’s explicit message is that each additional block of $$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$5 predicted steps reduces regret by a multiplicative factor $$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$6, while prediction errors farther ahead are exponentially attenuated (Zhang et al., 2024).

In online quadratic control with asymmetric information structure, the information gap is ignorance of disturbance probabilities $$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$7. Regret is defined as

$$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$8

and the exact decomposition is

$$BR_n\le \alpha n+\sqrt{n\beta\, \operatorname{diam}_F(\mathcal D)}.$$9

Using the linear minimum mean square unbiased estimate,

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$0

the paper proves

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$1

hence

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$2

Here regret is the price of not knowing disturbance statistics from the start (Tan et al., 2018).

Regret-optimal filtering makes the same informational asymmetry explicit in estimation. A causal or strictly causal filter competes against a non-causal filter that has access to future observations. The central reduction is to a Nehari problem,

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$3

and the paper’s own interpretation is that the regret is the norm of the irreducible mismatch caused by the causal estimator’s lack of future information (Sabag et al., 2021).

A final, distinct use of the term appears in collaborative filtering. There regret is

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$4

and the paper’s main conceptual claim is that regret is controlled by how quickly the algorithm can acquire reusable information about latent structure. In the user-only regime and the item-only regime, the lower bounds formalize that unresolved latent ambiguity forces near-random recommendations, so regret is the price of identifying latent clusters (1711.02198).

6. Limitations, caveats, and contested points

Several recurring limitations characterize the literature.

First, many bounds are expectation-only rather than high-probability. This is stated explicitly for approximate regret matching under imperfect internal information (D'Orazio et al., 2019), for continuous dueling bandits (Kumagai, 2017), and for several Bayesian information-theoretic bandit and RL analyses (Russo et al., 2014, Hao et al., 2022).

Second, “imperfect information” does not mean the same thing across papers. In imperfect-information games it refers to information sets and counterfactual reach (Li et al., 2020, Zhou et al., 2018). In approximate regret matching it refers to estimated internal regrets rather than true regrets (D'Orazio et al., 2019). In control it refers to asymmetric knowledge of disturbance statistics (Tan et al., 2018). In predictive control it refers to finite-horizon look-ahead forecasts with horizon-dependent error (Zhang et al., 2024). In filtering it refers to the absence of future observations (Sabag et al., 2021). A plausible implication is that “information-regret analysis” is a methodological umbrella rather than a single formalism.

Third, some results are existential rather than fully algorithmic. In partial monitoring, the minimax theorem shows that prior-uniform Bayesian bounds imply minimax bounds, but the paper notes that the resulting minimax conclusions are existential (Lattimore et al., 2019). In minimax RL based on duality, the main contribution is likewise the equality

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$5

which transfers Bayesian information-theoretic bounds to minimax regret bounds under boundedness and continuity assumptions, but does not by itself produce a new minimax-optimal algorithm (Bongole et al., 2024).

Fourth, some claimed rates are stronger than standard theory would suggest. The ECFR paper claims a weighted-average equilibrium bound proportional to

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$6

while also acknowledging that several equations are typeset incorrectly and that the derivation is terse (Li et al., 2020). That caveat is important because it separates the empirical observation of faster exploitability reduction from the strength of the formal theorem.

Finally, several works emphasize that the relevant error is decision-relevant information distortion, not raw approximation error. “Bounds for Approximate Regret-Matching Algorithms” makes this point explicitly: what matters is

$$\mathcal{K}_0= \mathcal{L}\mathcal{H}^\ast (I + \mathcal{H}\mathcal{H}^\ast)^{-1},$$7

not merely a norm on raw regret tables (D'Orazio et al., 2019). Information-directed RL makes the same structural point at a different scale: the cumulative information term depends on the chosen learning target, and learning a compressed target can improve regret relative to learning the whole environment (Hao et al., 2022). This suggests that a mature information-regret analysis is usually target-dependent: it must specify which uncertainty is actually relevant for decision quality.