Applicability of Algorithmic Information Ratio (AIR) in competitive multi-agent reinforcement learning

Investigate whether the Algorithmic Information Ratio (AIR) framework can be applied to competitive multi-agent reinforcement learning settings, specifically two-player zero-sum and multi-player general-sum Markov games, and determine corresponding regret guarantees and algorithmic formulations under AIR in these environments.

Background

The paper develops Information-Directed Sampling (IDS) algorithms for multi-agent reinforcement learning (MARL) and discusses related concepts. Algorithmic Information Ratio (AIR) is a recent framework for analyzing frequentist regret using Bayesian-type algorithms, extending the information ratio by incorporating alignment to a reference distribution.

While AIR has been studied for bandit and single-agent reinforcement learning problems, its use in competitive MARL (e.g., Markov games) has not been established. Clarifying whether AIR can be effectively formulated and analyzed in multi-agent settings would bridge methodological gaps between IDS and AIR and potentially yield new regret analyses for MARL.