Sublinear Social-Welfare Regret in MA-MAB

• The paper introduces sublinear social-welfare regret as a performance measure in sequential learning, highlighting its formulation in multi-agent bandit models with fairness constraints.
• The methodology employs an optimistic linear programming approach with confidence-bound exploration, achieving provable O(√T) social-welfare regret.
• Empirical validations on synthetic and real-world datasets confirm the theoretical trade-offs between social welfare and fairness, underscoring the algorithms' robustness.

Sublinear social-welfare regret formalizes the efficiency of sequential learning algorithms in maximizing social welfare—typically defined as the aggregate or equitable utility of multiple agents—over a time horizon, relative to the best static or fair allocation in hindsight. Recent research rigorously quantifies this notion in multi-agent multi-armed bandit (MA-MAB) models with fairness constraints, yielding algorithmic frameworks that provably achieve regret sublinear in the number of rounds. These works draw sharp boundaries on achievable rates, delineate trade-offs between welfare and fairness regret, and introduce new analytical techniques for establishing matching upper and lower bounds.

1. Formal Model and Social-Welfare Regret

In the multi-agent multi-armed bandit setting with $n$ agents and $m$ arms, the reward matrix $A \in [0,1]^{n \times m}$ is unknown and $A_{i,j}$ denotes agent $i$'s expected reward from arm $j$. At each round $t$, the learning algorithm selects a distribution $\pi^t \in \Delta_m$ over arms, inducing expected utility $\langle A_i, \pi^t \rangle$ to agent $i$. The total social welfare in round $t$ is

$$SW(\pi^t) = \sum_{i=1}^n \langle A_i, \pi^t \rangle.$$

The fairness requirement is encoded via a vector $C \in [0,1]^n$, and the class of fair policies is given by

$$\mathcal{F} = \left\{\pi \in \Delta_m \mid A\pi \geq C \cdot A \right\}.$$

Define $\pi^*$ as the optimal fair policy maximizing social welfare:

$\pi^* = \arg\max_{\pi\in \mathcal{F}} SW(\pi).$

Cumulative social-welfare regret after $T$ rounds is

$$R_{SW}(T) = \sum_{t=1}^T [SW(\pi^*) - SW(\pi^t)] = T\cdot SW(\pi^*) - \sum_{t=1}^T SW(\pi^t).$$

A policy is said to have sublinear social-welfare regret if $R_{SW}(T) = o(T)$; equivalently, the per-round welfare approaches that of the optimal fair policy as $T \to \infty$ (Manupriya et al., 21 Feb 2025).

2. RewardFairUCB Algorithm and Regret Guarantees

The RewardFairUCB algorithm is the canonical solution for achieving sublinear social-welfare regret subject to minimum-reward-guarantee fairness. The method proceeds as follows:

• Exploration phase: Each arm is pulled in round-robin fashion for $t' = m \lceil \sqrt{T} \rceil$ rounds, ensuring balanced arm sampling ($N_j^{t'} = \lceil \sqrt{T} \rceil$ for all $j$).
• Exploitation phase: For each $t > t'$, empirical mean estimates $\widehat{A}_{i,j}^t$ are maintained. UCB and LCB indices for each agent-arm pair are constructed:

$$\overline{A}_{i,j}^t = \widehat{A}_{i,j}^t + \varepsilon_{i,j}^t, \quad \underline{A}_{i,j}^t = \widehat{A}_{i,j}^t - \varepsilon_{i,j}^t, \quad \varepsilon_{i,j}^t = \sigma \sqrt{\frac{2 \ln(8mnT)}{N_j^t}}.$$

• At every round, solve the optimistic LP:

$$\max_{\pi \in \Delta_m} \sum_{i=1}^n \langle \overline{A}_i^t, \pi \rangle \quad \text{such that} \quad \overline{A}^t \pi \geq C \cdot \underline{A}^t,$$

and play the resulting $\pi^t$.

Main theoretical guarantee: For any feasible MA-MAB instance with $T \geq 32n^2\sigma^2$ and $\sigma$ the sub-Gaussian parameter,

$$\mathbb{E}[R_{SW}(T)] \leq 4n\sqrt{2T}(\sigma\ln(2m^2T) + m + \sigma) = \tilde{O}(\sqrt{T})\text{.}$$

The dominant scaling is $O(n\sigma\sqrt{T}\log(mT) + mn\sqrt{T})$ (Manupriya et al., 21 Feb 2025).

Lower bound: Any MA-MAB algorithm necessarily incurs social-welfare regret $\Omega(\sqrt{T})$, by reduction to classical MAB lower bounds.

3. Technical Approach: Analysis and Proof Structure

The proof leverages the following components:

• Exploration Regret Bound: Uniform exploration incurs regret at most $mn\sqrt{T}$ (due to suboptimal arm pulls).
• Optimistic Concentration: By Hoeffding’s inequality and a union bound, with high probability, the confidence intervals contain the true means, so optimistic policies are feasible for the true problem.
• LP Solution Robustness: The solution to the optimistic LP does not underestimate the true welfare of $\pi^*$ due to upper-bound pessimism in the fairness constraints and optimism in the objective.
• Martingale and Deviation Control: Using Azuma–Hoeffding martingale bounds, the sum $$\sum_{t>t'} \mathbb{E}_{j\sim\pi^t}[\sqrt{1/N_j^t}]$$ is shown to be $O(\sqrt{T}\log T)$.
• Tight Gap Analysis: Per-round welfare gap is $$SW(\pi^*) - SW(\pi^t) \leq 2n\mathbb{E}_{j\sim\pi^t}[\varepsilon_{\cdot,j}]$$.

These arguments collectively show that the cumulative welfare regret is $\tilde{O}(\sqrt{T})$ and instance-independent, achieving the minimax optimal rate up to logarithmic factors (Manupriya et al., 21 Feb 2025).

4. Algorithmic Trade-offs: Fairness Versus Welfare Regret

RewardFairUCB is near-optimal for social-welfare regret ($\tilde{O}(\sqrt T)$) but achieves fairness regret of $\tilde{O}(T^{3/4})$. Alternative strategies (e.g., Explore-First or dual-based heuristics) can attain fairness regret matching $\tilde{O}(\sqrt T)$ more closely, but at the cost of increased social-welfare regret $\tilde{O}(T^{2/3})$ or worse. This exposes a precision trade-off:

• Prioritizing welfare regret slows convergence to fairness guarantees.
• Prioritizing fairness regret increases efficiency loss in overall welfare.

Thus, no single algorithm can optimize both regrets simultaneously to the theoretical minimum rates; rather, algorithms trace out a Pareto frontier between social-welfare and fairness regret (Manupriya et al., 21 Feb 2025).

5. Empirical Validation and Observations

Experiments on both synthesized and real-world datasets substantiate the theoretical findings:

• Simulated data ($n=4, m=3, C_i=0.3, T$ up to $10^5$): RewardFairUCB’s social-welfare regret $R_{SW}$ empirically grows as $T^{1/2}$ (log-log plot slope $\approx 1/2$), while Explore-First baselines follow $T^{2/3}$ scaling.
• MovieLens 1M dataset ($n \approx 6,000, m=18, C_i=1/m$): Again, RewardFairUCB achieves sublinear welfare regret rate with exponent close to $1/2$, outperforming all baselines.
• Constant scaling factors in empirical curves align with the theoretical form $$R_{SW} \approx \text{const} \cdot n \sigma \sqrt{T} \log(mT)$$ (Manupriya et al., 21 Feb 2025).

6. Extensions and Theoretical Context

Sublinear social-welfare regret with fairness is a strictly harder objective than maximizing arithmetic-average reward (classical regret). The analysis generalizes to sub-Gaussian rewards, and the information-theoretic lower bound demonstrates that $\Omega(\sqrt{T})$ is unavoidable, even under strong symmetry or restricted reward matrices.

This research positions sublinear welfare regret as an attainable but nontrivial benchmark in stochastic online learning with fairness guarantees. It also provides constructive evidence that principled algorithmic design (confidence-bound-based optimization and careful exploration) suffices for minimax-optimal welfare regret, while exposing new open questions on multi-objective regret trade-offs and extensions to contextual or adversarial settings (Manupriya et al., 21 Feb 2025).