DP-NCB: Differentially Private Nash Confidence Bound

• The paper introduces DP-NCB, a framework that jointly guarantees differential privacy and Nash fairness, achieving order-optimal Nash regret bounds.
• DP-NCB combines a uniform exploration phase with a private adaptive exploitation phase, using Laplace noise for both global and local differential privacy models.
• The framework demonstrates improved fairness and privacy over existing methods, making it ideal for critical applications like clinical trials and personalized medicine.

Differentially Private Nash Confidence Bound (DP-NCB) is a unified algorithmic paradigm for multi-armed bandit problems in which both differential privacy and Nash fairness are central algorithmic constraints. DP-NCB algorithms ensure that actions taken over sequential rounds protect sensitive information about participants (differential privacy) while also providing fairness guarantees with respect to Nash regret, a metric that penalizes disparities in individual outcomes. The DP-NCB framework generalizes to settings with both global and local differential privacy models and achieves order-optimal Nash regret, matching known lower bounds up to logarithmic factors. This makes DP-NCB a principled solution for deploying bandit algorithms in high-stakes, socially sensitive domains such as clinical trials, adaptive experimentation, and personalized interventions.

1. Algorithmic Structure and Privacy Models

DP-NCB algorithms operate in two major phases: an initial "Uniform Exploration" phase and a "Private Adaptive Exploitation" phase. During uniform exploration, all arms are sampled uniformly to obtain private initial estimates of their means; in the subsequent exploitation phase, the algorithm leverages a Nash Confidence Bound (NCB) to select arms, incorporating corrections for both statistical and privacy-induced uncertainty.

DP-NCB is instantiated under two differential privacy models:

• Global Differential Privacy (GDP-NCB): Assumes a trusted server has access to raw rewards and releases aggregate statistics through episodic, noise-adding mechanisms (e.g., binary tree/episodic release as in Chan et al.), with differentially private means released only after accumulating a batch of observations. This reduces privacy loss per interaction, allowing finer calibration of noise.
• Local Differential Privacy (LDP-NCB): Each reward is privatized at the source—each user perturbs their own reward using, for example, Laplace noise of scale $\sim1/\epsilon$—and thus provides a strictly stronger privacy guarantee at the expense of higher noise.

The core mechanism guaranteeing $\epsilon$-differential privacy is Laplace noise addition, with the noise scale chosen according to the global sensitivity of the collected statistics and the desired privacy budget, as formalized by ensuring:

$$\Pr[M(D) \in S] \leq \exp(\epsilon) \Pr[M(D') \in S]$$

for all neighboring datasets $D, D'$, and all measurable sets $S$.

2. Nash Confidence Bounds and Regret Metrics

The Nash Confidence Bound is a variant of the classical upper confidence bound tailored to fairness considerations. For arm $i$ at time $t$, it is defined as:

$$\mathrm{NCB}(i, t) = \pi_i + \text{noise compensation terms}$$

where $\pi_i$ is the (private) empirical mean estimate, and the compensation terms account for the total uncertainty (statistical plus privacy-induced). The exact form of these terms depends on the privacy model and the noise injection mechanism; for instance, in LDP, the adjustment reflects the high-variance Laplace noise on each observed reward.

Nash regret, denoted $NRT$, is the shortfall in Nash social welfare:

$$NRT = u^* - \left( \prod_{t=1}^T \mathbb{E}[r_t] \right)^{1/T}$$

where $u^*$ is the optimal Nash social welfare (geometric mean of rewards achieved by an oracle strategy), and $r_t$ is the reward at round $t$. Unlike average (utilitarian) regret, Nash regret captures the distributional fairness of rewards across arms and rounds, penalizing policies that neglect minority outcomes even if average performance is satisfactory.

3. Privacy-Fairness Trade-offs and Theoretical Guarantees

DP-NCB provides formal guarantees for both privacy and fairness under tight information-theoretic bounds:

• Differential Privacy: Both GDP-NCB and LDP-NCB are proven to achieve the specified $\epsilon$-differential privacy, with privacy loss controlled by careful episodic/private release strategies and per-round budget accounting.
• Order-Optimal Nash Regret: For both GDP-NCB and LDP-NCB, Nash regret is shown to obey:

$$NRT = O\left( \frac{k \log T}{\epsilon} + \mathrm{polylog}(T, k) \right)$$

up to logarithmic factors, where $k$ is the number of arms and $T$ is the horizon. Notably, these bounds match information-theoretic minimax lower bounds for Nash regret, even when compared to non-private baselines. In LDP-NCB, the bound incurs a worse dependence on $1/\epsilon$ due to the stronger privacy guarantee (higher noise per observation).

Preceding private bandit algorithms typically optimized for average regret and did not address fairness, while fairness-aware algorithms largely ignored privacy constraints. DP-NCB is the first framework to jointly achieve both objectives with provable optimality.

4. Empirical Performance and Simulations

Comprehensive simulations are conducted on synthetic multi-armed bandit instances, including heterogeneous Bernoulli bandit problems with large numbers of arms and varied reward gaps. The evaluation compares:

• Non-private NCB,
• AdaP-UCB (a state-of-the-art average-regret-optimal private bandit algorithm),
• GDP-NCB,
• LDP-NCB.

Results indicate DP-NCB achieves substantially lower Nash regret than AdaP-UCB, especially when there are arms with small expected rewards or pronounced disparities. As privacy is strengthened (smaller $\epsilon$), the Nash regret increases, but the increase follows the theoretical bounds. In settings with mixed or heterogeneous reward distributions, DP-NCB robustly maintains fairness, preventing any population subgroup or arm from being persistently disadvantaged.

The GDP-NCB model demonstrates faster decay of Nash regret for large $k$ and moderate $\epsilon$, while LDP-NCB, although incurring a penalty due to individual-level privacy, still considerably outperforms previous LDP methods in Nash regret.

5. Applications and Implications

DP-NCB is directly applicable to any bandit-based system where privacy, fairness, or social sensitivity is paramount. Example application domains include:

• Clinical Trials: DP-NCB ensures that no patient receives systematically inferior treatment in adaptive clinical trial arms, while individual-level privacy is strictly preserved.
• Personalized Medicine and Recommendation: Adaptive experiments (e.g., drug assignments, content recommendations) can be conducted with rigorous guarantees that both protect personal health data and prevent systematic discrimination.
• Decision-Making in Public Policy: Environments requiring the balancing of individual welfare and privacy (e.g., adaptive allocation of social resources) can directly benefit from DP-NCB's principled approach.

The integration of Nash social welfare with differential privacy represents a foundational advance for socially responsible machine learning and sequential decision-making, motivating further research into other settings (e.g., contextual bandits, reinforcement learning) where these constraints are jointly critical.

6. Extensions, Generalizations, and Future Directions

The general structure of DP-NCB is modular, accommodating alternative privacy models (approximate DP, concentrated DP), extended bandit formulations (e.g., contextual bandits, nonparametric bandits), and more complex fairness metrics (beyond Nash social welfare). The order-optimality results motivate the adaptation of the DP-NCB concept to broader classes of online learning and interactive inference.

Potential directions include:

• Extending Nash confidence bounds to account for covariate shift and auxiliary data (as in (Ma et al., 11 Mar 2025)), thus accelerating fair learning under privacy constraints.
• Integrating DP-NCB with noise-aware Bayesian inference or multiple imputation (as in (Räisä et al., 2022)) for robust uncertainty quantification under privacy in strategic or game-theoretic environments.
• Investigating the limit regimes of privacy-fairness trade-off, leveraging information-theoretic lower bounds and refining noise calibration mechanisms to minimize worst-case regret.

A plausible implication is that DP-NCB sets a new benchmark for fair, private online learning, with rigorous theoretical underpinnings and flexibility for high-impact, real-world deployments.