ParetoBandit: Multi-Objective Bandit Methods

• ParetoBandit is a family of bandit methods that use Pareto optimality to evaluate multiple objectives simultaneously, defining optimal sets of arms.
• It encompasses a range of methodologies including fixed-budget and fixed-confidence Pareto set identification, regret minimization, and cost-aware routing for non-stationary systems.
• These algorithms employ adaptive sampling, robust statistics, and structured linear models to efficiently handle trade-offs between competing performance criteria.

ParetoBandit is a label used in arXiv literature for bandit methods organized around Pareto optimality rather than a single scalar utility. In multi-objective multi-armed bandits, the central object is usually the Pareto set of arms whose mean vectors are not strictly dominated by any other arm; the associated tasks are Pareto set identification, Pareto regret minimization, and related pure-exploration problems (Kone et al., 2023). In a separate line of work on production inference systems, ParetoBandit denotes an open-source adaptive router for non-stationary LLM serving that treats quality and dollar cost as competing criteria under explicit budget ceilings (Taberner-Miller, 31 Mar 2026). This suggests that the term is best understood as a family resemblance term spanning several mathematically distinct uses of the Pareto principle in bandit design.

1. Terminological scope and the Pareto principle

In the multi-objective bandit setting, each arm produces a vector-valued observation, and dominance is defined coordinate-wise. A standard formulation says that arm $j$ weakly dominates arm $i$ when $\mu_i^\ell \le \mu_j^\ell$ for all objectives $\ell$, with strict domination when at least one coordinate is strictly smaller. The Pareto-optimal set is then the set of arms not strictly dominated by any other arm, for example

$$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$

This definition is common across fixed-budget, fixed-confidence, Bayesian anytime, robust, and structured variants of Pareto set identification (Kone et al., 2023).

A second use of “Pareto” in bandit research concerns Pareto frontiers between competing performance criteria rather than Pareto dominance among vector rewards. Representative examples include the frontier between regret minimization and best-arm identification, the per-action regret frontier, the trade-off between regret and allocation variability, and the frontier of regret rates for model selection in linear bandits (Zhong et al., 2021, Lattimore, 2015, Chen et al., 7 Feb 2026, Zhu et al., 2021). In this sense, “ParetoBandit” refers to algorithms that move along an efficiency frontier where improving one criterion necessarily worsens another.

A third use appears in cost-aware routing for model portfolios. There, the Pareto object is the quality-cost trade-off in a contextual bandit with dollar-denominated constraints. The router is evaluated by how well it traces the true Pareto frontier between fixed models while keeping mean per-request cost within a target ceiling (Taberner-Miller, 31 Mar 2026).

2. Canonical multi-objective bandit model

The canonical stochastic model has $K$ arms, each arm $i$ yielding i.i.d. $d$-dimensional observations from an unknown distribution with mean vector $\mu_i\in\mathbb R^d$. In the fixed-budget formulation, the learner has a total sampling budget $B$, chooses arms sequentially, and after $i$0 pulls outputs an estimate $i$1 of the true Pareto set $i$2; the objective is to minimize the misidentification probability $i$3 (Kone et al., 2023). In the fixed-confidence formulation, the learner instead chooses a stopping time $i$4 and must return a correct set with probability at least $i$5, while minimizing $i$6 (Kone et al., 2023, Crépon et al., 29 Jan 2025).

A recurrent technical ingredient is a gap notion quantifying how robustly an arm belongs, or does not belong, to the Pareto front. In fixed-budget Pareto set identification, the definitions use

$i$7

together with arm-specific gaps $i$8. The resulting complexity measure is

$i$9

where $\mu_i^\ell \le \mu_j^\ell$0 are the ordered gaps (Kone et al., 2023). In the multi-output linear model, the structured analogues are

$\mu_i^\ell \le \mu_j^\ell$1

and the paper states that the difficulty mainly depends on the sub-optimality gaps of $\mu_i^\ell \le \mu_j^\ell$2 arms only (Kone et al., 6 Jul 2025).

The same general model admits important extensions. Constrained Pareto set identification adds a known convex polyhedron $\mu_i^\ell \le \mu_j^\ell$3, so the target becomes the Pareto set of feasible arms (Kone et al., 9 Jun 2025). Robust Pareto set identification replaces mean vectors by coordinate-wise medians under $\mu_i^\ell \le \mu_j^\ell$4-contaminated feedback and introduces an unavoidable bias $\mu_i^\ell \le \mu_j^\ell$5 determined by the contamination model (Korkmaz et al., 2022). Piecewise-stationary formulations let the mean vectors change at unknown breakpoints and redefine regret relative to the time-varying Pareto front $\mu_i^\ell \le \mu_j^\ell$6 (Balef et al., 2023).

3. Pareto set identification algorithms

The literature now contains fixed-budget, fixed-confidence, anytime Bayesian, constrained, robust, and structured algorithms. The common pattern is adaptive sampling near the boundary between Pareto-optimal and suboptimal arms, but the mechanisms differ substantially.

Variant	Representative method	Stated guarantee
Fixed-budget exact PSI	EGE-SR, EGE-SH	$\mu_i^\ell \le \mu_j^\ell$7 decays exponentially in $\mu_i^\ell \le \mu_j^\ell$8 with exponent of order $\mu_i^\ell \le \mu_j^\ell$9
Fixed-confidence relaxed PSI	APE, $\ell$0-APE-$\ell$1	Correctness for $\ell$2-PSI-$\ell$3 with explicit sample bound
Anytime Bayesian PSI	TTPFTS	$\ell$4
Constrained PSI	e-cAPE	$\ell$5-correct with near-optimal sample complexity up to logarithmic factors
Robust contaminated PSI	R-PSI	$\ell$6-PAC under contamination
Structured linear PSI	GEGE, Track-and-Stop-style ParetoBandit, PFIwR	Nearly optimal guarantees exploiting linear structure

The fixed-budget milestone is “Empirical Gap Elimination,” a round-based elimination template parameterized by active-set sizes and per-round sample budgets. Its two concrete instances, EGE-SR and EGE-SH, are the first algorithms for the fixed budget Pareto Set Identification task. The paper proves upper bounds of the form

$\ell$7

and complements them with an information-theoretic lower bound based on $\ell$8, establishing unimprovability up to constant and $\ell$9 factors in the worst case (Kone et al., 2023).

In fixed-confidence PSI, “Adaptive Pareto Exploration” provides a single sampling rule that can be paired with different stopping rules for three correctness criteria: $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$0-Pareto Set Identification, $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$1-cover, and $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$2-PSI-$$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$3. The paper emphasizes that exact Pareto set identification can have very large sample complexity, and that allowing a relevant subset of the Pareto set or additional near-optimal arms can significantly reduce sampling cost. Its upper bound for $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$4-APE-$$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$5 depends on $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$6, and the experiments on COV-BOOST report that exact PSI uses $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$7–$$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$8 fewer samples than PSI-Unif-Elim on average, while $$S^\star=\{\,i\in[K]: \text{there is no } j\in[K] \text{ with } \mu_i\prec \mu_j\,\}.$$9-relaxation can reduce sample complexity by up to $K$0 (Kone et al., 2023).

The first anytime Bayesian PSI algorithm is Top-Two Pareto Front Thompson Sampling. At each round it samples posterior means for all arms, computes a first sampled Pareto front $K$1, with probability $K$2 computes a second front $K$3, and plays uniformly from the selected front. The recommendation $K$4 is the Pareto set of posterior means, and the main theorem gives asymptotic correctness in probability. The same work introduces an uncertainty metric

$K$5

where $K$6 is a Bhattacharyya-coefficient overlap between posterior distributions; empirically, $K$7 tracks performance and provides a ground-truth-free stopping signal (Saerens et al., 17 Jun 2026).

Several specialized variants extend PSI beyond the basic stochastic setting. Constrained Pareto Set Identification with Bandit Feedback introduces e-cAPE, which jointly reasons about feasibility and dominance; its expected sample complexity is controlled by a problem-dependent quantity $K$8, and the lower bound shows near-optimality in the small-$K$9 regime. Robust Pareto Set Identification with contaminated feedback introduces R-PSI, a sample median-based adaptive elimination method that is $i$0-PAC and, in the subgaussian case, satisfies

$i$1

For structured models, two lines are especially prominent. In the multi-output linear model, GEGE uses G-optimal design and least-squares estimation and achieves nearly optimal guarantees in both fixed-budget and fixed-confidence settings. In the Gaussian fixed-confidence setting, “Sequential Learning of the Pareto Front for Multi-objective Bandits” adapts Track-and-Stop, solves the minimax transportation oracle in time $i$2 per round, and is asymptotically optimal as $i$3 (Kone et al., 9 Jun 2025, Korkmaz et al., 2022, Kone et al., 6 Jul 2025, Crépon et al., 29 Jan 2025). A related linear-bandit line, “Learning the Pareto Front Using Bootstrapped Observation Samples,” introduces a mixed doubly robust estimator, obtains sample complexity optimal up to a logarithmic factor, and controls the Pareto regret incurred during estimation within a logarithmic factor of the optimal regret among all algorithms that identify the Pareto front (Kim et al., 2023).

4. Pareto regret and other frontier formulations

One major strand of work treats Pareto optimality directly in regret analysis rather than in pure exploration. “Pareto Regret Analyses in Multi-objective Multi-armed Bandit” defines a scalarization-free distance to the Pareto front,

$i$4

and shows the equivalent expression

$i$5

It then proposes MO-KS, which runs UCB or EXP3.P on a chosen coordinate depending on whether the regime is stochastic or adversarial, and MO-US, an EXP3++-style best-of-both-worlds method with $i$6 in stochastic settings and $i$7 in adversarial settings. The same paper also shows that Pareto-UCB can be driven to linear regret by a small online attack of cost $i$8 (Xu et al., 2022).

A non-stationary version appears in the piecewise-stationary multi-objective Bernoulli model, where ParetoBandit combines restartable Bayesian online change-point detection with a multi-dimensional UCB region. The instantaneous loss is a Pareto gap $i$9, and the cumulative Pareto regret is $d$0. The stated regret is $d$1 when the number of breakpoints $d$2 is known and $d$3 otherwise (Balef et al., 2023).

A different cluster of papers uses “Pareto” to describe efficiency frontiers between objectives. “Achieving the Pareto Frontier of Regret Minimization and Best Arm Identification in Multi-Armed Bandits” shows that no algorithm can simultaneously be optimal for both regret minimization and fixed-budget best-arm identification, and introduces BoBW-lil'UCB$d$4, which traces an order-wise optimal trade-off curve between $d$5 and $d$6 (Zhong et al., 2021). “The Pareto Regret Frontier for Bandits” studies vectors of per-action worst-case regrets $d$7 and characterizes the achievable region up to constants, with unbalanced MOSS matching the frontier in the stochastic setting (Lattimore, 2015). “Bandit Allocational Instability” proves that worst-case regret $d$8 and worst-case allocation variability $d$9 must satisfy $\mu_i\in\mathbb R^d$0 whenever $\mu_i\in\mathbb R^d$1, and shows that UCB-$\mu_i\in\mathbb R^d$2 can attain the Pareto curve $\mu_i\in\mathbb R^d$3 (Chen et al., 7 Feb 2026). In linear bandit model selection, “Pareto Optimal Model Selection in Linear Bandits” proves that adaptation to the unknown intrinsic dimension $\mu_i\in\mathbb R^d$4 has an unavoidable cost and proposes a Pareto-optimal algorithm whose rate family is $\mu_i\in\mathbb R^d$5 (Zhu et al., 2021).

5. ParetoBandit for non-stationary LLM serving

In LLM systems, ParetoBandit is a contextual router rather than a Pareto set identifier. The problem is a cost-aware contextual bandit: at each step $\mu_i\in\mathbb R^d$6, the system observes prompt features $\mu_i\in\mathbb R^d$7, chooses an arm $\mu_i\in\mathbb R^d$8, receives quality reward $\mu_i\in\mathbb R^d$9 and dollar cost $B$0, and seeks

$B$1

The core selection rule is a budget-augmented LinUCB score

$B$2

where $B$3 is an online dual variable updated by an EMA-smoothed ascent (Taberner-Miller, 31 Mar 2026).

The router combines three mechanisms. First, an online primal-dual budget pacer enforces a per-request cost ceiling over an open-ended stream: $B$4 with $B$5, $B$6, and $B$7. Second, geometric forgetting discounts each chosen arm’s sufficient statistics by $B$8, with $B$9 and effective e-folding time $i$00 steps, so stale evidence decays and the policy adapts to shifts in model quality or price. Third, a hot-swap registry allows operators to add or remove models at runtime; newcomers receive a forced-exploration burn-in of $i$01 pulls, after which standard UCB selection resumes (Taberner-Miller, 31 Mar 2026).

The evaluation uses a three-tier portfolio spanning $i$02 per request, $i$03 hold-out prompts, and seven budget ceilings from $i$04 per request. Across those budgets, mean per-request cost never exceeds the target by more than $i$05. When the costliest model’s price is cut by $i$06, the router automatically shifts traffic and achieves up to $i$07 quality lift under a tight budget; when a silent quality regression drops Mistral’s reward from $i$08 to $i$09, the system detects the degradation via bandit feedback, reallocates away, and recovers to $i$10 of Phase 1 quality within the $i$11-prompt recovery window while holding cost within ceiling $i$12. A cold-started fourth arm reaches stable adoption within $i$13 steps without budget overshoot. End-to-end routing latency is $i$14 ms on CPU, and the routing decision alone takes $i$15 per request (Taberner-Miller, 31 Mar 2026).

The paper is explicit that classical $i$16 BwK bounds do not directly apply because of non-stationarity, EMA smoothing, the hard ceiling, and log-normalized cost. Instead it identifies three invariants that hold by construction: $i$17, staleness inflation ensures eventual re-exploration of any neglected arm, and when $i$18 the hard ceiling caps any single-request cost (Taberner-Miller, 31 Mar 2026).

6. Applications, empirical patterns, and open technical issues

The application range is broad. Multi-objective pure exploration papers evaluate on COV-BOOST vaccination strategies, SNW sorting networks, convex Pareto curves, clustered groups, uniform-gap arms, circle-shaped fronts, and high-dimensional fronts (Kone et al., 2023). Relaxed fixed-confidence PSI is studied on a Covid-19 vaccine design problem based on the COV-BOOST phase 2 trial with $i$19 vaccination strategies and three immunogenicity endpoints (Kone et al., 2023). Bayesian anytime PSI is demonstrated on eight synthetic environments and on molecular discovery over a combinatorial $i$20 million-molecule library, where TTPFTS reaches $i$21 by $i$22 steps and remains high (Saerens et al., 17 Jun 2026). Constrained PSI is evaluated on Secukinumab clinical trial data and CovBoost-19 vaccine data (Kone et al., 9 Jun 2025). Robust PSI is tested under review bombing and diabetes management scenarios, including the UVA/PADOVA diabetes simulator (Korkmaz et al., 2022). Non-stationary Pareto regret is illustrated on a joint communications and sensing toy problem, a synthetic $i$23 Bernoulli dataset, and the Yahoo! R6A click/pay dataset (Balef et al., 2023). Structured linear variants report benchmarks on real multi-criteria tasks such as NoC designs and energy-efficiency data, and PFILin is evaluated on SW-LLVM (Kone et al., 6 Jul 2025, Kim et al., 2023).

Several empirical regularities recur across these papers. In fixed-budget exact PSI, the observed $i$24 versus budget curves are straight lines on a log-linear plot, confirming exponential decay, and EGE-SR and EGE-SH dramatically outperform Uniform Allocation (Kone et al., 2023). Relaxations that allow additional near-optimal arms or require only $i$25 Pareto-optimal arms can reduce sample complexity dramatically (Kone et al., 2023). Exploiting structure matters: in linear or low-intrinsic-dimensional models, complexity depends on the ambient arm count far less strongly than in unstructured baselines (Kone et al., 6 Jul 2025, Kim et al., 2023). Robust statistics are essential under contamination, where mean-based elimination can fail catastrophically (Korkmaz et al., 2022). In constrained settings, jointly reasoning about feasibility and dominance is substantially more sample-efficient than a two-stage pipeline (Kone et al., 9 Jun 2025).

Open issues are also consistent across the literature. Exact Pareto set identification may have very large sample complexity, which motivates relaxed criteria (Kone et al., 2023). The Bayesian anytime line provides asymptotic correctness but no explicit finite-time bounds; the paper conjectures exponential decay at a rate determined by the Pareto gaps (Saerens et al., 17 Jun 2026). Exact Pareto front extraction can scale poorly in $i$26 and $i$27, so approximate sorting or pruning heuristics may be needed for very large action sets (Saerens et al., 17 Jun 2026). In piecewise-stationary settings, false-alarm and detection-delay tuning remains delicate (Balef et al., 2023). In linear contextual front learning, the initial exploration cost may dominate when all gaps are large (Kim et al., 2023). These recurring limitations indicate that “ParetoBandit” remains an active research area rather than a closed algorithmic template.