Budgeted-KMR: Bandit and Model Efficiency

• The paper introduces a budget-aware KL-UCB extension that integrates fractional knapsack randomization to achieve asymptotically optimal performance under soft per-round budget constraints.
• Budgeted-KMR is defined in two distinct ways: as a stochastic multi-play bandit algorithm with cost-aware indices and as an iterative framework for model efficiency within the Knob–Meter–Rule abstraction.
• The approach emphasizes explicit margin randomization and precise budget saturation, ensuring methodical exploration and instance-optimal guarantees in both online bandit and model optimization settings.

Searching arXiv for papers that use or define “Budgeted-KMR” to ground the article and disambiguate the term. Budgeted-KMR denotes budget-aware algorithmic schemes in at least two distinct arXiv literatures. In stochastic multiple-play bandits with arm-dependent costs, the term is associated with a KL-UCB extension that embeds per-arm upper confidence bounds into a fractional knapsack oracle and randomizes on the decision boundary, yielding an asymptotically optimal policy under a soft per-round budget constraint (Luedtke et al., 2016). In a later and unrelated formalism for model efficiency, Budgeted-KMR denotes an iterative procedure in the Knob–Meter–Rule framework that repeatedly applies cost-reducing model transformations until a target budget is met, with optional fine-tuning and a policy for choosing the next transformation (Tyagi et al., 19 Aug 2025). The two usages share the budgeted-optimization motif, but they arise from different problem classes and employ different mathematical objects.

1. Terminological scope and principal usages

In the available literature, the label is not tied to a single canonical algorithmic object. One usage concerns budgeted multiple-play bandits; another concerns model-efficiency optimization under a formal KMR abstraction.

Usage	Domain	Core mechanism
Budgeted-KLMR / Budgeted-KMR	Budgeted multiple-play bandits	KL-UCB indices, greedy fractional knapsack, randomization on the margin
Budgeted-KMR	Knob–Meter–Rule model efficiency	Policy-selected rule applications until $C(M) \leq B$

The bandit usage is the more classical algorithmic one. There, the budget is part of the online decision process: each arm $a$ has a known cost $c_a>0$, the agent has a budget $B$ at each round, and the action is a subset of arms chosen so that expected cost satisfies a soft constraint. The KMR-framework usage is broader and more abstract: a model is transformed by deterministic rules controlled by knobs, evaluated by meters, and iteratively driven toward a cost target (Luedtke et al., 2016, Tyagi et al., 19 Aug 2025).

2. Budgeted multiple-play bandits: decision model and oracle structure

The budgeted multiple-play bandit setting generalizes the multi-armed bandit problem with multiple plays by attaching a cost to each arm and imposing a per-round budget. There are $K$ arms, each arm $a$ has a reward distribution $\nu_a$ and a known cost $c_a>0$, and at round $t$ the agent chooses a potentially random subset $\mathcal{A}(t)$ of arms with selection probabilities $a$0. The expected cost is

$a$1

with the soft budget constraint

$a$2

Rewards take the form

$a$3

and gain is defined by

$a$4

where $a$5 is an indifference point for an external fallback action. The objective is to maximize cumulative expected gain $a$6 (Luedtke et al., 2016).

If the vector of means $a$7 were known, the optimal policy would solve

$a$8

This is a fractional knapsack problem. Its structure is expressed through the reward-to-cost ratios $a$9 and a threshold $c_a>0$0. Arms in

$c_a>0$1

satisfy $c_a>0$2, arms in

$c_a>0$3

satisfy $c_a>0$4 so as to fill the budget exactly, and arms in

$c_a>0$5

satisfy $c_a>0$6. The corresponding per-round maximum gain is

$c_a>0$7

3. Budgeted-KMR as a KL-UCB extension

The algorithmic idea in the bandit literature is to replace unknown means by optimistic indices and to solve, at each round, the same greedy fractional knapsack problem that the oracle would solve if means were known. For each arm $c_a>0$8, Budgeted-KMR computes an upper confidence bound

$c_a>0$9

where $B$0 is the empirical mean and the KL divergence is taken in the relevant model class. For one-parameter exponential families, including Bernoulli rewards, the specification is

$B$1

whereas for finitely supported bounded rewards it is

$B$2

(Luedtke et al., 2016).

The selection step treats $B$3 as if they were the true means. Arms are greedily sorted by $B$4, and a current threshold $B$5 is induced by the knapsack fill rule. The inclusion probabilities are then

$B$6

Each arm is drawn independently with probability $B$7. This randomization is not an auxiliary implementation detail: it is the mechanism that handles the on-margin set and ensures that the budget constraint is satisfied in expectation.

This construction differs from standard KL-UCB in two ways. First, the algorithm is a multiple-play method rather than a single-arm selection rule. Second, the decision step is not “pick the largest index”; it is “solve a greedy fractional knapsack using the current indices.” The same paper emphasizes that the hard-budget variant, in which the per-round constraint must hold deterministically, is NP-hard; the algorithm and its theory therefore target the soft constraint because of tractability.

4. Regret lower bounds, optimality, and the thick-margin regime

Regret is defined relative to the gain oracle: $B$8 With $B$9, the decomposition given in the source paper is \begin{align} \mathrm{Reg}(T,V,\mathrm{Alg}) &= \sum_{a^\star \in L} c_{a^\star} (\rho_{a^\star} - \rho^\star) \left[T - \mathbb{E}V[N{a^\star}(T)]\right] \ &\qquad + \sum_{a \in N} c_a (\rho^\star - \rho_a) \mathbb{E}V[N_a(T)] \ &\qquad + (\rho^\star - \rho)\left(BT - \sum{a=1}^K c_a \mathbb{E}_V[N_a(T)]\right). \end{align} The third term is the penalty for leaving budget unspent; it appears when the margin is “thick,” that is, when $K$0 (Luedtke et al., 2016).

The information-theoretic lower bound is expressed through

$K$1

For uniformly efficient algorithms, the lower-bound theorem states

$K$2

The asymptotic claim attached to Budgeted-KMR is strong: the KL-UCB variant and the corresponding Thompson sampling variant are asymptotically optimal, both in rate and in the leading problem-dependent constants, including the thick margin setting where multiple arms fall on the decision boundary (Luedtke et al., 2016).

The thick-margin case is algorithmically delicate because the optimal set is not unique. The source attributes optimality to the interaction between the fractional knapsack structure and explicit randomization on the margin: the algorithms induce the correct arm-pulling frequencies, saturate the budget when $K$3, and avoid unnecessary pulls of strictly suboptimal arms.

5. Relation to standard KL-UCB and Thompson sampling

Budgeted-KMR is best understood as a budgeted multi-play extension of index-based stochastic bandit algorithms. Standard KL-UCB computes an upper confidence index and chooses a single arm. Budgeted-KMR computes the same type of index per arm but uses those indices as surrogate means inside a greedy knapsack oracle. Thompson sampling admits an analogous modification: sample $K$4 from the posterior for each arm, solve the same fractional knapsack using the sampled means, randomize on the margin as needed, then pull the resulting subset (Luedtke et al., 2016).

The distinction is structural rather than cosmetic. In the budgeted setting, optimal play depends on mean-to-cost ratios and on the threshold $K$5, not merely on pairwise index comparisons. A plausible implication is that the randomization at the boundary is part of the optimal asymptotic mechanism rather than a tie-breaking convention. The same source explicitly contrasts this design with prior multiple-play algorithms whose indexing or selection mechanisms do not adapt to the margin randomization required by the budgeted setting; such algorithms need not achieve the instance-optimal regret constant.

The paper also draws a methodological lesson: proper handling of the margin is both necessary and sufficient for asymptotic optimality in budgeted multi-play bandit settings. In this formulation, the budget is not merely a feasibility constraint. It changes the oracle, the regret decomposition, the asymptotic lower bound, and the geometry of exploration.

6. Budgeted-KMR in the Knob–Meter–Rule framework

A separate usage of the same label appears in a formalism for model efficiency. There, KMR abbreviates Knob–Meter–Rule. The framework abstracts efficiency techniques into knobs $K$6, meters $K$7, and deterministic rules $K$8. Knobs are tunable hyperparameters such as prune ratio, bitwidth, rank, or architecture size; meters evaluate cost and quality; rules are deterministic model transformations based on knob values. The associated constrained problem is

$K$9

with

$a$0

(Tyagi et al., 19 Aug 2025).

The Budgeted-KMR algorithm in this framework is an iterative, policy-driven procedure. Starting from an initial model $a$1, a budget $a$2, meters $a$3, a knob set $a$4, a rule set $a$5, a policy $a$6, a dataset $a$7, and a maximum iteration count $a$8, it repeats the following loop while $a$9 and the iteration cap has not been reached: choose a knob and value $\nu_a$0; retrieve the associated rule $\nu_a$1; apply the transformation $\nu_a$2; terminate early if $\nu_a$3; otherwise accept the update, optionally fine-tune on $\nu_a$4, and continue. If the final model still violates the budget, the algorithm returns Failure; otherwise it returns $\nu_a$5 (Tyagi et al., 19 Aug 2025).

The theoretical guarantees attached to this formulation are monotonicity and termination. Each accepted iteration satisfies strict cost reduction, $\nu_a$6; the procedure cannot run for more than $\nu_a$7 accepted iterations; and the complexity bound is

$\nu_a$8

The framework also defines a composed version, “Composed Budgeted-KMR for Multiple Instantiations,” in which the knob set and rule set are unions over several efficiency methods such as pruning, quantization, knowledge distillation, and parameter-efficient architectures. The paper explicitly states that it does not provide concrete quantitative experimental results for Budgeted-KMR itself; its emphasis is formalization and algorithmic abstraction rather than benchmark reporting.

7. Conceptual significance and common misconceptions

A frequent misconception is to treat “Budgeted-KMR” as a unique named algorithm with a stable meaning across subfields. The arXiv record does not support that reading. In one line of work it is a statistically efficient bandit policy derived from KL-UCB and fractional knapsack structure; in another it is a general-purpose template for cost-constrained model transformation (Luedtke et al., 2016, Tyagi et al., 19 Aug 2025).

Another misconception is to reduce the bandit version to “KL-UCB with a budget.” That description omits its defining elements: the soft budget constraint, the gain objective $\nu_a$9, the fractional knapsack oracle, and the randomized treatment of arms on the margin. Likewise, it is inaccurate to interpret the KMR-framework version as a specific pruning or quantization algorithm. The paper presents it as a universal, iterative procedure whose behavior depends on the chosen policy, knobs, meters, rules, and optional fine-tuning.

Taken together, the two usages show how the same label can denote either a specific asymptotically optimal policy in stochastic online learning or a method-agnostic scaffold for budgeted model optimization. This suggests that any technical discussion of “Budgeted-KMR” should specify the ambient literature and the precise optimization problem before invoking algorithmic properties.