Combined Auction-Bandit Model

• Combined auction-bandit models are repeated market formulations that integrate auction mechanisms with online learning to optimize allocation and payments based on partial feedback.
• They employ methodologies ranging from multiplicative weights and contextual bandits to reinforcement learning in order to navigate strategic interactions under censored and side-observation feedback.
• The models highlight trade-offs between exploration, regret minimization, and incentive compatibility, impacting applications in procurement, advertising, and electricity markets.

The combined auction-bandit model denotes, in the literature surveyed here, a class of repeated-market formulations in which an auction or procurement mechanism determines allocation, payment, and strategic interdependence, while online learning or bandit feedback determines how bidders, sellers, platforms, or both adapt from partial observations over time. In these models, bids, bid functions, or allocation rules are chosen sequentially; utilities, welfare, or revenue are benchmarked against fixed-action or full-information comparators; and the feedback is typically neither full-information nor ordinary one-arm bandit feedback, because the auction itself reveals structured but incomplete information about counterfactual outcomes (Karaca et al., 2019, Weed et al., 2015, Basu et al., 2022).

1. Scope and conceptual structure

A combined auction-bandit model is not a single formalism but a family of closely related repeated-interaction models. One broad class studies a bidder or seller as the learner inside a repeated auction. Another studies the auctioneer or platform as the learner, often with strategic agents on the other side. A third studies two-sided market learning, where both allocation and pricing evolve under partial feedback. Taken together, these works place such models between auction theory, online learning, contextual bandits, partial monitoring, and mechanism design (Karaca et al., 2019, Chen et al., 7 Jul 2026).

Paradigm	Auction environment	Learning object
Bidder-side repeated bidding	Vickrey, multi-unit, procurement, sponsored search	Bid, bid vector, or bid function
Platform-side mechanism learning	Sponsored search, procurement, posted pricing	Allocation, score, reserve, or payment policy
Two-sided market learning	Double auctions, matching-like markets, simultaneous auctions	Joint selection, price discovery, or budget split

In bidder-side models, the learner repeatedly chooses a bid or strategy and receives utility generated by the auction outcome. In mechanism-side models, the platform repeatedly chooses whom to allocate to, or how to score and price agents, while learning latent quality, value, or conversion parameters. In two-sided models, the allocation rule and the data-collection process are jointly endogenous: a trade, impression, or procurement decision both generates current payoff and determines which latent variables become observable (Basu et al., 2022, Bachoc et al., 7 Aug 2025).

Several papers make the hybrid structure explicit. “No-Regret Learning from Partially Observed Data in Repeated Auctions” (Karaca et al., 2019) formulates repeated procurement auctions as multi-agent online learning with auction-specific side observations. “Characterizing Truthful Multi-Armed Bandit Mechanisms” (0812.2291) treats sponsored-search allocation with unknown click-through rates as a strategic version of the multi-armed bandit problem. “Contextual Procurement Auctions with Bandit Learning” (Chen et al., 7 Jul 2026) casts procurement with private costs and unknown context-dependent values as a contextual bandit with strategic arms. “Double Auctions with Two-sided Bandit Feedback” (Basu et al., 2022) extends the picture to both buyers and sellers learning unknown valuations in a repeated double auction.

2. Formal repeated-auction formulations

A canonical repeated-auction formulation appears in procurement markets. In (Karaca et al., 2019), bidder $\ell$ has a finite strategy set $\mathcal K_\ell$, the auction repeats over $T$ rounds, and each round computes an allocation $x_\ell^*(\mathcal B(t))$ and payment $p_\ell(\mathcal B(t))$ from the submitted bid functions. Utility is

$$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$

For a fixed bidder, regret is defined against the best fixed strategy in hindsight: $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$ This formulation already exhibits the essential combined structure: the action is a bid strategy, utility depends on market clearing against strategic opponents, and the learning benchmark is adversarial regret rather than static equilibrium computation (Karaca et al., 2019).

A second canonical model is repeated second-price bidding with endogenous censoring. In (Weed et al., 2015), at round $t$ the learner submits bid $b_t$, faces highest competing bid $m_t$, and has private value $\mathcal K_\ell$0. The net utility is

$\mathcal K_\ell$1

If the learner wins, the threshold $\mathcal K_\ell$2 and a possibly noisy measurement of value are observed; if the learner loses, $\mathcal K_\ell$3 is still observed but $\mathcal K_\ell$4 is not. The resulting exploration problem is auction-specific: bidding higher both increases the probability of winning and increases the probability of observing one’s own value (Weed et al., 2015). “Efficient Algorithms for Stochastic Repeated Second-price Auctions” (Achddou et al., 2020) studies the same structural problem in a stochastic setting with i.i.d. values $\mathcal K_\ell$5, market prices $\mathcal K_\ell$6, and regret defined relative to the optimal truthful bid $\mathcal K_\ell$7.

Multi-unit auction models generalize this structure from scalar bids to bid schedules. In (Li et al., 27 Jun 2026), the learner chooses a nonincreasing $\mathcal K_\ell$8-dimensional bid vector $\mathcal K_\ell$9, competes in discriminatory-price and uniform-price auctions, and receives utility

$T$0

The action is therefore structured and continuous rather than a finite arm. The paper explicitly treats repeated multi-unit bidding as sequential decision-making with bandit feedback, then broadens it into an actor-critic reinforcement-learning formulation with opponent modeling and a composite reward (Li et al., 27 Jun 2026).

Mechanism-side formulations reverse the viewpoint. In (Chen et al., 7 Jul 2026), a procurement platform observes context $T$1, selects one producer $T$2 or the outside option, pays the selected producer, and observes only $T$3, where

$T$4

Producer $T$5 has private cost $T$6, submits a fixed ask bid $T$7, and the efficient procurement score is

$T$8

Regret is welfare loss relative to the full-information efficient allocation. This is a contextual procurement auction in which the platform must both learn $T$9 from bandit feedback and elicit strategic costs (Chen et al., 7 Jul 2026).

3. Feedback structures and partial monitoring

The defining technical feature of combined auction-bandit models is the feedback structure. In standard full-information learning, the learner observes the whole counterfactual payoff vector. In a standard bandit, only the realized payoff of the chosen action is observed. Auction environments often lie strictly between these two regimes (Karaca et al., 2019).

The intermediate case is particularly explicit in repeated procurement auctions. In (Karaca et al., 2019), when bidder $x_\ell^*(\mathcal B(t))$0 loses, utility is exactly zero, and the bidder may infer a subset $x_\ell^*(\mathcal B(t))$1 of other actions that would also have lost. For those actions, the loss is known exactly. The paper defines revelation probabilities $x_\ell^*(\mathcal B(t))$2 and an unbiased loss estimator that exploits these side observations. It further defines

$x_\ell^*(\mathcal B(t))$3

so that $x_\ell^*(\mathcal B(t))$4 corresponds to bandit feedback and $x_\ell^*(\mathcal B(t))$5 to full information. The regret bound

$x_\ell^*(\mathcal B(t))$6

shows that auction-generated revelation probabilities directly interpolate between bandit and full-information regimes (Karaca et al., 2019).

Second-price auctions exhibit a different partial-monitoring structure. In (Weed et al., 2015), every round reveals the market threshold $x_\ell^*(\mathcal B(t))$7, but the bidder’s own value is revealed only when the auction is won. This creates one-sided or censored feedback: all bids below $x_\ell^*(\mathcal B(t))$8 are known to lose, while bids above $x_\ell^*(\mathcal B(t))$9 would have won but differ in payment exposure. The paper’s ExpTree estimator exploits this threshold structure to infer gains for all bids on one side of $p_\ell(\mathcal B(t))$0 from a single observation (Weed et al., 2015). In (Achddou et al., 2020), the same second-price environment is described as action-dependent censored feedback, because the number of observed value samples is

$p_\ell(\mathcal B(t))$1

so the bid simultaneously affects payoff and observation rate.

Multi-unit auctions again enlarge the feedback geometry. In (Li et al., 27 Jun 2026), the observed allocation signal is

$p_\ell(\mathcal B(t))$2

and in the uniform-price case the bidder additionally observes

$p_\ell(\mathcal B(t))$3

This means the payment itself can reveal some components of the rival bid vector. The paper therefore distinguishes discriminatory auctions, where payment reveals essentially no new information beyond win/loss, from uniform-price auctions, where payment can reveal partial opponent information (Li et al., 27 Jun 2026).

Other models intensify the partial-information constraint. In sequential posted pricing, the seller posts a vector of prices but observes only realized revenue, not buyers’ values and not the revenue of alternative prices (Singla et al., 2023). In practical sponsored-search bidding with unknown valuation models, the auction is treated as a black box and feedback is delayed and batched, so the learner receives only aggregate value, aggregate payment, and click-through rate after a delay (Provodin et al., 2023). These formulations are still auction-bandit models, but they sit closer to adversarial bandits or partial monitoring than to side-observation bandits.

A common misconception is that richer-than-bandit auction feedback is equivalent to full information. The literature does not support that view. Even when losing reveals counterfactual losses, winning often does not reveal exact counterfactual utilities because payments can change endogenously with the bid, and many practically important models retain censoring, thresholding, or aggregation rather than full utility vectors (Karaca et al., 2019, Weed et al., 2015).

4. Incentives, truthfulness, and equilibrium implications

A second defining theme is the interaction between learning and incentive compatibility. Some combined auction-bandit models take the bidder’s perspective and impose no truthfulness requirement on the mechanism. Others place the mechanism designer at the center and study which learning rules remain truthful or approximately truthful under strategic reporting.

The canonical impossibility result is in sponsored-search MAB mechanisms. “Characterizing Truthful Multi-Armed Bandit Mechanisms” (0812.2291) shows that, under observable-click payment computation, deterministic truthful mechanisms must satisfy strong structural constraints. In the two-agent case, normalized truthfulness is equivalent to pointwise monotonicity and exploration-separated allocation. Exploration-separated means that influential rounds are bid-independent. The paper also proves that truthful deterministic mechanisms incur much worse regret than ordinary stochastic MAB algorithms: the lower bound is

$p_\ell(\mathcal B(t))$4

while a truthful naive mechanism achieves

$p_\ell(\mathcal B(t))$5

This is the foundational statement that truthful learning and efficient adaptive exploration are in tension (0812.2291).

The multi-slot extension sharpens the same point. “Multi-Armed Bandit Mechanisms for Multi-Slot Sponsored Search Auctions” (Sarma et al., 2010) proves that in the unknown, unconstrained multi-slot CTR model, a deterministic DSIC mechanism exists if and only if the allocation rule is strongly pointwise monotone and weakly separated, with the usual envelope payment formula. The consequence is severe: in the unrestricted setting, truthful mechanisms suffer linear worst-case regret $p_\ell(\mathcal B(t))$6. Under stronger structure, such as separable CTRs, the paper reports experimental evidence of $p_\ell(\mathcal B(t))$7 regret for a simple truthful mechanism, but the broad unrestricted model remains highly rigid (Sarma et al., 2010).

Procurement settings reveal the same trade-off in reverse auctions. In (Chen et al., 7 Jul 2026), an exactly truthful explore-then-commit mechanism uses a bid-independent exploration phase of length

$p_\ell(\mathcal B(t))$8

and achieves

$p_\ell(\mathcal B(t))$9

welfare regret. A frozen-payment UCB mechanism keeps adaptive UCB allocation but freezes payment estimates after an initial exploration phase. It then attains a regret-incentive tradeoff: near-UCB tuning gives $$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$0 welfare regret and $$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$1 total incentive error for fixed $$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$2, while balanced tuning gives $$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$3 on both scales. The paper also proves a matching lower bound within the frozen-payment framework (Chen et al., 7 Jul 2026).

Reverse-auction contextual bandits extend truthfulness to adaptive model selection. In (Patra et al., 16 Feb 2026), the mechanism defines provider utility

$$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$4

uses reverse-auction virtual costs

$$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$5

and combines a monotone contextual bandit allocation rule with reverse self-resampling. The resulting mechanism is EPIC and EPIR, and the contextual bandit component learns with sublinear regret. Here the combined model is neither bidder-side learning nor classical truthful MAB in forward auctions, but truthful reverse procurement with contextual quality learning (Patra et al., 16 Feb 2026).

By contrast, bidder-side repeated-auction learning usually studies equilibrium implications through regret rather than truthfulness. In (Karaca et al., 2019), if all bidders use no-regret learning, the empirical distribution of play converges to a coarse-correlated equilibrium of the one-shot auction game. That convergence, however, does not ensure efficiency; the Swiss reserve-procurement case study in the same paper reports that truthful bidding yields lower social cost than the learned outcomes under pay-as-bid. A common misconception is therefore that no-regret learning implies efficient market outcomes. The literature supports convergence to CCE, not a general efficiency guarantee (Karaca et al., 2019).

5. Algorithmic families and performance regimes

Algorithmically, combined auction-bandit models range from finite-action multiplicative weights to contextual linear UCB, continuous-action interval methods, and reinforcement-learning architectures.

Auction-aware multiplicative weights appears in (Karaca et al., 2019). The bidder maintains $$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$6, samples an action, forms the auction-specific unbiased loss estimator using revelation probabilities, and updates

$$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$7

This is best understood as an extended Exp3 or auction-aware MWU: the update rule is standard, but the estimator exploits losing-side observations unavailable to ordinary bandits (Karaca et al., 2019).

Second-price bidder learning has produced both stochastic and adversarial algorithms. In (Weed et al., 2015), UCBid bids

$$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$8

and achieves logarithmic or $$u_\ell(t)=p_\ell(\mathcal B(t))-c_\ell(x_\ell^*(\mathcal B(t))).$$9-type pseudo-regret depending on the margin condition. The same paper develops ExpTree and ExpTree.P for adversarial settings with continuous bids and obtains $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$0-type regret, together with matching lower bounds (Weed et al., 2015). In (Achddou et al., 2020), the stochastic repeated second-price setting yields UCBID, kl-UCBID, and Bernstein-UCBID. Under bounded-density conditions, kl-UCBID attains worst-case

$$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$1

while ETG-style algorithms are shown to be fundamentally inferior in the minimax sense, with $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$2-type worst-case behavior (Achddou et al., 2020).

Mechanism-side bandit algorithms reveal a different rate structure. In truthful sponsored-search MAB mechanisms, deterministic truthfulness leads to the characteristic $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$3 frontier rather than the ordinary $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$4 frontier (0812.2291). Multi-slot truthfulness without strong structural assumptions can force linear regret $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$5 (Sarma et al., 2010). Bidimensional procurement with unknown qualities, as in (Bhat et al., 2015), yields 2D-OPT when qualities are known and 2D-UCB when qualities must be learned; the main formal guarantee for 2D-UCB is stochastic BIC and IR rather than a regret theorem (Bhat et al., 2015).

Contextual and stateful models push beyond classical bandits. In (Patra et al., 16 Feb 2026), the reverse-auction contextual mechanism TRCM-UCBOPT combines a monotone contextual UCB allocation rule with a truthful reverse-auction transformation and obtains $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$6-type regret, more explicitly $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$7. In (Li et al., 27 Jun 2026), A3M moves beyond finite-armed bandits to an actor-critic DRL backbone with opponent modeling and multi-objective reward design. Its empirical evaluation reports that A3M reduces final regret by $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$8 in standard settings, remains robust against adversarial strategy shifts, and scales favorably with the number of units $$R_\ell(T)=\max_{k\in\mathcal K_\ell}\sum_{t=1}^T u_\ell(\{\mathcal B_{-\ell}(t),b_\ell^k\})-\sum_{t=1}^T u_\ell(t).$$9, but the paper is explicit that A3M itself does not come with formal regret guarantees (Li et al., 27 Jun 2026).

Platform-side ranking problems in advertising and posted pricing illustrate yet another regime. “Bandit Sequential Posted Pricing via Half-Concavity” (Singla et al., 2023) proves $t$0 regret for regular distributions and $t$1 for general distributions in sequential posted pricing with revenue-only feedback. “Optimizing Online Advertising with Multi-Armed Bandits: Mitigating the Cold Start Problem under Auction Dynamics” (Soboleva et al., 3 Feb 2025) studies a PBM ranking problem with known per-click prices and unknown CTRs, ranks ads by $t$2, and proves a logarithmic instance-dependent regret bound for AuctionUCB-PBM. These are auction-aware ranking bandits rather than strategic mechanism-design models, but they still belong to the combined family because the reward is auction revenue and the feedback is generated by auction exposure (Singla et al., 2023, Soboleva et al., 3 Feb 2025).

6. Applications, empirical findings, and persistent tensions

Electricity markets are a recurring application because they produce repeated, optimization-based auctions with structured public outputs. In (Karaca et al., 2019), the extended Exp3 method is evaluated on a simple single-good electricity procurement market, an IEEE 14-bus optimal power flow market, and a Swiss reserve-procurement market. The results show that exploiting auction-specific losing-side observations can make performance approach full-information Hedge in settings where $t$3 is close to $t$4 (Karaca et al., 2019).

Sponsored-search and app-store advertising provide the most prominent empirical combined auction-bandit settings. In (Rashid et al., 28 Aug 2025), the platform runs a single-slot pay-per-install second-price auction with Thompson Sampling quality scores. The paper shows that the exploration policy that maximizes allocative efficiency can be far below the exploration policy that maximizes revenue: under a uniform entrant prior, the efficiency-maximizing prior mean is $t$5, while the revenue-maximizing prior mean is $t$6. The revenue from the revenue-maximizing prior is about $t$7 higher than the revenue from the efficiency-maximizing prior. This makes explicit that, in auction environments, exploration changes not only learning and allocation but also competition and prices (Rashid et al., 28 Aug 2025).

Practical ad-auction modeling under partial feedback yields a more mechanism-agnostic view. “Advancing Ad Auction Realism: Practical Insights & Modeling Implications” (Chen et al., 2023) models advertisers as Hedge or EXP3-IX learners over bids and targeting clauses in repeated auctions with query-dependent values and CTRs, unknown competitors, partial feedback, and partially known payment rules. The paper finds that soft floors can improve revenues in multi-query environments even when bidder types are drawn from the same distribution, but can yield lower revenues than suitably chosen reserve prices in asymmetric single-query environments. “Bandits for Sponsored Search Auctions under Unknown Valuation Model” (Provodin et al., 2023) goes further toward black-box robustness by treating the auction mechanism as opaque and using BatchEXP3 with delayed and batched feedback in production (Chen et al., 2023, Provodin et al., 2023).

Two-sided and procurement markets expose the same design tensions in different form. “Double Auctions with Two-sided Bandit Feedback” (Basu et al., 2022) proves $t$8 social regret and $t$9-type individual regret for trading agents under confidence-bound bidding and Average Pricing, while showing that $b_t$0 individual regret and $b_t$1 social regret are unattainable in certain markets. “Stochastic Bandits for Crowdsourcing and Multi-Platform Autobidding” (Bachoc et al., 7 Aug 2025) studies simplex-valued budget allocation across $b_t$2 simultaneous tasks or auctions and proves minimax-optimal $b_t$3 regret, improved to $b_t$4 under diminishing-returns conditions. Both papers illustrate that combined auction-bandit models are not restricted to single-winner auctions or bidder-side learning; they also encompass market-wide price discovery and simultaneous budget allocation (Basu et al., 2022, Bachoc et al., 7 Aug 2025).

Several persistent tensions run through the literature. Exact truthfulness often requires bid-independent exploration, frozen estimates, or exploration-separated dynamics, and these restrictions generally worsen achievable regret (0812.2291, Chen et al., 7 Jul 2026). Richer auction feedback can shrink variance dramatically, but it rarely collapses to full information (Karaca et al., 2019). Empirically strong RL frameworks can exploit nonstationarity and multi-objective criteria, yet may lack theorem-level regret guarantees (Li et al., 27 Jun 2026). Revenue-optimal exploration in ad auctions can differ sharply from efficiency-optimal exploration because exploration changes prices as well as learning (Rashid et al., 28 Aug 2025). A plausible implication is that the combined auction-bandit model is best understood not as a single theorem schema, but as a research program on repeated strategic allocation under endogenous partial information, where the central technical problem is to align learning, incentives, and market objectives without assuming away the information constraints created by the auction itself.