Bayesian Combinatorial Selection

• Bayesian Combinatorial Selection is a framework that integrates Bayesian uncertainty quantification with combinatorial optimization for selecting optimal discrete structures under complex constraints.
• It spans various applications such as Bayesian optimization over binary or categorical spaces, sequential data acquisition, ranking and selection, auction design, and preference elicitation.
• Efficient algorithms—including semidefinite programming, graph-cut relaxations, and continuous embedding methods—address the NP-hard selection problem while ensuring robust performance.

to=arxiv_search.search 彩神争霸是 彩神争霸破解? to=arxiv_search.search 久久免费热在线精品{"query":"Bayesian combinatorial selection arXiv combinatorial Bayesian optimization matroid persuasion ranking selection", "max_results": 10} to=arxiv_search.search 天天彩票网{"query":"Bayesian combinatorial selection arXiv", "max_results": 5} to=arxiv_search иҟазുന്നു{"query":"Bayesian combinatorial selection", "max_results": 5} Bayesian combinatorial selection denotes a family of problems in which posterior uncertainty, or a posterior-derived acquisition rule, is used to choose a discrete structured object from an exponentially large feasible set. The selected object may be a subset, a binary design, a categorical assignment, a path, a tree topology, an allocation, or a recommended action under combinatorial constraints. Across Bayesian optimization, sequential data acquisition, ranking and selection, preference elicitation, and auction design, the common structure is a Bayesian model over latent values or parameters together with a combinatorial decision layer that must optimize a posterior expected utility, a posterior safety criterion, or a sampled surrogate objective (Baptista et al., 2018, Deshwal et al., 2020, Fujii et al., 2021).

1. Problem classes and action spaces

A first axis of variation is the action space itself. In Bayesian optimization over combinatorial structures, the domain is often written as a binary or categorical configuration space. BOCS takes $\mathcal{X}=\{0,1\}^n$ and treats each $\mathbf{x}$ as a combinatorial structure whose expensive objective value must be learned and optimized sequentially (Baptista et al., 2018). PSR-based work extends this view to sets, sequences, trees, and graphs represented by discrete variables, with acquisition optimization itself becoming a combinatorial problem (Deshwal et al., 2020). COMBO represents the search space as the vertex set of a graph Cartesian product over per-variable subgraphs, thereby covering binary, categorical, ordinal, and mixed discrete spaces (Oh et al., 2019).

A second class consists of Bayesian data-selection problems in which the action is the choice of data to add. In the decision-theoretic formulation of pseudo-label selection, the action space is the current unlabeled pool, the state space is the parameter space $\Theta\subseteq\mathbb{R}^q$, and the utility of selecting $(z,\mathcal Y)$ is the pseudo-label likelihood

$$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$

The resulting Bayes criterion is the pseudo posterior predictive

$p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$

and the implemented procedure is explicitly sequential one-point selection rather than joint subset optimization (Rodemann, 2024).

A third class is finite-set ranking and selection under noisy estimates. In Bayesian ranking and selection, there are $K$ candidates with latent values $\mu_1,\dots,\mu_K$, observed through $Y\sim\mathcal N(\mu,\Sigma)$, and the target action is a subset $\hat S\subseteq\{1,\dots,K\}$ intended to recover the true top-$\mathbf{x}$0 set $\mathbf{x}$1 (Bowen, 2022). This is combinatorial in the restricted sense that the decision is a subset of a finite ground set, but the paper explicitly does not study knapsacks, matroids, matchings, or other richer feasibility structures (Bowen, 2022).

A fourth class arises when the feasible actions are combinatorially constrained by design. In algorithmic Bayesian persuasion with combinatorial actions, the receiver chooses $\mathbf{x}$2, where $\mathbf{x}$3 may be a matroid family or the set of $\mathbf{x}$4-$\mathbf{x}$5 paths in a graph, and the sender chooses a signaling scheme that induces posteriors under which the recommended combinatorial action is optimal for the receiver (Fujii et al., 2021). In combinatorial auctions and assignment, the relevant action is an allocation $\mathbf{x}$6 with feasibility

$\mathbf{x}$7

and the mechanism’s selection problem is to choose prices, queries, or bundle recommendations that move the market toward a high-welfare feasible allocation (Weissteiner et al., 2022, Brero et al., 2017).

The literature also contains sequential inference problems in which the latent object is itself combinatorial and the goal is to maintain a posterior over structures rather than select a single subset. Variational combinatorial sequential Monte Carlo for phylogenetic inference treats a tree topology as a sequentially constructed combinatorial object and learns distributions over partial and complete structures through particle methods and variational objectives (Moretti et al., 2021). A plausible implication is that Bayesian combinatorial selection includes both direct action selection and posterior inference over structured latent spaces, provided the decision layer is combinatorial.

2. Bayesian objectives and probabilistic formulations

The decision-theoretic formulation of Bayesian data selection provides one of the clearest generic templates. A canonical decision problem is given by $\mathbf{x}$8, with action space $\mathbf{x}$9, state space $\Theta\subseteq\mathbb{R}^q$0, and utility $\Theta\subseteq\mathbb{R}^q$1. Under posterior $\Theta\subseteq\mathbb{R}^q$2, the Bayes criterion is

$\Theta\subseteq\mathbb{R}^q$3

For pseudo-label selection, this reduces to the posterior predictive score

$\Theta\subseteq\mathbb{R}^q$4

and the approximate Bayes-optimal criterion is

$\Theta\subseteq\mathbb{R}^q$5

The paper’s central claim is that this posterior-averaged, dataset-level criterion is more robust than confidence heuristics and mitigates confirmation bias in self-training (Rodemann, 2024).

In Bayesian ranking and subset selection, the primitive uncertainty is over latent candidate qualities rather than over black-box functions. The posterior over $\Theta\subseteq\mathbb{R}^q$6 induces posterior probabilities of rank events such as $\Theta\subseteq\mathbb{R}^q$7 and $\Theta\subseteq\mathbb{R}^q$8. This supports both interval-based and direct selection rules. The direct Bayesian FDR method greedily adds candidates in descending order of $\Theta\subseteq\mathbb{R}^q$9 while maintaining an estimated posterior false discovery rate constraint, whereas the Bayesian FWER method greedily removes likely non-top candidates until the posterior probability that all remaining items are truly top-$(z,\mathcal Y)$0 is sufficiently high (Bowen, 2022).

In Bayesian optimization over binary combinatorial spaces, the dominant formulation is posterior modeling of the unknown objective. BOCS uses the second-order surrogate

$(z,\mathcal Y)$1

with a sparsity-inducing horseshoe prior on $(z,\mathcal Y)$2. Thompson sampling then draws $(z,\mathcal Y)$3 from the posterior and selects the next structure by solving the sampled acquisition problem

$(z,\mathcal Y)$4

possibly with a regularization term such as $(z,\mathcal Y)$5 (Baptista et al., 2018, Deshwal et al., 2020).

Graph-based Gaussian-process formulations replace low-order regression with kernels on combinatorial graphs. COMBO defines a GP prior on the graph Cartesian product of per-variable graphs and uses an ARD diffusion kernel

$(z,\mathcal Y)$6

This permits high-order interactions between variables without fixing an interaction order in advance, while a Horseshoe prior on the $(z,\mathcal Y)$7 acts as an explicit Bayesian variable-selection mechanism (Oh et al., 2019).

Older Bayesian-network-based estimation-of-distribution algorithms supply a different probabilistic viewpoint. There, the selected promising individuals form a dataset over variables $(z,\mathcal Y)$8, and a Bayesian network factorization

$(z,\mathcal Y)$9

is learned from those elites and then sampled to generate new candidate solutions (Larrañaga et al., 2013). This is Bayesian in model structure, but the resulting procedure is best understood as repeated learning and simulation of a probability model over high-fitness combinatorial solutions rather than posterior expected-utility maximization.

3. Optimization mechanisms for the combinatorial selection step

Once a Bayesian objective has been defined, the central difficulty is the selection step itself. In BOCS, optimizing the sampled quadratic surrogate is a binary quadratic program. The paper’s central algorithmic idea is a semidefinite relaxation: after mapping $$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$0 to a $$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$1 representation, the acquisition problem is relaxed to

$$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$2

followed by factorization and randomized rounding. This is the basis of BOCS-SDP; BOCS-SA replaces the SDP with simulated annealing on the sampled acquisition (Baptista et al., 2018).

PSR addresses the same acquisition bottleneck from a submodular-relaxation perspective. The quadratic objective is decomposed into positive and non-positive parts,

$$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$3

with $$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$4 submodular and $$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$5 handled by an affine lower bound parameterized by $$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$6. The relaxed problem

$$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$7

is solvable by minimum $$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$8-$$u((z,\mathcal{Y}), \theta) = p(\mathcal{D} \cup (z, \hat{y}(z))\mid \theta, M).$$9 cut, while proximal gradient descent updates $p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$0 to tighten the relaxation (Deshwal et al., 2020). This separates the Bayesian part, which still comes from BOCS and Thompson sampling, from the combinatorial solver, which is replaced by graph-cut machinery.

MerCBO attacks the same issue by making COMBO-like diffusion kernels explicit in feature space. For the hypercube graph, the diffusion kernel admits explicit Mercer features

$p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$1

and truncating to second-order features yields a Bayesian linear model whose Thompson-sampled objective becomes a binary quadratic form

$p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$2

That BQP is then optimized by submodular relaxation and graph cuts, giving a tractable acquisition optimizer while retaining a diffusion-kernel surrogate (Deshwal et al., 2020).

Alternative work avoids direct optimization in the original combinatorial space by embedding the domain into a continuous one. Random mapping to convex polytopes first encodes each $p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$3 as a Boolean vector $p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$4, then maps it to $p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$5, and performs Gaussian-process Bayesian optimization on the convex hull

$p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$6

The practical CBO-Lookup variant then decodes a continuous query by nearest lookup in the embedded table (Kim et al., 2020). This suggests a distinct algorithmic philosophy: rather than solve a hard discrete acquisition problem exactly, one may solve a continuous relaxation and decode back to a valid combinatorial object.

COMBO, by contrast, stays in the combinatorial graph and optimizes expected improvement by a graph-local search strategy. The procedure evaluates the acquisition on a large random exploration set, augments it with “spray” vertices near the current best point, chooses high-value seeds, and performs breadth-first local search over adjacent graph vertices until no improving move exists (Oh et al., 2019). Its efficiency depends on the graph Cartesian product structure, which reduces Graph Fourier Transform computation from exponential to additive complexity across variable-level subgraphs (Oh et al., 2019).

Earlier Bayesian-network EDAs solve the selection step by repeated probabilistic simulation. EBNAPC, EBNABIC, and EBNAK2+pen all maintain a population, select the best $p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$7 individuals, learn a Bayesian network over them, and generate new solutions by Probabilistic Logic Sampling in ancestral order (Larrañaga et al., 2013). The optimization step is therefore indirect: one learns a distribution over promising configurations and samples from it, rather than solving a posterior acquisition maximization problem.

4. Representative instantiations across domains

In semi-supervised learning, Bayesian selection appears as pseudo-label acquisition. The decision-theoretic formulation in self-training repeatedly scores each unlabeled candidate by the pseudo posterior predictive of the enlarged dataset and selects the maximizer

$p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$8

The same approximate criterion is reported for generalized linear models, semi-parametric generalized additive models, and Bayesian neural networks, with the principal empirical message being mitigation of confirmation bias relative to confidence-based pseudo-labeling (Rodemann, 2024).

In finite-population ranking and selection, the action is the extraction of a high-confidence top set from noisy scores. The paper develops Bayesian marginal and simultaneous rank confidence intervals, as well as direct Bayesian FDR and FWER methods for selecting candidates believed to be among the top $p(\mathcal D \cup (x_i,\hat y_i)\mid \mathcal D),$9 (Bowen, 2022). Empirically, the Bayesian procedures are reported to select more candidates while maintaining correct error rates on average, and the ranking intervals are shorter than frequentist alternatives, although simultaneous coverage is only approximately correct in some settings (Bowen, 2022).

In combinatorial assignment and auctions, Bayesian combinatorial selection becomes a mechanism-design problem. BOCA models bidder-specific upper uncertainty bounds $K$0 over unelicited bundle values and defines the acquisition

$K$1

then solves

$K$2

as a winner-determination problem to decide which bundles to query next (Weissteiner et al., 2022). The mechanism is evaluated on LSVM, SRVM, and MRVM spectrum auction domains, where it is reported to achieve higher allocative efficiency than prior ML-based elicitation approaches (Weissteiner et al., 2022).

Related auction work casts clearing-price computation itself as Bayesian inference. In the Bayesian clearing mechanism, prices $K$3 and values $K$4 are coupled through a clearing potential

$K$5

so that clearing prices are MAP estimates under a generative model over valuations and prices (Brero et al., 2017). The later general Bayesian iterative combinatorial auction uses Monte Carlo expectation maximization to choose the next price vector by maximizing the likelihood of near-clearing prices under the current posterior over bidder valuations (Brero et al., 2018). Both papers are Bayesian combinatorial selection in the sense that the mechanism repeatedly selects prices in order to induce a feasible welfare-maximizing allocation.

Preference elicitation provides another variant. In multiobjective combinatorial optimization, the latent parameter is a weight vector $K$6 in a scalarization

$K$7

and the selected object is the current recommendation $K$8 minimizing max expected regret. Pairwise comparison queries update the posterior over $K$9, while mixed-integer optimization identifies both the current minimax-expected-regret recommendation and its best challenger (Bourdache et al., 2020). Here the Bayesian component is over preferences, not over objective values, but the selected action is still a combinatorial solution.

Variable selection, sparse structure learning, and phylogenetic inference broaden the scope further. Thompson Variable Selection turns subset selection into a multiple-play Thompson sampling problem with arm-specific Beta posteriors and an oracle set

$\mu_1,\dots,\mu_K$0

linking combinatorial binary bandits to spike-and-slab logic and the median probability model (Liu et al., 2020). The $\mu_1,\dots,\mu_K$1-ball projection prior instead creates exact zeros by projecting a continuous latent variable onto an $\mu_1,\dots,\mu_K$2-type ball, thereby supporting posterior inference on supports, change-points, mixture dimensions, and ranks (Xu et al., 2020). VCSMC and VNCSMC take a different route: they learn proposal distributions over sequentially built tree structures and thereby perform variational Bayesian inference over a combinatorial latent space (Moretti et al., 2021). This suggests that Bayesian combinatorial selection includes both explicit action selection and distribution learning over combinatorial structures when the latter is the inferential target.

5. Guarantees, hardness, and computational tradeoffs

Theoretical results in this area are heterogeneous. Some are positive performance guarantees for specific algorithms. COMBO’s central complexity claim is that if $\mu_1,\dots,\mu_K$3, then Graph Fourier Transform computation scales as

$\mu_1,\dots,\mu_K$4

rather than

$\mu_1,\dots,\mu_K$5

which is what makes graph-based Gaussian processes feasible on exponentially large discrete spaces (Oh et al., 2019). Random mapping to convex polytopes supplies a high-probability cumulative regret bound

$\mu_1,\dots,\mu_K$6

where the distortion $\mu_1,\dots,\mu_K$7 induced by the random embedding appears explicitly in the regret term (Kim et al., 2020). VCSMC proves that nested CSMC particles and weights are properly weighted for the target increment, so nested CSMC is an exact approximation to the locally optimal proposal in the relevant SMC sense (Moretti et al., 2021). Thompson Variable Selection derives logarithmic-in-$\mu_1,\dots,\mu_K$8 regret bounds and an almost-sure selection consistency result under its identifiability assumptions (Liu et al., 2020).

Other guarantees are statistical rather than optimization-theoretic. Bayesian ranking and selection reports approximately correct marginal coverage for marginal rank intervals, approximately correct simultaneous coverage for simultaneous rank intervals, and empirical FDR/FWER control for the direct Bayesian selection rules, but it also explicitly notes that exact simultaneous coverage fails in some datasets and that the direct Bayesian methods are supported mainly by simulation evidence rather than full finite-sample theorems (Bowen, 2022). The $\mu_1,\dots,\mu_K$9-ball prior proves sparse linear-regression posterior concentration and a Bernstein–von Mises theorem on the true support under its stated conditions, thereby connecting exact-zero Bayesian priors to classical sparse minimax rates (Xu et al., 2020).

A contrasting strand gives hardness results. Algorithmic Bayesian persuasion with combinatorial actions shows that for any constant $Y\sim\mathcal N(\mu,\Sigma)$0, constant-factor approximation is NP-hard for partition matroid, uniform matroid, and graphic matroid constraints, and for any constant $Y\sim\mathcal N(\mu,\Sigma)$1, constant-factor approximation is NP-hard for path constraints (Fujii et al., 2021). The same paper also gives an exact polynomial-time algorithm for general matroids when the number of states of nature is constant, by reducing the action set to posterior-relevant independent sets and enumerating cells of a hyperplane arrangement in posterior space (Fujii et al., 2021). This juxtaposition is important: Bayesian combinatorial selection can be tractable when the posterior geometry and the feasible family align, but it can also inherit strong inapproximability from the underlying structured action space.

The computational tradeoffs are correspondingly central. In BN-based EDAs, the bottleneck is Bayesian-network structure learning, which is NP-hard in general and motivates greedy search, local search, and parent-count bounds (Larrañaga et al., 2013). In BOCS, the bottleneck is acquisition optimization via SDP, which PSR was designed to replace with graph-cut-based submodular relaxation because standard SDP complexity is cited as $Y\sim\mathcal N(\mu,\Sigma)$2 whereas graph-cut methods are cited as roughly $Y\sim\mathcal N(\mu,\Sigma)$3 or $Y\sim\mathcal N(\mu,\Sigma)$4 (Deshwal et al., 2020). Categorical variables often worsen the problem because one-hot encoding inflates the number of binary variables, a difficulty emphasized both by PSR and by continuous-relaxation alternatives such as BVO (Deshwal et al., 2020, Wu et al., 2020). A plausible implication is that the field’s central tension is not only statistical expressiveness versus data efficiency, but also surrogate quality versus the tractability of the induced combinatorial optimizer.

6. Scope, misconceptions, and open directions

A common misconception is that Bayesian combinatorial selection names a single canonical method. The literature instead spans several non-equivalent paradigms: posterior expected-utility maximization over candidate data points (Rodemann, 2024), posterior-safe extraction of top-ranked subsets (Bowen, 2022), Bayesian optimization with sampled surrogates over binary or categorical domains (Baptista et al., 2018, Oh et al., 2019), persuasion and mechanism design over combinatorial action sets (Fujii et al., 2021, Brero et al., 2017), and posterior inference over latent combinatorial structures (Moretti et al., 2021). These families share a Bayesian decision layer and a combinatorial action layer, but they differ sharply in objective, feedback, and computational primitive.

A second misconception is that any paper using the phrase “Bayesian selection” necessarily contains a substantive combinatorial-selection method. The supplied record for “Bayesian Selection for Efficient MLIP Dataset Selection” explicitly states that the document made available there is an IOP Publishing author-guidelines document and contains no Bayesian selection criterion, no subset-selection algorithm, no ACE/MACE features, and no silicon surface-energy experiment (Rocke et al., 28 Feb 2025). This is not merely a bibliographic anomaly; it underscores that the term should be anchored in an explicit probabilistic objective and a genuine combinatorial decision problem.

The literature also repeatedly warns against overgeneralization from restricted action spaces. The decision-theoretic pseudo-label framework is a sequential singleton selector, not a developed treatment of constrained subset choice (Rodemann, 2024). Bayesian ranking and selection addresses top-$Y\sim\mathcal N(\mu,\Sigma)$5 subsets of a finite candidate set but not budgets, matroids, diversity constraints, or graph structure (Bowen, 2022). Even highly successful Bayesian optimization systems often rely on specific representational assumptions: BOCS assumes low-order interactions, COMBO assumes graph smoothness over one-variable local moves, random-mapping methods depend on the quality of the embedding, and BVO depends on continuous relaxations of fixed-length binary or categorical encodings (Baptista et al., 2018, Oh et al., 2019, Kim et al., 2020, Wu et al., 2020).

The main open directions stated across these works are consistent. Several papers point to the need for stronger scalability, better handling of categorical and constrained spaces, and tighter integration of probabilistic modeling with combinatorial solvers (Deshwal et al., 2020, Deshwal et al., 2020). Others highlight the gap between posterior formulations that are principled but singleton or low-structure, and the broader goal of joint subset selection under constraints (Rodemann, 2024, Bowen, 2022). Bayesian mechanism-design papers suggest further work on incentive compatibility, richer valuation models, and more expressive price or query spaces (Weissteiner et al., 2022, Brero et al., 2018). Work on exact-zero priors and variational particle methods suggests another direction: rather than only selecting a combinatorial action, Bayesian methods can represent uncertainty over the combinatorial structure itself, which may be the more faithful formulation when support, rank, segmentation, or topology is the scientific object of interest (Xu et al., 2020, Moretti et al., 2021).

Taken together, the literature portrays Bayesian combinatorial selection as a broad research area organized around a single recurring challenge: a posterior distribution may be statistically natural, but extracting the next action, structure, or recommendation from it is usually a hard combinatorial optimization problem. The most successful methods are therefore those that make both sides explicit: a probabilistic model strong enough to quantify uncertainty, and an optimization mechanism precise enough to navigate exponentially large feasible families.