Accelerated p-KGFN Algorithm

• Accelerated p-KGFN Algorithm is a Bayesian optimization method for complex, network-structured functions that enables cost-aware, partial evaluations.
• It replaces nested, per-node global optimization with a single simulation and discrete candidate set search, dramatically reducing computational overhead.
• Empirical results report up to 16× speedup with minimal query efficiency loss, making it ideal for expensive evaluations in scientific and industrial applications.

The Accelerated p-KGFN Algorithm is a recent advancement in Bayesian optimization for complex function networks, enabling efficient cost-aware partial evaluations in domains where function queries are expensive and the objective is structured as a directed acyclic network. Accelerated p-KGFN—also termed "Fast p-KGFN"—addresses the primary computational bottleneck of its predecessor (p-KGFN) by introducing innovations in candidate selection and acquisition function approximation, achieving substantial reductions in wall-clock optimization time while retaining most improvements in query efficiency.

1. Problem Setting and Motivation

Bayesian Optimization of Function Networks (BOFN) involves optimizing objectives comprised of interdependent black-box functions, each represented as a node in a network. Real-world applications, such as molecular design or sequential manufacturing, often exhibit networked objectives where:

• Each node may have variable, substantial evaluation cost.
• Nodes are partly independent—allowing selective, partial evaluations within the same experiment.

The original p-KGFN method reduced the number of expensive full-network queries by allowing cost-aware, node-level evaluations. However, the core limitation was high computational overhead: at each iteration, for each node, a nested Monte Carlo acquisition function needed global optimization, resulting in prohibitive cumulative runtimes for realistic network sizes.

2. Algorithmic Structure of Fast p-KGFN

Fast p-KGFN introduces two key modifications:

1. Single Global Simulation for Candidate Generation: Instead of optimizing a separate acquisition function per node, a single global candidate input $\hat{x}_n$ is selected per iteration using an Expected Improvement surrogate (EIFN). This drastically reduces the number of global optimizations.
2. Discrete Candidate Set for Acquisition Maximization: The standard p-KGFN acquisition function involves computing an expected improvement, which requires inner maximization over the entire (typically continuous) input space after hypothetical "fantasy" observations. Fast p-KGFN replaces this with maximization over a small, strategically constructed discrete candidate set $\mathcal{A}$, thereby minimizing inner-loop computation.

Pseudocode Outline

x_hat = argmax_x EIFN_n(x) simulated_outputs = sample_gp_posteriors(x_hat) z_hat_k = generate_node_candidate(simulated_outputs, node_k) A = build_candidate_set() alpha_k = acquisition_function(z_hat_k, A) k_star = argmax_k alpha_k evaluate_node(k_star, z_hat_k) update_gp_models()

All global and node-level candidate generation is driven by a single network-wide simulation, and the core computational bottlenecks of nested optimization are replaced with efficient, discrete search.

3. Mathematical Framework

• Node-wise observation model: Each node $y_k(x)$ represents a function $f_k$ whose evaluation may depend on both external inputs and outputs from parent nodes ($\mathcal{J}(k)$).
• Posterior mean (main outcome): $\nu_n(x) = \mathbb{E}[y_K(x) | \mathcal{D}_n]$
• Best posterior mean: $\nu_n^* = \max_{x \in \mathcal{X}} \nu_n(x)$
• Acquisition function (per-node, per-candidate):

$$\alpha_{n,k}(z_k) = \frac{\mathbb{E}[\max_{x \in \mathcal{A}} \nu_{n+1}(x; z_k)] - \nu_n^*}{c_k(z_k)}$$

This formula quantifies the expected improvement in the maximal posterior mean per unit cost for evaluating node $k$ at candidate $z_k$, with inner maximization efficiently approximated over the discrete set $\mathcal{A}$.

4. Computational and Practical Advantages

The principal practical gain is a dramatic reduction in computational overhead without substantial loss in optimization quality:

• Inner-loop optimization reduced: The requirement for nested, full-space acquisition optimization for each node is replaced by a single continuous optimization (global candidate selection) and light-weight maximization over $\mathcal{A}$.
• Drastic reduction in acquisition time: Wall-clock runtime drops by up to a factor of 16 (e.g., FreeSolv benchmark), with query efficiency (as measured by improvement per cumulative cost) remaining close to or matching full p-KGFN.
• Scalability: The approach is robust to network size and discrete candidate set construction; hyperparameter sensitivity is low, provided critical points (posterior maximizer) are always included in $\mathcal{A}$.

	p-KGFN	Fast p-KGFN
Node candidate	Global optimization per node	Single global simulation + node-wise construction
Acquisition	Nested MC, continuous maximization	MC with fast discretized maximization over small set $\mathcal{A}$
Speedup	–	Up to 16×
Solution loss	Minimal (empirically negligible)	Minimal (empirically negligible)

5. Empirical Validation and Results

Three representative problems illustrate the performance of Fast p-KGFN:

• AckMat (Synthetic): Function network with independent nodes, fixed costs.
• FreeSolv (Benchmark): Pharmaceutical solvation network, partial node costs.
• Manu (Manufacturing): Realistic network, highly variable costs across nodes.

Results demonstrate:

• Nearly identical objective value trajectories to p-KGFN, and superior to baselines lacking partial evaluation or network awareness.
• Acquisition runtime per iteration decreased from several minutes (p-KGFN) to seconds or less (Fast p-KGFN).
• Robustness under ablation and hyperparameter scaling, with performance insensitive to moderate changes in discrete candidate set size or composition.

Problem	p-KGFN Time (min)	Fast p-KGFN Time (min)	Realized Speedup
FreeSolv	5.45	0.34	16.0×
AckMat	11.24	0.98	11.5×
Manu	7.8	1.4	5.6×

6. Implementation Considerations and Applicability

• Candidate Set Construction: Discrete set $\mathcal{A}$ should be formed via batch Thompson sampling (promoting exploration) and local sampling around current best (exploitation). Always include $\nu_n^*$ in $\mathcal{A}$ for robustness.
• Posterior Sampling Efficiency: Use Gaussian process posterior sampling to generate simulated intermediate outputs for all nodes from a single batch, amortizing computational cost over the network.
• Deployment: Particularly advantageous when evaluation costs dominate and acquisition computation is a bottleneck, such as in chemical, manufacturing, or high-throughput scientific experiment design.

This approach is particularly suited for domains involving hierarchical or compositional systems where evaluations are costly, partial, and network-structured.

7. Conclusions and Comparative Perspective

The Accelerated p-KGFN Algorithm represents an effective optimization of Bayesian optimization for complex function networks where partial evaluations and variable costs are inherent to the domain. It achieves this by leveraging network-wide candidate sharing and discrete acquisition maximization, offering order-of-magnitude computational savings (up to 16×) with only modest, typically negligible, reductions in query efficiency. This operational efficiency enables the practical application of BOFN with partial, cost-aware evaluation to larger and more complex real-world problems than previously feasible.

For further technical details, formulas, ablation studies, and ready-to-use implementations, see the paper and official code repository.