Knowledge-Based Optimization (KBO) Overview

• Knowledge-Based Optimization (KBO) is a family of methods that leverages formalized knowledge—such as ontologies and experimental data—to guide optimization under uncertainty and constraints.
• KBO systems integrate varied forms of knowledge, from expert intuition to empirical databases, to structure problems, prune search spaces, and guide evaluations.
• KBO methodologies have shown improvements in search efficiency and solution quality across domains like materials design, radiotherapy planning, and hyperparameter tuning.

Knowledge-Based Optimization (KBO) denotes a family of optimization strategies in which prior or accumulated knowledge is used to organize problem classes, guide search, constrain evaluation, or reinterpret intermediate results. In one formulation, it is “a systematic approach to optimization that leverages structured knowledge—formalized through ontological analysis—to connect, organize, and adapt models, methods, and problems” (Nasution, 2012). Across later work, the operative knowledge may be an experimental database, configuration-performance pairs, elicited expert preferences, inter-variable relationships, dose predictions, physical symmetries, historical populations, or compressed cross-agent summaries; correspondingly, KBO appears not as a single algorithm but as a design pattern for optimization under expensive evaluation, high dimensionality, structural uncertainty, or practical decision constraints.

1. Conceptual foundations and ontological framing

The ontological formulation of KBO treats optimization as a knowledge domain whose entities, attributes, and relations can be explicitly represented. In that view, ontology is a formal, structured representation of concepts, entities, their attributes, and the relationships among them within optimization. The paper on the ontology of KBO organizes optimization problems into categories including convex programming, nonlinear programming, combinatorial optimization, and stochastic and fuzzy optimization, while emphasizing relations such as hierarchy and reduction, duality, and adaptation between problem classes (Nasution, 2012).

This framing is motivated by the claim that traditional optimization methods are often rigid, problem-specific, and limited in their ability to transfer knowledge across domains or instances. The same source distinguishes heuristic, inductive, and deductive approaches: heuristic approaches are specific rules for problem types and are not adaptive; inductive approaches adapt models on a per-instance basis but have limited transferability; deductive approaches apply general principles to multiple instances but may miss predictive power. Ontology is proposed as a unifying structure that can overcome these limitations by connecting models, methods, and theoretical foundations.

The ontological account is also mathematical rather than purely philosophical. It explicitly places linear programming, second-order cone programming, semidefinite programming, geometric programming, and other formulations inside a structured map of “convex optima,” and states in Proposition 2 that least squares, linear programming, quadratic programming, conic optimization, and related formulations are convex optima provided they can be transformed into convex minimization problems by variable change. In this sense, KBO is not restricted to metadata or expert rules; it includes formal knowledge about reducibility, equivalence, sensitivity, and admissible methods.

2. Forms of knowledge used by KBO systems

The literature uses “knowledge” in multiple operational forms. In some cases it is a curated experimental database; in others it is procedural knowledge inferred from prior evaluations, user preference structure, variable-wise design relations, or privacy-preserving summaries of decentralized search. The common feature is that search is not driven solely by undifferentiated objective queries.

Knowledge source	Representation	Representative use
Experimental measurements	Database of 915 experimentally tested WGET-SWH units	ML prediction and HTS design screening (Li et al., 2017)
Configuration-performance history	Experience set of configuration-performance pairs	NNAdjust and NNVerify for CHPO (Wang et al., 2019)
Expert intuition	Pairwise preference dataset $\mathcal{D}_g$ over latent utility $g$	PBNN elicitation and multi-task BO (Huang et al., 2022)
Inter-variable regularities	Constant, equality, power law, and inequality rules in a Variable Relation Graph	Interactive repair in IK-EMO (Ghosh et al., 2022)
Domain structure	Deterministic surrogate $\mathcal{G}(x,D)$ or transformation $x' = T(x)$	Acquisition correction and coordinate decoupling (Xie et al., 2023)
Historical populations	Population-fitness trajectories and accumulated knowledge $M$	Pre-training and self-tuning in OKAEM (Wang et al., 4 Jan 2025)
Private local outcomes	Knowledge tokens $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$	Decentralized black-box optimization in ADKO (Rillo et al., 8 May 2026)
Predicted treatment plans	Dose predictions and objective values $g_m^*$	Dose mimicking in radiotherapy KBP pipelines (Babier et al., 2022)

These representations differ in semantics and granularity. Experimental databases and dose predictions encode empirical priors over feasible high-performing regions. Preference datasets encode relative expert judgments rather than absolute target values. Rule graphs encode algebraic regularities among variables. Knowledge tokens encode directional signal, advantage, optional language-model insight, and embeddings while withholding raw data and model parameters. This variety suggests that KBO is best understood as an optimization paradigm defined by how information is structured and reused, rather than by a fixed surrogate family or acquisition rule.

3. Core algorithmic patterns: utilization, pruning, and search shaping

A central KBO pattern is to exploit prior evaluations to bias future sampling under strict evaluation budgets. ExperienceThinking formalizes this in constrained hyperparameter optimization (CHPO), where the aim is

$$\lambda^* = \mathop{\arg \max}_{\lambda \in \Lambda_{PN}} f(\lambda, A, D)$$

subject to only $N$ configuration evaluations. Its two complementary components are “Human Experience” and “Parameter Analysis.” The first maintains configuration-performance pairs and learns “procedural knowledge” with neural networks. It defines

$$PDiffer(\lambda \rightarrow \lambda') = \frac{f(\lambda',A,D)-f(\lambda,A,D)}{|f(\lambda,A,D)|}\times 100\%$$

and uses NNAdjust to infer adjustments toward the optimum and NNVerify to predict the improvement induced by those adjustments. The second converts evaluated configurations into a classification task, extracts Random Forest feature importances, and selects the smallest $g$0 whose cumulative importance exceeds $g$1, after which only those hyperparameters are sampled while others are inherited from the best-known configuration (Wang et al., 2019).

A related pattern appears in knowledge-based machine learning for solar water heaters. There, the optimization loop begins with a large experimental database, trains predictive models, generates a very large set of hypothetical designs, screens them computationally, experimentally validates selected candidates, and feeds those results back into the database. The high-throughput screening procedure evaluates approximately $g$2 designs computationally, and weight analysis from the trained ANN determines how finely each variable should be explored, so knowledge affects both surrogate prediction and search-space discretization (Li et al., 2017).

Search shaping can also be implemented directly at the acquisition level. DKIBO augments a standard BO acquisition by adding a domain-informed deterministic predictor: $g$3 Here $g$4 is not the main surrogate but an assistive correction, and $g$5 ramps up its influence as more data become available. The paper’s rationale is that domain knowledge should steer sampling without permanently dominating posterior inference, and it therefore adds an early-stopping condition that disables the corrective term if the optimizer becomes stuck (Xie et al., 2023).

Across these examples, the recurring mechanism is not merely “use prior knowledge,” but use it in one of three precise ways: to infer promising moves, to eliminate low-value dimensions or combinations, or to reshape the criterion by which candidate points are selected.

4. Bayesian, GP-supported, and decentralized variants

A large portion of recent KBO work is implemented inside Bayesian optimization or closely related surrogate-based frameworks. One explicit formulation calls the method itself “Knowledge-Based Optimization (KBO)” and incorporates expert knowledge into BO through a two-stage architecture. Expert beliefs are treated as a latent utility function $g$6, elicited from pairwise preference queries using a Preferential Bayesian Neural Network with a Siamese architecture, active learning via PBALD, and variational inference. The elicited preference model is then transferred into the BO task through a multi-task learning architecture with shared hidden representation and task-specific output layers for $g$7 and the true objective $g$8. On benchmark functions, the preference model improved preference prediction accuracy by $g$9–$\mathcal{G}(x,D)$0 over GP-based preference models and ran $\mathcal{G}(x,D)$1 faster; in simulated-expert experiments, speed-ups of $\mathcal{G}(x,D)$2–$\mathcal{G}(x,D)$3 were reported when expert accuracy was at least $\mathcal{G}(x,D)$4; with actual human experts, average preference accuracy was in the $\mathcal{G}(x,D)$5–$\mathcal{G}(x,D)$6 range and the method outperformed standard BO on all tested functions (Huang et al., 2022).

A second line uses domain structure rather than human preference. In materials design, DKIBO injects structural knowledge through the additive corrective model above, arguing that prior knowledge should not be encoded only in the GP prior or mean because such priors can dominate late-stage predictions if inaccurate. In autonomous alignment of scientific instruments, domain knowledge instead appears as a coordinate transformation derived from physical insight. For a 12-dimensional, 6-crystal Split-and-Delay system, the transformation aligns “difference” and “sum” modes with the search axes so that the objective’s active subspaces become axis-aligned in transformed coordinates; combined with a reverse annealing schedule

$\mathcal{G}(x,D)$7

in a UCB acquisition, this yielded reliable convergence to the global optimum in a setting where standard BO, TuRBO, and multi-objective BO were reported to be unsatisfactory (Mishra et al., 11 Feb 2026).

Hybrid model-based KBO extends this further by using mechanistic dynamics as the base model and Gaussian processes as residual correctors. In optogenetic ATPase control for E. coli fermentations, the dynamic model is reduced from previous models’ approximately $\mathcal{G}(x,D)$8 states to $\mathcal{G}(x,D)$9, producing a single-level optimal control problem. GP regressors are trained on model error terms for glucose, biomass, and lactate dynamics, and the GP mean is used to improve deterministic optimization. The framework is explicitly positioned as simpler than previous bilevel formulations and experimentally achieved $x' = T(x)$0–$x' = T(x)$1 increases in lactate yield relative to uninduced controls (Espinel-Ríos et al., 2024).

Decentralized KBO generalizes the surrogate paradigm to multiple private agents. ADKO assigns each agent a private GP surrogate and replaces raw-data sharing with knowledge tokens. Token fidelity is quantified by

$x' = T(x)$2

and cumulative regret is decomposed into GP modeling error, LM bias, LM noise, and compression loss. The main theoretical result states that sublinear regret requires, among other conditions, vanishing LM bias and average token fidelity $x' = T(x)$3, which the paper addresses with fidelity-aware token pruning. Experiments on neural architecture search and scientific discovery are reported to achieve consistent improvements over strong baselines while preserving privacy and using orders of magnitude less bandwidth than federated or model-sharing alternatives (Rillo et al., 8 May 2026).

A nearby but distinct BO line is the constrained knowledge-gradient method $x' = T(x)$4-KG, which combines expected gain in the best feasible surrogate objective with probabilities of feasibility. Because the “knowledge” here refers to the knowledge-gradient acquisition rather than external domain knowledge, this is adjacent to KBO rather than a direct example of knowledge injection. The distinction is useful because it shows that not every knowledge-labeled BO method is knowledge-based optimization in the ontological, expert-guided, or domain-informed sense (Chen et al., 2021).

5. Interactive and evolutionary knowledge adaptation

Knowledge-driven search in evolutionary optimization often takes the form of explicit relation discovery and repair. IK-EMO is built around simple algebraic relationships between variables, including constant rules $x' = T(x)$5, equality rules $x' = T(x)$6, power-law rules, and inequalities. These relations are extracted from the current non-dominated set, filtered by score thresholds, and represented in a Variable Relation Graph (VRG). Users can rank, reject, or strengthen rules before the graph is converted into a directed acyclic graph and used to repair offspring every $x' = T(x)$7 generations. The framework supports synchronous and asynchronous interaction, and an ensemble repair agent updates repair-operator probabilities according to offspring survival rates. Reported case studies include stepped beam design, optimal power flow, and a 1,100-member truss problem, where moderate rule usage and medium adherence performed best, and ensemble or power-law-based repair accelerated convergence relative to the base NSGA-II (Ghosh et al., 2022).

OKAEM replaces hand-designed evolutionary operators with learnable, attention-based modules parameterizing selection, crossover, and mutation. It is trained in two phases: pre-training on accumulated optimization knowledge $x' = T(x)$8, and adaptive optimization by self-tuning during the target run. Selection is represented by an attention-based matrix over population and fitness; crossover applies MLPs with dropout to attention-weighted population features; mutation uses attention over gene coordinates to model gene interactions. The pre-training objective minimizes the discrepancy between predicted and actual next-generation populations, while the self-tuning objective pulls generated offspring toward elite individuals. The reported empirical findings are that OKAEM exploits prior knowledge for significant performance gains across knowledge-transfer settings, achieves competitive performance through self-tuning alone, outperforms state-of-the-art black-box baselines in a vision-LLM tuning case, improves with growing knowledge, and exhibits interpretable patterns resembling natural selection and genetic recombination (Wang et al., 4 Jan 2025).

These evolutionary frameworks differ from BO-centered KBO in representation and update rule, but they preserve the same architectural idea: knowledge is first externalized into an object that can be acted upon—rules, matrices, graphs, or learned operator parameters—and is then re-injected into variation or repair so that future samples are not independent of prior structure.

6. Domain implementations and empirical record

In solar energy systems, knowledge-based machine learning is used to replace slow physical testing with predictive and optimization surrogates. The target system is the water-in-glass evacuated tube solar water heater, for which direct measurement of heat collection rate and heat loss coefficient traditionally requires $x' = T(x)$9 days per unit. Using a database of $M$0 experimentally tested units, the study compares GRNN, MLFN, SVM, and ELM. All models achieved $M$1 accuracy within a $M$2 tolerance for both outputs, with RMSE values reported as $M$3 for GRNN, $M$4 for MLFN, $M$5 for SVM, and $M$6 for ELM on HCR/HLC. The same work introduced “WaterHeater” software for PC and Android and used computational HTS to screen approximately $M$7 candidate designs (Li et al., 2017).

In machine-learning hyperparameter optimization, ExperienceThinking addresses small-budget CHPO by combining knowledge utilization and pruning. The paper reports comparisons with Random Search, Grid Search, and Bayesian Optimization across tabular, image, architecture search, and feature-selection settings. Its analysis overhead is described as only slightly higher than RS and GS but much lower than BO; for $M$8 evaluations, the reported analysis time is approximately $M$9 minutes for ExperienceThinking versus more than $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$0 minutes for BO. Feature-selection experiments with up to $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$1 hyperparameters are used to argue that BO becomes prohibitively slow or infeasible, while ExperienceThinking remains efficient, and ablation results show that removing either Human Experience or Parameter Analysis reduces performance (Wang et al., 2019).

In bioprocess optimization, the optogenetic ATPase study uses knowledge-based and GP-supported models to optimize light inputs in E. coli fermentations with lactate as the main product. The method uses a simplified quasi-unstructured model with four dynamic states and formulates a single-level optimal control problem over time-varying light input and initial glucose concentration. Both knowledge-based and hybrid models recommended a two-phase light input with an initial phase of no induction followed by maximum light. Experimental implementation of the open-loop optimal profiles produced increases in lactate yield of $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$2–$$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$3 relative to uninduced controls (Espinel-Ríos et al., 2024).

In radiotherapy, OpenKBP-Opt assembles an unusually large end-to-end benchmark of knowledge-based planning pipelines. The framework combines dose predictions from $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$4 KBP models with $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$5 optimization models to form $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$6 unique pipelines and generate $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$7 plans for $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$8 head-and-neck cancer patients. The rank-order correlation between dose-score quality of predictions and corresponding pipelines ranges from $$k_i^t = \{s_i^t, c_i^t, z_i^t, \varphi(\theta_i^t)\}$$9 to $g_m^*$0, indicating a generally positive correlation. At the same time, optimization improves on raw predictions: the KBP-generated plans were significantly better on $g_m^*$1 of $g_m^*$2 DVH criteria, each optimization model satisfied a higher percentage of planning criteria than the reference plans, the best optimization models averaged $g_m^*$3–$g_m^*$4 criteria satisfaction versus $g_m^*$5 for predictions and $g_m^*$6 for reference plans, and the best pipeline reached $g_m^*$7 (Babier et al., 2022).

These applications span experimental energy systems, machine-learning tuning, fermentation control, and clinical planning, yet they share a recognizable workflow: knowledge acquisition, surrogate or rule construction, guided candidate generation, selective validation, and feedback into the knowledge store.

7. Terminology, limitations, and open directions

Several misconceptions recur in discussions of KBO. First, KBO is not reducible to prior initialization. Some methods inject knowledge into acquisition functions, others into coordinate systems, rule-based repair, operator parameterization, or post-prediction optimization. Second, knowledge need not be symbolic. It may be embodied in experimental databases, model residuals, preference judgments, or decentralized token streams. Third, more knowledge is not automatically better. The expert-augmented BO results show that near-random expert preferences can degrade performance, whereas moderate expert accuracy yields gains; DKIBO was motivated by the risk that inaccurate prior structure in the GP mean can dominate later predictions; ADKO proves that persistent LM bias or low token fidelity leads to linear regret; and the optogenetic hybrid-model paper explicitly warns that GP corrections may not extrapolate reliably outside observed operating regimes (Huang et al., 2022).

The same caution appears in interactive and model-based frameworks. IK-EMO uses score thresholds, user vetting, and adherence levels precisely to avoid over-constraining the search with spurious relations; OpenKBP-Opt shows that better predictions generally, but not always, yield better plans, implying nontrivial interaction between knowledge representation and downstream optimizer; and the ontology paper leaves open “Conjecture 1: Optimization in the incomplete has an optimum solution,” which underscores that incomplete or indiscernible problem specification remains a foundational issue rather than a solved engineering detail (Ghosh et al., 2022).

Open directions are explicit in the surveyed work. The ontology framework argues for automatic or semi-automatic generation of models and solution strategies from an ontological map. The optogenetic study notes that GP predictive variance could support future robust or risk-aware MPC, although only open-loop optimization was implemented. DKIBO points toward meta-learning or automated model selection for the corrective model $g_m^*$8. OKAEM argues that optimization capability can improve as the knowledge base grows. ADKO extends this growth principle to privacy-constrained multi-agent settings through fidelity-aware communication and optional LM-guided reasoning (Nasution, 2012).

A final terminological issue is that the acronym “KBO” is not unique. In a 2024 paper on multi-modal optimization, KBO denotes “kinetic-based optimization,” and GKBO denotes a genetic-dynamics extension of that method. That usage concerns consensus-based, gradient-free particle dynamics and is conceptually separate from knowledge-based optimization, even though the acronym coincides (Ferrarese et al., 2024). This ambiguity makes explicit definition essential in technical writing: in contemporary usage, “KBO” may refer either to optimization informed by structured knowledge or, in a different subliterature, to a kinetic particle method.