SMT 2.0: Surrogate Modeling Toolbox

• SMT 2.0 is an open-source toolbox that builds surrogate models for expensive simulations by handling mixed-variable and hierarchical design spaces.
• It integrates advanced Gaussian process techniques with novel kernel implementations for both quantitative and categorical data.
• The package offers flexible design space abstractions, robust sampling methods, and explainability layers to enhance uncertainty quantification and design exploration.

Searching arXiv for SMT 2.0 and closely related SMT papers to ground the article in cited sources. arXiv search query: SMT 2.0 Surrogate Modeling Toolbox hierarchical mixed variables Gaussian processes The Surrogate Modeling Toolbox (SMT 2.0) is an open-source Python package for surrogate modeling of expensive simulations that combines surrogate models, sampling methods, benchmark/sample problems, and applications such as Bayesian optimization (Saves et al., 2023). In its 2023 major release, SMT 2.0 extends earlier SMT functionality beyond continuous-variable settings toward mixed-variable and hierarchical or conditionally active design spaces, while also broadening support for sampling, noisy and multifidelity data, new surrogate models, and derivative-related computations for Kriging (Saves et al., 2023). The release is presented as, to the authors’ knowledge, the first open-source surrogate-modeling library that natively supports both mixed-variable and hierarchical-input Gaussian process/Kriging models in a unified Python toolbox (Saves et al., 2023).

1. Historical position and scope

SMT 2.0 emerged from a broader SMT framework already oriented toward derivative-aware surrogate modeling, including Kriging/Gaussian processes with gradient information, KPLS/GEKPLS, and spline-based models (Saves et al., 2023). The new release preserves that orientation while broadening the toolbox toward real engineering settings in which the design space is not a purely continuous Euclidean box (Saves et al., 2023).

The paper presents SMT 2.0 simultaneously as a scientific contribution and a software release. At the software level, the repository is organized into submodules such as sampling_methods, problems, and surrogate_models, and the release also highlights applications and interactive notebooks (Saves et al., 2023). At the methodological level, the release introduces native handling of mixed discrete/continuous variables and hierarchical or conditionally active variables, adds new surrogate families such as GENN and MGP, extends sampling methods, supports noisy and multifidelity data, and computes variance and kernel derivatives for Kriging (Saves et al., 2023).

Intended use cases include design space exploration, uncertainty quantification, surrogate-assisted optimization, Bayesian optimization/EGO, multifidelity modeling, and data fusion, especially for computationally expensive engineering simulations (Saves et al., 2023). Canonical engineering problems available in the package include wing weight, robot arm, borehole, and beam problems, alongside new examples specifically designed to exercise mixed and hierarchical variables (Saves et al., 2023).

A central conceptual distinction, made clearer by comparison with optimization-oriented frameworks such as MATSuMoTo, is that SMT 2.0 is fundamentally modeling-first rather than optimization-first. MATSuMoTo is described as a full surrogate-based optimization framework for expensive black-box global optimization, whereas SMT 2.0 is primarily a toolbox for constructing and using surrogate models, often to be embedded within larger optimization workflows (Mueller, 2014). This suggests that SMT 2.0’s identity lies in flexible metamodel construction and design-space handling rather than in prescribing a single end-to-end optimization loop.

2. Core problem classes: mixed, hierarchical, and structured inputs

The main motivation for SMT 2.0 is that modern engineering design problems often involve continuous variables, ordered or integer variables, categorical variables, and hierarchical or conditional variables that are active only under certain architectural choices (Saves et al., 2023). Examples given in the paper include aircraft propulsion architectures, structural sections or materials, neural-network hyperparameter tuning, multidisciplinary architecture optimization, chemical process design, and mixed-integer nonlinear black-box optimization more broadly (Saves et al., 2023).

SMT 2.0 supports design spaces containing continuous, integer/ordinal, and categorical variables (Saves et al., 2023). Ordinal and integer variables are handled similarly to continuous variables through ordered relaxation, while categorical variables are treated with specialized kernels (Saves et al., 2023). A further step, developed in later work, is the representation of mixed-variable, hierarchical, conditional, heterogeneous, tree-structured, and graph-structured design spaces using a unified framework implemented in SMT 2.0 (Saves et al., 27 Jun 2025).

Within that unified framework, a point is written as

$\mathbf{X} = (X_1,\ldots,X_n),$

and variables may have four data types: continuous, integer, ordinal, and categorical (Saves et al., 27 Jun 2025). Variables are also assigned structural roles derived from a graph of dependencies: meta, meta-decreed, decreed, and neutral (Saves et al., 27 Jun 2025). A meta variable governs the existence or support of other variables; a decreed variable depends on a parent variable for inclusion or exclusion; and the framework further introduces partially-decreed variables, which remain present but with support restricted by parent context (Saves et al., 27 Jun 2025).

That later extension formalizes support restriction using sets. For variable $x_i$ with full value space $\mathcal{X}_i$ and parent set $P_i$,

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$

is the admissible support under the graph $\mathcal{G}$, and the variable is excluded iff

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$

A variable is partially decreed by parent $x_j$ when

$$\mathcal{S}_{\mathcal{G}}(x_i \mid x_j) \subset \mathcal{X}_i \quad \text{and} \quad \mathcal{S}_{\mathcal{G}}(x_i \mid x_j) \neq \emptyset$$

(Saves et al., 27 Jun 2025). This generalizes the simpler on/off inclusion logic emphasized in the 2023 SMT 2.0 paper.

The same 2025 framework introduces Design Space Graphs and Architecture Design Space Graphs as graph-based encodings of such relationships, extending hierarchical design-space handling from simple trees toward DAG-based structures with mixed node and relation types (Saves et al., 27 Jun 2025). A cautious implication is that SMT 2.0 evolved from supporting hierarchical variables in Kriging models to supporting graph-aware design-space semantics more generally.

3. Gaussian-process and Kriging methodology

SMT 2.0’s flagship methodological contribution is the extension of Gaussian process/Kriging models to mixed-variable and hierarchical inputs (Saves et al., 2023). The release paper assumes the standard Kriging framework, with predictive mean and variance of the form

$$\mu(x) = f(x)^\top \hat{\beta} + r(x)^\top R^{-1}(y - F\hat{\beta}),$$

$x_i$0

(Saves et al., 2023). Hyperparameters are optimized through GP log-likelihood, and SMT 2.0 extends the implementation to support kernel derivatives, variance prediction, and variance derivatives for Kriging (Saves et al., 2023).

For mixed variables, SMT 2.0 treats quantitative and categorical parts separately and combines them multiplicatively (Saves et al., 2023). For quantitative variables, the paper writes

$x_i$1

For categorical variables, a unified construction is given as

$x_i$2

where $x_i$3 is a symmetric positive definite matrix-valued mapping and $x_i$4 depends on the categorical model (Saves et al., 2023).

SMT 2.0 implements four categorical kernels:

Kernel	SMT name in paper	Hyperparameter structure
Gower-distance-based kernel	GOWER_KERNEL	1 hyperparameter per categorical variable
Continuous relaxation kernel	CONT_RELAX_KERNEL	$x_i$5 hyperparameters for variable $x_i$6
Homoscedastic hypersphere kernel	HOMO_HSPHERE_KERNEL	$x_i$7 hyperparameters
Exponential homoscedastic hypersphere kernel	EXP_HOMO_HSPHERE_KERNEL	$x_i$8 hyperparameters

The paper gives the following specific parameterizations (Saves et al., 2023). For SMT GD,

$x_i$9

$\mathcal{X}_i$0

For SMT CR,

$\mathcal{X}_i$1

$\mathcal{X}_i$2

For SMT EHH,

$\mathcal{X}_i$3

$\mathcal{X}_i$4

For SMT HH,

$\mathcal{X}_i$5

$\mathcal{X}_i$6

Here $\mathcal{X}_i$7 is built by a hypersphere decomposition, ensuring positive definiteness of the categorical correlation matrix (Saves et al., 2023).

For hierarchical spaces, SMT 2.0 adopts a decomposition into neutral variables $\mathcal{X}_i$8, meta variables $\mathcal{X}_i$9, and decreed variables $P_i$0, with

$P_i$1

(Saves et al., 2023). The hierarchical kernel takes the form

$P_i$2

SMT 2.0 provides two dedicated hierarchical kernels: the SMT Arc-Kernel and the SMT Alg-Kernel (Saves et al., 2023). The Alg-Kernel, introduced as novel in the paper, factorizes the meta-decreed component as

$P_i$3

If decreed variable $P_i$4 is active in both points,

$P_i$5

while if it is not jointly active,

$P_i$6

(Saves et al., 2023). The release paper argues that this gives Arc-like behavior without extra hyperparameters and with improved numerical stability (Saves et al., 2023).

The later unified-framework paper places these kernels within a broader theory of hierarchical distances and graph-structured kernels. It defines variable-wise distances

$P_i$7

with

$P_i$8

and the aggregate distance

$P_i$9

(Saves et al., 27 Jun 2025). This later development suggests a generalization from hierarchical kernels on fixed conditional spaces toward kernels defined over graph-constrained structured domains.

4. Design-space abstraction, software architecture, and extensibility

A major API addition in SMT 2.0 is the DesignSpace class, through which users define variables with explicit types such as FloatVariable, OrdinalVariable, CategoricalVariable, and IntegerVariable (Saves et al., 2023). Conditional activity is declared through declare_decreed_var, allowing a variable to be active only when a meta variable takes designated values (Saves et al., 2023).

The typical mixed or hierarchical GP workflow described in the paper is: define a DesignSpace, declare conditional activity, generate a valid design of experiments, evaluate the expensive function, instantiate a surrogate such as KRG, optionally wrap it with MixedIntegerKrigingModel, call set_training_values, train the model, and then use predict_values and predict_variances (Saves et al., 2023). The example in the paper combines MixedIntegerSamplingMethod, MixedIntegerKrigingModel, and KRG(design_space=..., corr="abs_exp", categorical_kernel=..., hierarchical_kernel=...) (Saves et al., 2023).

The design-space layer also performs correction and imputation. Correction forces values into valid domains, such as rounding integers, while imputation replaces inactive variables with default values—$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$0 for discrete variables and midpoint of bounds for continuous variables (Saves et al., 2023). The paper treats such imputation as a baseline rather than as the preferred modeling strategy for hierarchical inputs (Saves et al., 2023).

Later SMT 2.0 work extends the design-space abstraction through integration with ConfigSpace and adsg-core, introducing implementation classes ConfigSpaceDesignSpaceImpl and AdsgDesignSpaceImpl (Saves et al., 27 Jun 2025). The newer AdsgDesignSpaceImpl supports mixed hierarchical variables, nested hierarchy, incompatibility and exclusion constraints, explicit graph structure, and explicit graph visualization (Saves et al., 27 Jun 2025). The 2025 paper reports that, compared with ConfigSpaceDesignSpaceImpl, AdsgDesignSpaceImpl achieves 76% speedup for correcting 1000 invalid points and computing activeness, 36% speedup for generating one point by each discrete possibility, and 15% speedup for generating 100 valid points (Saves et al., 27 Jun 2025).

SMT 2.0 is open source, distributed under the New BSD license, documented publicly, and designed to be extensible through abstract APIs for SurrogateModel, SamplingMethod, and Problem (Saves et al., 2023). The project includes user and developer documentation, automated regression testing, notebooks reproducing paper results, and API conventions for adding new surrogates, samplers, and problems (Saves et al., 2023). The package also uses Numba to accelerate some code paths; for a mixed Kriging model with 150 training points, the paper reports up to 80% speedup in training time, with about 24 seconds of one-time JIT compilation overhead per install or upgrade (Saves et al., 2023).

A separate 2025 SMT paper extends extensibility specifically at the GP kernel layer, introducing a base kernel class, refactoring standard kernels into class-based structures, and enabling kernel composition with Python operators such as + and * (Gonel et al., 13 Jul 2025). That paper adds a rational quadratic kernel and a periodic kernel, intended for “frequency-aware” behavior in forecasting and oscillatory systems, and integrates them into SMT 2.0’s open-source framework (Gonel et al., 13 Jul 2025). This suggests that SMT 2.0’s architectural trajectory is toward increasingly modular and user-extensible kernel engineering.

5. Sampling, additional surrogate families, and data regimes

SMT 2.0 retains classic sampling families—Random, Full Factorial, and LHS—while extending them for mixed and hierarchical spaces (Saves et al., 2023). The toolbox supports five LHS criteria: center, maximin, centermaximin, correlation, and ese (Saves et al., 2023). Two additions are particularly emphasized. NestedLHS is introduced for multifidelity sampling, where low- and high-fidelity datasets should be nested, and expand_lhs enlarges an existing LHS while preserving the ese criterion (Saves et al., 2023). The samplers were adapted to generate valid points in mixed and hierarchical spaces using DesignSpace rules (Saves et al., 2023).

Beyond classical Kriging variants, SMT 2.0 adds GENN and MGP (Saves et al., 2023). GENN is a Gradient-Enhanced Neural Network: a fully connected multilayer perceptron trained on both function values and partial derivatives, with a loss incorporating response and gradient error (Saves et al., 2023). MGP, or Marginal Gaussian Process, targets high-dimensional problems by assuming the function depends on a low-dimensional linear embedding. If

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$1

and $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$2, then one uses a kernel

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$3

(Saves et al., 2023).

For high-dimensional problems, SMT 2.0 continues and extends KPLS and KPLSK, reducing the effective number of correlation hyperparameters from $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$4 to $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$5 and adding multifidelity versions MFKPLS and MFKPLSK, together with an automatic reduced-dimension criterion based on Wold’s $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$6 criterion (Saves et al., 2023). The release also supports noisy Kriging and heteroscedastic noise via the use_het_noise option, and expands multifidelity modeling through Variable-Fidelity Modeling (VFM), Multi-Fidelity Kriging (MFK), MFKPLS, and MFKPLSK (Saves et al., 2023).

Subsequent papers reinforce several of these directions. A 2017 study on large-scale variable-fidelity surrogate modeling addresses the familiar GP bottleneck that exact inference becomes prohibitive when low-fidelity datasets reach thousands of points, and proposes Nyström-based sparse variable-fidelity GP and a prediction-time black-box low-fidelity update (Burnaev et al., 2017). Although that work is not itself an SMT 2.0 paper, it is directly relevant to SMT’s multi-fidelity GP use cases, and a plausible implication is that it outlines one path for extending SMT’s multi-fidelity backend toward larger low-fidelity datasets.

The 2025 frequency-aware kernel paper further expands SMT 2.0’s GP regime by emphasizing user-defined kernels, compositional kernels, and periodic or multi-scale covariance structures for forecasting, oscillatory mechanics, and cyclic dynamics (Gonel et al., 13 Jul 2025). It introduces the rational quadratic kernel

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$7

and the periodic kernel

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$8

(Gonel et al., 13 Jul 2025). Composite kernels such as

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) \subseteq \mathcal{X}_i$$9

and

$\mathcal{G}$0

are used to model oscillatory signals and trend-plus-seasonality structures (Gonel et al., 13 Jul 2025).

6. Empirical results, applications, and explainability

The 2023 SMT 2.0 paper evaluates mixed and hierarchical GP capabilities on both surrogate-modeling and Bayesian-optimization tasks (Saves et al., 2023). In a mixed cantilever beam benchmark with continuous variables and one categorical cross-section variable, the paper compares four categorical kernels. With a 98-point LHS training set and $\mathcal{G}$1 validation points, the reported displacement RMSE and likelihood values are:

• SMT GD: RMSE $\mathcal{G}$2, likelihood $\mathcal{G}$3, 3 hyperparameters
• SMT CR: RMSE $\mathcal{G}$4, likelihood $\mathcal{G}$5, 14 hyperparameters
• SMT EHH: RMSE $\mathcal{G}$6, likelihood $\mathcal{G}$7, 68 hyperparameters
• SMT HH: RMSE $\mathcal{G}$8, likelihood $\mathcal{G}$9, 68 hyperparameters

(Saves et al., 2023). The paper concludes that HH and EHH are markedly more accurate, but much more expensive to train, and recommends CR by default as the trade-off between model complexity and performance (Saves et al., 2023).

In a hierarchical neural-network benchmark, the number of hidden layers acts as a meta variable and neuron counts in later layers are decreed variables (Saves et al., 2023). With a 99-point LHS training set and a 3000-point validation set, the reported results are:

• SMT Alg-Kernel: RMSE $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$0, likelihood $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$1, 10 hyperparameters
• SMT Arc-Kernel: RMSE $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$2, likelihood $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$3, 10 hyperparameters
• Imp-Kernel: RMSE $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$4, likelihood $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$5, 10 hyperparameters

(Saves et al., 2023). This benchmark is used to argue that the new Alg-Kernel outperforms the simplified Arc implementation and pure imputation in that setting (Saves et al., 2023).

The Bayesian-optimization examples further emphasize that kernel fidelity affects not only static prediction but also optimization performance (Saves et al., 2023). In a mixed-variable toy function from CAT-EGO, EGO with HH and EHH converges in about 18 evaluations, whereas CR and GD take around 26 (Saves et al., 2023). In a hierarchical modified Goldstein problem with 11 variables, both hierarchical kernels outperform imputation and random search, and one run of the Alg-Kernel finds the true minimum, though the authors note this is not statistically significant (Saves et al., 2023).

A 2025 extension, SMT-EX, shifts attention from prediction to interpretability (Robani et al., 25 Mar 2025). SMT-EX is an explainability layer around SMT that adds SHAP, PDP, and ICE, along with Sobol’ indices for continuous-variable problems and conformal prediction (Robani et al., 25 Mar 2025). It is model-agnostic at the SMT level because it only requires access to surrogate predictions, and is designed to work particularly well with SMT’s mixed-categorical GP models (Robani et al., 25 Mar 2025).

SMT-EX demonstrates this on a 10-variable wing weight problem and a mixed-categorical cantilever beam bending problem (Robani et al., 25 Mar 2025). For the wing weight case, a GP with squared exponential covariance is trained on 300 points with an 80/20 train/test split and achieves test RMSE $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$6 (Robani et al., 25 Mar 2025). For the mixed-categorical cantilever beam case, a mixed-categorical GPR model is trained on 300 samples with an 80/20 split and achieves test RMSE $$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$7 (Robani et al., 25 Mar 2025). The paper further uses learned categorical correlation matrices from SMT’s mixed GP to show that category levels cluster by thickness or hollowness rather than by gross shape family, which it interprets as physically meaningful structure (Robani et al., 25 Mar 2025).

These explainability results reinforce an important characteristic of SMT 2.0: its mixed-variable GP models do not merely accommodate heterogeneous inputs; they also learn internal structures, such as categorical-level correlations, that can themselves be inspected (Robani et al., 25 Mar 2025). This suggests that SMT 2.0’s contribution is not limited to predictive flexibility but also includes a richer structural interpretation of engineering design spaces.

7. Comparisons, limitations, and later developments

The SMT 2.0 paper compares the toolbox against BoTorch, Dakota, DiceKriging, KerGP, LVGP, Parmoo, and Spearmint, and argues that SMT 2.0 uniquely combines mixed variables, GD/CR/HH/EHH kernels, hierarchical variables, Python implementation, and BSD licensing (Saves et al., 2023). The authors’ main claim is that many libraries support mixed variables only through simple methods such as Gower distance or continuous relaxation, that KerGP supports richer categorical kernels but is in R, and that no open-source library other than SMT 2.0 supports hierarchical-variable Kriging models of the implemented type (Saves et al., 2023).

The paper is nevertheless explicit about limitations. HH and EHH can be much more accurate, but their hyperparameter count grows as

$$\mathcal{S}_{\mathcal{G}}(x_i \mid P_i) = \emptyset.$$8

per categorical variable, making training costlier and optimization harder (Saves et al., 2023). The dedicated hierarchical kernel derivation assumes quantitative decreed variables, and rich mixed or hierarchical models remain bottlenecked by hyperparameter optimization (Saves et al., 2023). Noise support exists but is not the central focus of the main benchmarks (Saves et al., 2023).

Later work sharpens both the scope and the constraints of SMT 2.0. The graph-based hierarchical framework emphasizes that many real domains are heterogeneous, tree-structured, or DAG-structured, and that points may not share the same active variables or bounds (Saves et al., 27 Jun 2025). The kernel-framework paper notes that current demonstrations of periodic and compositional kernels are all one-dimensional, and warns that the number of hyperparameters multiplies with dimension, making modeling time quickly prohibitive (Gonel et al., 13 Jul 2025). SMT-EX, in turn, notes that explanations are only as good as the surrogate and that many explanation methods depend strongly on sampled data distributions (Robani et al., 25 Mar 2025).

Comparison with other surrogate-related toolboxes clarifies SMT 2.0’s niche. GTApprox, for example, is presented as an industrial surrogate modeling environment emphasizing automated model selection, hints, tensorized approximations, and workflow-oriented features for medium-scale engineering data (Belyaev et al., 2016). MATSuMoTo, by contrast, is an optimization engine built around surrogate modeling for expensive black-box global optimization, especially with mixed-integer variables (Mueller, 2014). SMT 2.0 differs from both: it is less automation-heavy than GTApprox and less optimization-prescriptive than MATSuMoTo, but more oriented toward an open, extensible, research-grade surrogate-modeling ecosystem in Python (Saves et al., 2023).

Taken together, these papers present SMT 2.0 as an evolving framework whose 2023 release established native support for mixed-variable and hierarchical-input Gaussian processes, and whose later extensions broadened this foundation toward graph-structured design spaces, composable kernels, and post-fit explainability (Saves et al., 2023, Saves et al., 27 Jun 2025, Gonel et al., 13 Jul 2025, Robani et al., 25 Mar 2025). A plausible implication is that SMT 2.0’s defining contribution is not a single surrogate family, but a unifying software and methodological layer for surrogate modeling on structured engineering domains.