ASTRO-MFDF: Adaptive Multi-Fidelity Trust-Region Method

• The paper introduces ASTRO-MFDF, an adaptive multi-fidelity simulation optimization framework that integrates trust-region strategies with multi-fidelity Monte Carlo sampling to handle noisy, expensive evaluations.
• It builds local surrogate models at various fidelities and dynamically adjusts fidelity selection using a correlation vector to mitigate low-fidelity bias while guiding the search.
• Empirical results demonstrate that ASTRO-MFDF achieves faster convergence and lower objective values compared to traditional methods under fixed computational budgets.

Searching arXiv for the cited ASTRO-MFDF paper and closely related trust-region work to ground the article. ASTRO-MFDF is an adaptive sampling trust-region method for multi-fidelity simulation optimization that targets expensive stochastic objectives observed only through noisy zeroth-order simulation outputs. It is formulated for problems in which the highest-fidelity objective is an expectation, $f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)]$, with $x\in\mathbb{R}^d$, $f^0$ nonconvex and bounded below, and additional lower-fidelity simulators $F^i(x,\xi^i)$, $i=1,\dots,q$, that are cheaper but biased approximations of the high-fidelity response (Ha et al., 5 Aug 2025). The method extends ASTRO-DF into a multi-fidelity setting through a Multi-Fidelity Adaptive Sampling mechanism based on multi-fidelity Monte Carlo, and it is designed to answer two coupled questions: where and how much to sample, and when low-fidelity models are informative enough to guide the search (Ha et al., 5 Aug 2025).

1. Problem formulation and motivation

ASTRO-MFDF is defined for simulation optimization settings in which the true objective is available only as an expectation over a stochastic simulator and each function evaluation is expensive (Ha et al., 5 Aug 2025). The high-fidelity optimization problem is

$$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$

where the index $0$ denotes the highest-fidelity simulation. Lower-fidelity levels $i=1,\dots,q$ provide cheaper but biased approximations $F^i(x,\xi^i)$, with corresponding mean responses $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$ (Ha et al., 5 Aug 2025).

The method operates with zeroth-order stochastic oracles: only noisy function values are available, not gradients. Given $x\in\mathbb{R}^d$0 iid replications at fidelity $x\in\mathbb{R}^d$1, the sample mean and sample variance estimator are

$x\in\mathbb{R}^d$2

$x\in\mathbb{R}^d$3

and the covariance between fidelity $x\in\mathbb{R}^d$4 and fidelity $x\in\mathbb{R}^d$5 is estimated by

$x\in\mathbb{R}^d$6

The decision variables are continuous, and the algorithm is tailored to unconstrained continuous optimization, although the reported experiments use bounded domains (Ha et al., 5 Aug 2025).

The motivation for multi-fidelity is computational. High-fidelity runs may require longer horizons, more detailed physics, or more components, whereas lower fidelities can be produced by simplifying the system or shortening the simulation run length in steady-state problems (Ha et al., 5 Aug 2025). Because these models often share structure with the high-fidelity system, they are frequently correlated with $x\in\mathbb{R}^d$7, and thus can provide directional information, help build local models with fewer high-fidelity calls, and improve finite-time performance under a fixed budget (Ha et al., 5 Aug 2025).

The central difficulty is bias. The low-fidelity expectations $x\in\mathbb{R}^d$8 are biased approximations to $x\in\mathbb{R}^d$9, and their optima can be far from the high-fidelity optimum. Naive use of low-fidelity data can mislead surrogate construction, drive the search toward regions with poor high-fidelity performance, and slow convergence or prevent discovery of good solutions (Ha et al., 5 Aug 2025). ASTRO-MFDF is specifically constructed to manage this tension through adaptive sampling and correlation-aware fidelity selection.

2. Trust-region architecture and model hierarchy

At a high level, each iteration $f^0$0 of ASTRO-MFDF maintains multiple trust-region radii $f^0$1, builds local interpolation models $f^0$2 for each fidelity level, uses a correlation vector $f^0$3 to determine which fidelities to trust, invokes Multi-Fidelity Adaptive Sampling to determine sample sizes and whether to use Monte Carlo or multi-fidelity Monte Carlo, chooses a candidate $f^0$4, evaluates that candidate in high fidelity with adaptive sampling, and then updates trust regions and correlation measures (Ha et al., 5 Aug 2025).

The underlying framework is a stochastic trust-region scheme. At iteration $f^0$5, the method maintains the current point $f^0$6, a high-fidelity trust-region radius $f^0$7, and a local surrogate $f^0$8 that approximates $f^0$9 on the ball

$F^i(x,\xi^i)$0

ASTRO-MFDF augments this with radii $F^i(x,\xi^i)$1 and models $F^i(x,\xi^i)$2 for all fidelities $F^i(x,\xi^i)$3 (Ha et al., 5 Aug 2025).

Models are constructed from function values at a design set of $F^i(x,\xi^i)$4 points in the trust region. The method adopts a derivative-free, model-based trust-region structure in the style of Conn-Gould-Toint and ASTRO-DF, typically using quadratic interpolation or regression models to approximate gradients and Hessians locally and then solving subproblems of the form

$F^i(x,\xi^i)$5

After computing a candidate, the method evaluates the high-fidelity function and forms the success ratio

$F^i(x,\xi^i)$6

If $F^i(x,\xi^i)$7 exceeds the threshold $F^i(x,\xi^i)$8, the step is accepted and the trust region is expanded; otherwise it is rejected and the trust region is shrunk: $F^i(x,\xi^i)$9 The parameters satisfy the standard trust-region conditions $i=1,\dots,q$0, $i=1,\dots,q$1, $i=1,\dots,q$2, and $i=1,\dots,q$3 (Ha et al., 5 Aug 2025).

A distinctive feature is the simultaneous management of local models at multiple fidelities. This implies that the trust-region logic is not merely replicated across fidelity levels; rather, the method uses those models competitively and selectively. A plausible implication is that the architecture is intended to exploit low-cost local structure when it has predictive value while preserving high-fidelity validation as the arbiter of progress.

3. Correlation vector and fidelity selection mechanism

The principal mechanism for deciding when lower fidelities should influence the search is the correlation vector $i=1,\dots,q$4, which quantifies the local usefulness of each low-fidelity model (Ha et al., 5 Aug 2025). For each $i=1,\dots,q$5, the method maintains a positive scalar $i=1,\dots,q$6 and compares it against a user-chosen threshold $i=1,\dots,q$7.

The interpretation is operational rather than purely statistical. If a model $i=1,\dots,q$8 repeatedly proposes candidates that deliver genuine high-fidelity improvement, then fidelity $i=1,\dots,q$9 is behaving similarly to fidelity $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$0 in the local region and $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$1 is increased. If it repeatedly fails, $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$2 is decreased (Ha et al., 5 Aug 2025). The criterion is therefore a behavioral proxy for local usefulness, not an explicit estimation of correlation coefficients.

The correlation vector controls two aspects of the algorithm. First, it affects model construction priority. When $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$3, the algorithm builds a dedicated model $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$4 using its own design set inside $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$5. When $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$6, fidelity $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$7 is not prioritized to propose steps; instead, $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$8 may be built only partially, reusing points sampled for $$\min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)],$$9, primarily to continue updating $0$0 (Ha et al., 5 Aug 2025).

Second, it affects inner-loop step generation. The algorithm iterates fidelities from lower to higher, invokes a low-fidelity trust-region subroutine ASTRO-LFDF-$0$1, evaluates the resulting candidate in high fidelity, and, if the candidate produces sufficient reduction in $0$2, accepts it, expands $0$3, and increases $0$4 by multiplication with $0$5 (Ha et al., 5 Aug 2025). When all low-fidelity attempts fail, the method falls back to a high-fidelity-driven step in which models for all fidelities are constructed jointly with shared design points and the candidate is chosen primarily using $0$6.

In the joint-model case, each low fidelity $0$7 is evaluated through the ratio

$0$8

where $0$9 is a sufficient-reduction constant. If $i=1,\dots,q$0, then

$i=1,\dots,q$1

and otherwise

$i=1,\dots,q$2

This update rule is the screening mechanism by which useful low fidelities are promoted and unhelpful ones are downweighted (Ha et al., 5 Aug 2025).

The low-fidelity trust-region subroutine ASTRO-LFDF-$i=1,\dots,q$3 follows the same logic. If $i=1,\dots,q$4, it immediately sets $i=1,\dots,q$5, effectively declining to propose a low-fidelity-driven move. Otherwise it selects a design set in $i=1,\dots,q$6, uses MFAS to estimate $i=1,\dots,q$7 at those points, builds $i=1,\dots,q$8, computes

$i=1,\dots,q$9

evaluates the high-fidelity function at $F^i(x,\xi^i)$0 and $F^i(x,\xi^i)$1, and forms the low-fidelity success ratio

$F^i(x,\xi^i)$2

If $F^i(x,\xi^i)$3, the subroutine shrinks both the radius and the correlation parameter: $F^i(x,\xi^i)$4 This design ensures that low fidelity can influence candidate generation only when it has demonstrated actual predictive value for high-fidelity improvement (Ha et al., 5 Aug 2025).

4. Multi-Fidelity Adaptive Sampling and MFMC integration

ASTRO-MFDF uses Multi-Fidelity Adaptive Sampling as its sampling engine whenever it requires function estimates, whether for model construction, incumbent evaluation, or candidate evaluation (Ha et al., 5 Aug 2025). MFAS is guided by the adaptive sampling rule of ASTRO-DF, in which the sample size at a point $F^i(x,\xi^i)$5 and iteration $F^i(x,\xi^i)$6 is the smallest integer $F^i(x,\xi^i)$7 satisfying

$F^i(x,\xi^i)$8

where $F^i(x,\xi^i)$9 is a lower bound, $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$0 is a tuning constant, and $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$1 is a slowly growing sequence (Ha et al., 5 Aug 2025). The left-hand side is the stochastic error and the right-hand side is proportional to the optimality gap represented by $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$2. As $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$3, the required sample size grows like $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$4.

In the multi-fidelity setting, the method uses the multi-fidelity Monte Carlo estimator. For target fidelity $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$5, lowest fidelity index $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$6, and sample sizes $f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$7, the estimator is

$f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$8

with variance

$f^i(x):=\mathbb{E}[F^i(x,\xi^i)]$9

For ASTRO-MFDF, the primary case is $x\in\mathbb{R}^d$00, unbiased estimation of $x\in\mathbb{R}^d$01. The paper assumes positive correlation enforced via Common Random Numbers so that suitable control-variate coefficients $x\in\mathbb{R}^d$02 can reduce variance (Ha et al., 5 Aug 2025).

MFAS must determine sample sizes $x\in\mathbb{R}^d$03, coefficients $x\in\mathbb{R}^d$04, and whether standard Monte Carlo or MFMC is cheaper for the required accuracy. It maintains variance and covariance estimates at each fidelity based on previously collected samples, plugs those estimates into the MFMC variance formula, and solves the auxiliary optimization problem

$x\in\mathbb{R}^d$05

where $x\in\mathbb{R}^d$06 is the per-sample cost of fidelity $x\in\mathbb{R}^d$07 and $x\in\mathbb{R}^d$08 is the current number of collected samples (Ha et al., 5 Aug 2025).

In parallel, MFAS computes the cost of standard high-fidelity Monte Carlo: $x\in\mathbb{R}^d$09 MFAS chooses MFMC if $x\in\mathbb{R}^d$10, and otherwise chooses Monte Carlo (Ha et al., 5 Aug 2025). If the selected estimator already satisfies the variance bound, the current estimate is returned. Otherwise additional samples are allocated. In the MFMC case, if $x\in\mathbb{R}^d$11 for some $x\in\mathbb{R}^d$12, the algorithm chooses the highest fidelity index among such $x\in\mathbb{R}^d$13 and takes more replications there, then recomputes variance estimates and resolves the optimization problem (Ha et al., 5 Aug 2025).

This construction makes the sampling policy simultaneously trust-region aware, cost-aware, and sequential. A plausible implication is that the method seeks to decouple the use of low fidelity for variance reduction from the use of low fidelity for directional modeling; these are related but distinct roles inside the overall algorithm.

5. Statistical properties and asymptotic rationale

The paper does not establish new convergence theorems for ASTRO-MFDF, but it is explicitly built on ASTRO-DF’s convergence and iteration-complexity results and on trust-region theory for noisy oracles, including methods such as STRONG and STORM (Ha et al., 5 Aug 2025). Under standard assumptions in that literature—$x\in\mathbb{R}^d$14 bounded below and locally smooth enough, noise with finite variance and independent replications, and sample sizes satisfying the ASTRO-type adaptive sampling condition—ASTRO-DF-type methods produce iterates converging to first-order stationary points in expectation, with iteration complexity roughly matching deterministic trust-region rates up to logarithmic factors, and total sampling cost scaling like $x\in\mathbb{R}^d$15 to reach $x\in\mathbb{R}^d$16 (Ha et al., 5 Aug 2025).

ASTRO-MFDF is designed to preserve the key ingredients underlying that rationale. First, it maintains the same adaptive sampling condition as ASTRO-DF, namely that variance is dominated by trust-region radius. Second, it preserves unbiasedness of high-fidelity estimates through MFMC when Common Random Numbers and control variates are used correctly. Third, all accepted steps are validated through high-fidelity evaluations using MFAS (Ha et al., 5 Aug 2025).

This last point is central to the method’s treatment of low-fidelity bias. The bias of low-fidelity models does not directly affect the convergence proof structure because the accept/reject decision is based on high-fidelity evaluations $x\in\mathbb{R}^d$17, while low-fidelity models influence only the proposal mechanism (Ha et al., 5 Aug 2025). The correlation vector and success-ratio updates then form a feedback loop that downweights unhelpful fidelities over time. This suggests an asymptotic regime in which the method behaves similarly to a high-fidelity ASTRO-DF algorithm, but with improved finite-time behavior when lower fidelities are genuinely informative.

Potential misconceptions arise here. One is that ASTRO-MFDF assumes globally accurate low-fidelity surrogates; it does not. The selection mechanism is local and performance-based. Another is that low-fidelity information can be accepted without high-fidelity confirmation; it cannot, because successful progress is always evaluated against the high-fidelity objective (Ha et al., 5 Aug 2025).

6. Numerical experiments and empirical behavior

The reported experiments use the SimOpt library and compare ASTRO-MFDF against ASTRO-DF and Nelder-Mead variants (Ha et al., 5 Aug 2025). Two test families are considered: a stochastic multi-fidelity Rosenbrock problem and a continuous $x\in\mathbb{R}^d$18 inventory problem.

For the stochastic multi-fidelity Rosenbrock family in dimension $x\in\mathbb{R}^d$19, the deterministic fidelities are

$x\in\mathbb{R}^d$20

$x\in\mathbb{R}^d$21

$x\in\mathbb{R}^d$22

Stochasticity is introduced by Gaussian noise, with fidelity-specific constructions based on shared and independent noise components, and the per-sample cost vector is $x\in\mathbb{R}^d$23 (Ha et al., 5 Aug 2025).

In the two-dimensional example, the high-fidelity optimum is at $x\in\mathbb{R}^d$24, while the low-fidelity optima lie elsewhere, illustrating bias (Ha et al., 5 Aug 2025). Under a budget of 500 high-fidelity oracle equivalent calls from the starting point $x\in\mathbb{R}^d$25, ASTRO-DF after 11 iterations converges to approximately $x\in\mathbb{R}^d$26 with $x\in\mathbb{R}^d$27, whereas ASTRO-MFDF completes 24 iterations and converges to $x\in\mathbb{R}^d$28 with $x\in\mathbb{R}^d$29 (Ha et al., 5 Aug 2025). The trajectory behavior shows ASTRO-MFDF initially exploiting the lowest fidelity $x\in\mathbb{R}^d$30, then shifting to $x\in\mathbb{R}^d$31 when $x\in\mathbb{R}^d$32 stops producing improving steps, and later reusing $x\in\mathbb{R}^d$33 when it becomes useful again. In the same experiment, $x\in\mathbb{R}^d$34 is rarely used because $x\in\mathbb{R}^d$35 is poorly correlated with $x\in\mathbb{R}^d$36 along the observed trajectory, so $x\in\mathbb{R}^d$37 remains low (Ha et al., 5 Aug 2025).

For $x\in\mathbb{R}^d$38 and $x\in\mathbb{R}^d$39, with a budget equivalent to 5000 high-fidelity calls, ASTRO-MFDF shows faster convergence than ASTRO-DF, measured by lower objective values at the same budget with 95% confidence intervals (Ha et al., 5 Aug 2025). The reported explanation is that, as dimension grows, building high-fidelity models requires $x\in\mathbb{R}^d$40 design points and high sample sizes, making cheaper low-fidelity evaluations increasingly economical.

The second experimental class is continuous $x\in\mathbb{R}^d$41 inventory control, where the decision variables are reorder point $x\in\mathbb{R}^d$42 and order-up-to level $x\in\mathbb{R}^d$43, the objective is expected total cost including holding, ordering, and backlog costs, demand per period is exponential with mean $x\in\mathbb{R}^d$44, and lead time is Poisson with mean $x\in\mathbb{R}^d$45 (Ha et al., 5 Aug 2025). Multi-fidelity is generated by using different simulation run lengths: fidelity $x\in\mathbb{R}^d$46 uses 100 days, fidelity $x\in\mathbb{R}^d$47 uses 50 days, fidelity $x\in\mathbb{R}^d$48 uses 30 days, with cost ratios $x\in\mathbb{R}^d$49 (Ha et al., 5 Aug 2025). The noisy objective is described as highly irregular with small sample sizes, producing rough estimated landscapes.

In a representative experiment with $x\in\mathbb{R}^d$50, $x\in\mathbb{R}^d$51, starting point $x\in\mathbb{R}^d$52, and budget 1000 high-fidelity equivalent calls, ASTRO-DF converges to $x\in\mathbb{R}^d$53, while ASTRO-MFDF converges around $x\in\mathbb{R}^d$54, a region with lower estimated cost when evaluated with a large number of samples (Ha et al., 5 Aug 2025). The reported interpretation is that small trust regions can trap an algorithm in poor regions because the local estimated surface is misleading, whereas ASTRO-MFDF keeps a larger high-fidelity trust region while using low-fidelity models to explore more broadly (Ha et al., 5 Aug 2025).

Across multiple inventory instances with $x\in\mathbb{R}^d$55, $x\in\mathbb{R}^d$56, and two different starting points, the SimOpt evaluation protocol uses 20 macro-replications per solver and problem and 200 additional replications to estimate final performance (Ha et al., 5 Aug 2025). The reported metric is the fraction of problem instances solved within a 10% optimality gap threshold. ASTRO-MFDF solves a larger fraction of problems than ASTRO-DF and Nelder-Mead in most settings, and in some cases finds better solutions as well as converging faster (Ha et al., 5 Aug 2025).

7. Practical scope, limitations, and relation to adjacent literature

ASTRO-MFDF is most appropriate when multiple simulation models of differing fidelity and cost are available, high-fidelity simulations are expensive, local correlation between fidelities exists at least in some regions, and only noisy function values can be obtained (Ha et al., 5 Aug 2025). It is less beneficial when low-fidelity models are extremely poorly correlated with high fidelity everywhere, in which case the method effectively reduces to ASTRO-DF as the correlation weights decay. It may also be less effective when simulation noise is so large that local models are difficult to fit or when dimensionality is so high that even low-fidelity model construction with $x\in\mathbb{R}^d$57 design points becomes too costly (Ha et al., 5 Aug 2025).

The paper identifies the main hyperparameter classes but does not provide a full tuning guide. These include trust-region parameters such as initial radii $x\in\mathbb{R}^d$58 and $x\in\mathbb{R}^d$59, expansion and contraction factors $x\in\mathbb{R}^d$60 and $x\in\mathbb{R}^d$61, the acceptance threshold $x\in\mathbb{R}^d$62, and certification parameter $x\in\mathbb{R}^d$63; adaptive-sampling parameters such as $x\in\mathbb{R}^d$64, $x\in\mathbb{R}^d$65, and $x\in\mathbb{R}^d$66; and correlation parameters such as initial $x\in\mathbb{R}^d$67, threshold $x\in\mathbb{R}^d$68, and the update factors $x\in\mathbb{R}^d$69 (Ha et al., 5 Aug 2025). The paper states that one might start from ASTRO-DF defaults and tune multi-fidelity-specific parameters through pilot runs by monitoring the frequency of low-fidelity model acceptance (Ha et al., 5 Aug 2025). Since that statement is framed as practical advice rather than theorem-backed prescription, it is best interpreted as heuristic usage guidance.

Several limitations are explicit. The method is designed for unconstrained problems. No explicit finite-time theoretical guarantees are proved for the multi-fidelity extension. The MFMC subproblem relies on variance and covariance estimates that may themselves be noisy, particularly early in the run. The method also assumes that solving the small optimization problem for MFMC allocation is negligible relative to simulation time (Ha et al., 5 Aug 2025).

Within the literature, ASTRO-MFDF is positioned as a direct extension of ASTRO-DF and as part of the broader family of stochastic trust-region methods, including STRONG, STORM, and related noisy trust-region methods (Ha et al., 5 Aug 2025). Relative to multi-fidelity Bayesian optimization, it does not build a global multi-output surrogate and does not assume strong global correlation. Instead it builds local trust-region models and uses the local, performance-based $x\in\mathbb{R}^d$70 mechanism to decide whether low fidelity should influence the search (Ha et al., 5 Aug 2025). Relative to other multi-fidelity simulation optimization approaches, its distinguishing combination is a fully derivative-free stochastic trust-region design, adaptive sampling tied to trust-region radius, MFMC for variance reduction, and dynamic feedback-based selection of low-fidelity information (Ha et al., 5 Aug 2025).

This combination defines the central identity of ASTRO-MFDF: a multi-fidelity, simulation-based, derivative-free optimization framework that attempts to balance statistical accuracy, computational cost, and robustness to low-fidelity bias by separating proposal generation from high-fidelity validation and by adapting both sampling effort and fidelity usage online.