--- title: 'ASTRO-MFDF: Adaptive Multi-Fidelity Trust-Region Method' url: https://www.emergentmind.com/topics/astro-mfdf type: topic --- # ASTRO-MFDF: Adaptive Multi-Fidelity Trust-Region Method Searching arXiv for the cited ASTRO-MFDF paper and closely related trust-region work to ground the article. ASTRO-MFDF is an adaptive sampling trust-region method for multi-fidelity simulation optimization that targets expensive stochastic objectives observed only through noisy zeroth-order simulation outputs. It is formulated for problems in which the highest-fidelity objective is an expectation, \(f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)]\), with \(x\in\mathbb{R}^d\), \(f^0\) nonconvex and bounded below, and additional lower-fidelity simulators \(F^i(x,\xi^i)\), \(i=1,\dots,q\), that are cheaper but biased approximations of the high-fidelity response [2508.03901]. The method extends ASTRO-DF into a multi-fidelity setting through a Multi-Fidelity Adaptive Sampling mechanism based on multi-fidelity Monte Carlo, and it is designed to answer two coupled questions: where and how much to sample, and when low-fidelity models are informative enough to guide the search [2508.03901]. ## 1. Problem formulation and motivation ASTRO-MFDF is defined for simulation optimization settings in which the true objective is available only as an expectation over a stochastic simulator and each function evaluation is expensive [2508.03901]. The high-fidelity optimization problem is \[ \min_{x\in\mathbb{R}^d} f^0(x):=\mathbb{E}_{\Xi^0}[F^0(x,\xi^0)], \] where the index \(0\) denotes the highest-fidelity simulation. Lower-fidelity levels \(i=1,\dots,q\) provide cheaper but biased approximations \(F^i(x,\xi^i)\), with corresponding mean responses \(f^i(x):=\mathbb{E}[F^i(x,\xi^i)]\) [2508.03901]. The method operates with zeroth-order stochastic oracles: only noisy function values are available, not gradients. Given \(n\) iid replications at fidelity \(0\), the sample mean and sample variance estimator are \[ \hat{F}^0(x,n)=\frac{1}{n}\sum_{j=1}^n F^0(x,\xi_j^0), \] \[ \widehat{\sigma}^0(x,n)^2=\frac{1}{n}\sum_{j=1}^n\left(F^0(x,\xi_j^0)-\hat{F}^0(x,n)\right)^2, \] and the covariance between fidelity \(0\) and fidelity \(i\) is estimated by \[ \widehat{\sigma}^{0,i}(x,n)=\frac{1}{n}\sum_{j=1}^n \Big(F^0(x,\xi_j^0)-\hat{F}^0(x,n)\Big)\Big(F^i(x,\xi_j^i)-\hat{F}^i(x,n)\Big). \] The decision variables are continuous, and the algorithm is tailored to unconstrained continuous optimization, although the reported experiments use bounded domains [2508.03901]. The motivation for multi-fidelity is computational. High-fidelity runs may require longer horizons, more detailed physics, or more components, whereas lower fidelities can be produced by simplifying the system or shortening the simulation run length in steady-state problems [2508.03901]. Because these models often share structure with the high-fidelity system, they are frequently correlated with \(f^0\), and thus can provide directional information, help build local models with fewer high-fidelity calls, and improve finite-time performance under a fixed budget [2508.03901]. The central difficulty is bias. The low-fidelity expectations \(f^i(x)\) are biased approximations to \(f^0(x)\), and their optima can be far from the high-fidelity optimum. Naive use of low-fidelity data can mislead surrogate construction, drive the search toward regions with poor high-fidelity performance, and slow convergence or prevent discovery of good solutions [2508.03901]. ASTRO-MFDF is specifically constructed to manage this tension through adaptive sampling and correlation-aware fidelity selection. ## 2. Trust-region architecture and model hierarchy At a high level, each iteration \(k\) of ASTRO-MFDF maintains multiple trust-region radii \(\Delta_k^0,\dots,\Delta_k^q\), builds local interpolation models \(M_k^i(x)\) for each fidelity level, uses a correlation vector \(\boldsymbol{\alpha}_k=(\alpha_k^1,\dots,\alpha_k^q)\) to determine which fidelities to trust, invokes Multi-Fidelity Adaptive Sampling to determine sample sizes and whether to use Monte Carlo or multi-fidelity Monte Carlo, chooses a candidate \(X_k^s\), evaluates that candidate in high fidelity with adaptive sampling, and then updates trust regions and correlation measures [2508.03901]. The underlying framework is a stochastic trust-region scheme. At iteration \(k\), the method maintains the current point \(X_k\), a high-fidelity trust-region radius \(\Delta_k^0\), and a local surrogate \(M_k^0(x)\) that approximates \(f^0(x)\) on the ball \[ B(X_k,\Delta_k^0)=\{x:\|x-X_k\|\le \Delta_k^0\}. \] ASTRO-MFDF augments this with radii \(\Delta_k^i\) and models \(M_k^i\) for all fidelities \(i=0,\dots,q\) [2508.03901]. Models are constructed from function values at a design set of \(2d+1\) points in the trust region. The method adopts a derivative-free, model-based trust-region structure in the style of Conn-Gould-Toint and ASTRO-DF, typically using quadratic interpolation or regression models to approximate gradients and Hessians locally and then solving subproblems of the form \[ X_k^{s,i}\approx \arg\min_{\|x-X_k\|\le \Delta_k^i} M_k^i(x). \] After computing a candidate, the method evaluates the high-fidelity function and forms the success ratio \[ \rho_k=\frac{F^0(X_k^0)-F^0(X_k^s)}{M_k^0(X_k^0)-M_k^0(X_k^{s,0})}. \] If \(\rho_k\) exceeds the threshold \(\eta\), the step is accepted and the trust region is expanded; otherwise it is rejected and the trust region is shrunk: \[ (X_{k+1},\Delta_{k+1}^0)= \begin{cases} (X_k^s,\min\{\gamma_1\Delta_k^0,\Delta_{\max}\}) & \text{if } \rho_k \ge \eta \text{ and } \mu \|\nabla M_k^0(X_k)\| \ge \Delta_k^0,\\ (X_k,\gamma_2\Delta_k^0) & \text{otherwise}. \end{cases} \] The parameters satisfy the standard trust-region conditions \(\gamma_1>1\), \(\gamma_2\in(0,1)\), \(\eta\in(0,1)\), and \(\mu>0\) [2508.03901]. A distinctive feature is the simultaneous management of local models at multiple fidelities. This implies that the trust-region logic is not merely replicated across fidelity levels; rather, the method uses those models competitively and selectively. A plausible implication is that the architecture is intended to exploit low-cost local structure when it has predictive value while preserving high-fidelity validation as the arbiter of progress. ## 3. Correlation vector and fidelity selection mechanism The principal mechanism for deciding when lower fidelities should influence the search is the correlation vector \(\boldsymbol{\alpha}_k\), which quantifies the local usefulness of each low-fidelity model [2508.03901]. For each \(i\in\{1,\dots,q\}\), the method maintains a positive scalar \(\alpha_k^i\) and compares it against a user-chosen threshold \(\alpha_{\text{th}}>0\). The interpretation is operational rather than purely statistical. If a model \(M_k^i\) repeatedly proposes candidates that deliver genuine high-fidelity improvement, then fidelity \(i\) is behaving similarly to fidelity \(0\) in the local region and \(\alpha_k^i\) is increased. If it repeatedly fails, \(\alpha_k^i\) is decreased [2508.03901]. The criterion is therefore a behavioral proxy for local usefulness, not an explicit estimation of correlation coefficients. The correlation vector controls two aspects of the algorithm. First, it affects model construction priority. When \(\alpha_k^i\ge \alpha_{\text{th}}\), the algorithm builds a dedicated model \(M_k^i\) using its own design set inside \(B(X_k,\Delta_k^i)\). When \(\alpha_k^i<\alpha_{\text{th}}\), fidelity \(i\) is not prioritized to propose steps; instead, \(M_k^i\) may be built only partially, reusing points sampled for \(M_k^0\), primarily to continue updating \(\alpha_k^i\) [2508.03901]. Second, it affects inner-loop step generation. The algorithm iterates fidelities from lower to higher, invokes a low-fidelity trust-region subroutine ASTRO-LFDF-\(t\), evaluates the resulting candidate in high fidelity, and, if the candidate produces sufficient reduction in \(F^0\), accepts it, expands \(\Delta_k^t\), and increases \(\alpha_k^t\) by multiplication with \(\gamma_1\) [2508.03901]. When all low-fidelity attempts fail, the method falls back to a high-fidelity-driven step in which models for all fidelities are constructed jointly with shared design points and the candidate is chosen primarily using \(M_k^0\). In the joint-model case, each low fidelity \(t\in\{1,\dots,q\}\) is evaluated through the ratio \[ \hat\rho_k^t= \frac{F^0(X_k^0)-F^0(X_k^{s,t})} {\max\{\zeta(\Delta_k^0)^2,\; M_k^t(X_k^0)-M_k^t(X_k^{s,t})\}}, \] where \(\zeta>0\) is a sufficient-reduction constant. If \(\hat\rho_k^t\ge \eta\), then \[ \alpha_{k+1}^t=\gamma_1\alpha_k^t, \] and otherwise \[ \alpha_{k+1}^t=\gamma_2\alpha_k^t. \] This update rule is the screening mechanism by which useful low fidelities are promoted and unhelpful ones are downweighted [2508.03901]. The low-fidelity trust-region subroutine ASTRO-LFDF-\(t\) follows the same logic. If \(\alpha_k^t<\alpha_{\text{th}}\), it immediately sets \(X_k^{s,t}=X_k^0\), effectively declining to propose a low-fidelity-driven move. Otherwise it selects a design set in \(B(X_k^0,\Delta_k^t)\), uses MFAS to estimate \(F^t\) at those points, builds \(M_k^t\), computes \[ X_k^{s,t}\approx \arg\min_{\|X-X_k^0\|\le \Delta_k^t} M_k^t(X), \] evaluates the high-fidelity function at \(X_k^{s,t}\) and \(X_k^0\), and forms the low-fidelity success ratio \[ \rho_k= \frac{F_k^0(X_k^0)-F_k^0(X_k^{s,t})} {\max\{\zeta(\Delta_k^0)^2,\; M_k^t(X_k^0)-M_k^t(X_k^{s,t})\}}. \] If \(\rho_k<\eta\), the subroutine shrinks both the radius and the correlation parameter: \[ \Delta_k^t\leftarrow \gamma_2\Delta_k^t,\qquad \alpha_k^t\leftarrow \gamma_2\alpha_k^t. \] This design ensures that low fidelity can influence candidate generation only when it has demonstrated actual predictive value for high-fidelity improvement [2508.03901]. ## 4. Multi-Fidelity Adaptive Sampling and MFMC integration ASTRO-MFDF uses Multi-Fidelity Adaptive Sampling as its sampling engine whenever it requires function estimates, whether for model construction, incumbent evaluation, or candidate evaluation [2508.03901]. MFAS is guided by the adaptive sampling rule of ASTRO-DF, in which the sample size at a point \(x\) and iteration \(k\) is the smallest integer \(n\) satisfying \[ \frac{\max\{\sigma_{lb},\widehat{\sigma}^0(x,n)\}}{\sqrt{n}} \le \frac{\kappa \Delta_k^{2}}{\sqrt{\lambda_k}}, \] where \(\sigma_{lb}>0\) is a lower bound, \(\kappa>0\) is a tuning constant, and \(\lambda_k=O(\log k)\) is a slowly growing sequence [2508.03901]. The left-hand side is the stochastic error and the right-hand side is proportional to the optimality gap represented by \(\Delta_k^2/\sqrt{\lambda_k}\). As \(\Delta_k\to 0\), the required sample size grows like \(\Delta_k^{-4}\). In the multi-fidelity setting, the method uses the multi-fidelity Monte Carlo estimator. For target fidelity \(t\), lowest fidelity index \(q\), and sample sizes \(n^0<\dots