---
title: 'T2N-MusE: Neural Multifidelity Emulator'
url: https://www.emergentmind.com/topics/t2n-muse
type: topic
---

# T2N-MusE: Neural Multifidelity Emulator

Searching arXiv for the target paper and closely related cosmological emulation work.
arXiv.search query="T2N-MusE cosmological emulation Goku" max_results=5
T2N-MusE is a “Triple-2” neural-network framework for multifidelity cosmological emulation, introduced to replace older Gaussian-process multifidelity emulators for the matter power spectrum in regimes with high-dimensional cosmological parameter spaces and large training sets. Its design combines four coordinated elements—a 2-step multifidelity neural network, a 2-stage Bayesian hyperparameter optimization, a 2-phase \(k\)-fold training strategy, and a per-redshift PCA compression strategy—and is demonstrated on selected data from the Goku simulation suite. In the reported application, the framework is used to build GokuNEmu, described as the most powerful emulator for the matter power spectrum built in that program, with validation error reduced by more than a factor of five compared to previous Gaussian-process work [2507.07184].

## 1. Problem setting and motivation

The framework is motivated by a specific bottleneck in multifidelity cosmological emulation. Multifidelity simulation design can reduce simulation cost by combining low-fidelity and high-fidelity data, but earlier regression stages based on Gaussian processes become difficult to scale as both the training-set size and the cosmological parameter-space dimensionality increase. The paper identifies two limitations in particular: cubic scaling with sample size and degraded performance in higher-dimensional parameter spaces. This matters directly for the Goku suite, which spans a 10-dimensional cosmological parameter space, and for prospective extensions that may include additional physics such as nonstandard dark matter or baryonic effects [2507.07184].

T2N-MusE targets emulation of the nonlinear matter power spectrum \(P(z,k)\). The input is a 10-dimensional cosmological parameter vector,
\[
\mathbf{x}\in \mathbb{R}^{10},
\]
normalized in practice to \([-0.5,0.5]\). The output is the log power spectrum,
\[
y(z_i,k_j)=\lg P(z_i,k_j),
\]
assembled into a vector over redshift and wavenumber. For Goku-W, the configuration is \(n_z=6\), \(n_k=64\), and therefore \(d_{\rm out}=384\). The central objective is to emulate this high-dimensional output accurately while exploiting the correlation between low-fidelity and high-fidelity simulations.

A recurrent misconception is that T2N-MusE is simply a replacement of Gaussian processes by generic neural networks. The paper presents it differently: accuracy and practicality are attributed to the joint optimization of architecture, hyperparameter search, cross-validation procedure, and output compression rather than to the mere use of neural networks.

## 2. Core framework and multifidelity architecture

The framework’s four defining components are stated explicitly in the paper.

| Component | Function |
|---|---|
| 2-step multifidelity neural network | Learn LF mapping, then HF correction |
| 2-stage Bayesian hyperparameter optimization | Coarse search, then local fine-tuning |
| 2-phase \(k\)-fold training | Warm-start LF folds from a shared minimum |
| per-redshift PCA compression | Compress outputs redshift by redshift |

The multifidelity architecture is the central structural modification. In the original 2-step setup, one neural network \(NN_{\rm L}\) learns the low-fidelity map \(\mathbf{y}^{\rm L}=f^{\rm L}(\mathbf{x})\), and a second network \(NN_{\rm LH}\) takes both \((\mathbf{x},\mathbf{y}^{\rm L})\) as input to predict the high-fidelity output. T2N-MusE alters the second stage so that it learns a correction ratio rather than a direct LF-to-HF map. The second network learns
\[
\mathbf{r}=\mathcal{G}(\mathbf{x}),
\]
with training targets defined by element-wise division of the high-fidelity output by the low-fidelity output at shared cosmologies. The final high-fidelity prediction is
\[
\mathbf{y}^{\rm H}_{\rm NN}=\mathcal{G}_{\rm NN}(\mathbf{x})\odot f^{\rm L}_{\rm NN}(\mathbf{x}),
\]
where \(\odot\) denotes element-wise multiplication [2507.07184].

This modification has a specific computational rationale. In the original architecture, the second network receives an input of size \(d_{\rm in}+d_{\rm out}\), which is large because \(d_{\rm out}\gg d_{\rm in}\) for the matter power spectrum. The modified architecture reduces the second-network input to \(d_{\rm in}\) only. The paper reports that this architectural change alone reduces the correction-network error by about a factor of 3. In the context of power-spectrum emulation, this is significant because the output dimension is large and the high-fidelity sample count is small.

## 3. Hyperparameter optimization and training procedure

T2N-MusE uses Bayesian optimization via Hyperopt rather than manual tuning. The search covers three hyperparameters: the number of hidden layers \(L\), the number of neurons per hidden layer \(M\), and the \(L_2\) regularization strength \(\lambda\). The optimized objective is a regularized loss,
\[
\mathcal{L}(\mathbf{W},\mathbf{b})=\mathcal{L}_{\rm train}(\mathbf{W},\mathbf{b})+\lambda\|\mathbf{W}\|_2^2,
\]
where the training loss is mean squared error. During hyperparameter selection, the paper uses a \(k\)-fold objective that averages minimized training and validation losses across folds [2507.07184].

The search itself is split into two stages. Stage 1 is a coarse global search over
\[
L\sim \mathcal{U}(\{1,2,3,4,5,6,7\}),
\]
\[
M\sim \mathcal{U}(\{16,32,48,\ldots,512\}),
\]
and
\[
\lambda\sim \mathcal{LU}(10^{-9},5\times10^{-6}),
\]
using \(n_{\rm trial}=80\) trials in the main comparison. Stage 2 fixes \(L\) to the best stage-1 value, narrows the search for \(M\) to a discrete window around the stage-1 optimum, narrows \(\lambda\) to \(\mathcal{LU}(\lambda_1/2,2\lambda_1)\), and uses \(n_{\rm trial}^{\rm tune}=40\) trials. The reported effect is modest but positive: the first stage identifies a good region in hyperparameter space, and the second stage refines it more effectively than simply adding more single-stage trials.

The training strategy is also split in two, but only for the low-fidelity network. For the high-fidelity correction network, the high-fidelity set has only \(n_{\rm H}=21\) samples, so the paper uses leave-one-out cross-validation with \(k=n_{\rm H}\). Each fold is trained with \(n_{\rm seed}^{\rm LH}=5\) random seeds, and the best run is kept. For the low-fidelity network, where \(n_{\rm L}=564\), the authors introduce a 2-phase \(k\)-fold strategy. Phase 1 trains on the low-fidelity data excluding all high-fidelity cosmologies and uses the high-fidelity cosmologies as a validation-like set; this stage is repeated with \(n_{\rm seed}^{\rm L}=15\) random initializations to find a good local minimum. Phase 2 initializes each fold of the actual cross-validation from that phase-1 model and starts from the final phase-1 learning rate. The stated purpose is to force all fold models into the same local minimum region so that cross-validation error better approximates the behavior of the final model trained on all low-fidelity data.

Additional implementation details are reported explicitly. The activation is SiLU,
\[
\mathrm{SiLU}(x)=x\,\sigma(x)=\frac{x}{1+e^{-x}},
\]
the optimizer is AdamW, though the paper notes that with explicit \(L_2\) regularization the effective behavior is like Adam, and the learning rate is dynamically decreased when the combined training-plus-validation loss fails to improve for a patience interval. After hyperparameter tuning, the final low-fidelity model is initialized from the fold model with median regularized loss, and a safeguard prevents the final training loss from falling below \(80\%\) of the median training loss across folds.

## 4. Output representation and per-redshift PCA

Before training either neural network, T2N-MusE compresses the output with PCA. The paper compares a global PCA basis over all redshifts and wavenumbers with a local, per-redshift PCA basis. In the global scheme,
\[
y(z_i,k_j;\mathbf{x})=\mu(z_i,k_j)+\sum_{l=1}^{n_{\rm PCA}} a_l(\mathbf{x})\,\phi_l(z_i,k_j),
\]
and the compressed output is the vector of PCA coefficients. In the per-redshift scheme,
\[
y(z_i,k_j;\mathbf{x})=\mu^i(k_j)+\sum_{l=1}^{n_{\rm PCA}^i} a_l^i(\mathbf{x})\,\phi_l^i(k_j),
\]
and the compressed representation is the concatenation of the redshift-specific coefficient vectors [2507.07184].

The number of principal components is chosen as the smallest number for which the unexplained variance is below \(10^{-5}\). The paper’s argument for local PCA is physical as well as numerical: redshift evolution in the matter power spectrum is nonlinear, especially at low redshift where nonlinear structure growth is stronger, so a single global basis is too rigid to capture redshift-specific patterns efficiently. The reported result is that local PCA improves emulator accuracy by more than 10% relative to global PCA in both the low-fidelity network and the correction network.

This compression step is integral to the framework rather than ancillary preprocessing. Because the output dimension is large and the multifidelity correction stage is trained on only 21 high-fidelity cosmologies in the Goku-W study, reducing the effective output dimension while preserving redshift-dependent structure directly affects learnability.

## 5. Experimental setup on the Goku simulation suite

The demonstration is performed on the Goku-W subset of the Goku simulation suite, generated with MP-Gadget. The low-fidelity simulations use box size \(250\,{\rm Mpc}/h\), particle load \(750^3\), and \(n_{\rm L}=564\). The high-fidelity simulations use box size \(1000\,{\rm Mpc}/h\), particle load \(3000^3\), and \(n_{\rm H}=21\). The high-fidelity cosmologies are a subset of the low-fidelity cosmologies. The emulated redshifts are
\[
z = 0,\ 0.2,\ 0.5,\ 1,\ 2,\ 3.
\]

Because Goku-W does not provide an independent high-fidelity validation split, the main study uses leave-one-out cross-validation on the high-fidelity cosmologies. The paper also checks representativeness against a separate test-set study using the earlier Goku-pre-N simulation set, where the high-fidelity set contains 27 training cosmologies and 12 test cosmologies. The appendix reports that leave-one-out cross-validation error and separate test error are consistent, with the test error slightly smaller [2507.07184].

The paper organizes its ablation study around several named configurations. “Base” uses the original 2-step architecture, global PCA, 1-stage hyperparameter optimization, and 1-phase low-fidelity training. “Arch-0” uses the original 2-step architecture with local PCA. “PCA-0” uses the modified 2-step architecture with global PCA. “Mid” combines the modified 2-step architecture and local PCA but without 2-stage hyperparameter optimization and without 2-phase low-fidelity training. “NNL-1” adds 2-phase low-fidelity training to Mid. “NNL-0+” retains regular 1-phase low-fidelity training but increases the number of random seeds. “Optimal” combines the modified 2-step architecture, local PCA, 2-stage hyperparameter optimization, and 2-phase low-fidelity training.

## 6. Results, interpretation, and scope

The main quantitative claim is that the optimized neural multifidelity emulator significantly outperforms the earlier Gaussian-process emulator. The earlier GP-based GokuEmu had an error of about \(3\%\). Even the simplest neural model, Base, is reported to perform significantly better. The key mean validation errors reported in the ablation sequence are:
- Base: \(1.73\%\)
- Mid: \(1.03\%\)
- NNL-1: \(0.55\%\)
- Optimal: \(0.62\%\) in the main text summary

The paper also emphasizes that the overall validation error is reduced by more than a factor of five relative to previous GP-based work, and notes a nuance in the reported numbers while stressing that the strongest comparison shows approximately a factor-of-five improvement in the worst case [2507.07184].

The ablations attribute distinct roles to the framework’s components. Comparing Arch-0 to Mid shows that the modified correction network is easier to learn, with the average correction-network error dropping by roughly a factor of 3. Comparing PCA-0 to Mid shows that per-redshift PCA improves both component networks, with especially visible gains at \(z=0\). Comparing Mid, NNL-0+, and NNL-1 shows that 2-phase low-fidelity training is more effective than merely increasing the number of random seeds in ordinary training; the paper reports overall low-fidelity-network error of approximately \(0.97\%\) for Mid and approximately \(0.55\%\) for NNL-1. The 2-stage Bayesian optimization contributes a smaller additional gain beyond Mid.

The reported error structure is also physically interpretable. Errors are largest at low redshift and small scales, where nonlinear structure growth is strongest. This suggests that the remaining difficulty is concentrated in precisely the regime where the target statistic deviates most strongly from linear behavior. A plausible implication is that the framework’s engineering choices are particularly important because they stabilize learning where both the physics and the emulator geometry are least forgiving.

In practical terms, the paper presents T2N-MusE as a general framework rather than a one-off emulator. Its immediate output is GokuNEmu for the nonlinear matter power spectrum, but the intended scope includes other cosmological statistics. The paper explicitly mentions future application to the Lyman-\(\alpha\) forest flux power spectrum. Within that framing, T2N-MusE is best understood not as a single network architecture but as a complete emulation pipeline in which ratio-based multifidelity modeling, staged optimization, staged fold training, and redshift-local compression are treated as mutually dependent components.

Source: https://www.emergentmind.com/topics/t2n-muse