---
title: 'eSEM Models: Multidisciplinary Perspectives'
url: https://www.emergentmind.com/topics/esem-models
type: topic
---

# eSEM Models: Multidisciplinary Perspectives

The expression **eSEM models** is not a single canonical designation. In current arXiv usage it refers to several technically unrelated constructs: an **exploratory version of structural equation modeling** for directed dependencies in resting-state fMRI; the **Edge Saliency Enhancement Module** inside an RGB–thermal depth-estimation architecture; and a family of **eSEN-initialized property-prediction regressors** for magnetic materials screening. Closely related acronymic usages also appear in neighboring literatures, notably **semi-empirical models** in galaxy formation and **ESEm**, an Earth System Emulator for calibration and uncertainty quantification [1602.06827], [2503.04821], [2508.20556], [2502.12764], [2108.10747].

## 1. Terminological scope and disambiguation

The main difficulty in discussing eSEM models is lexical rather than mathematical: the same string is used for different model classes, different levels of abstraction, and different scientific domains.

| Term | Domain | Meaning |
|---|---|---|
| eSEM | Neuroimaging | Exploratory Structural Equation Modeling |
| ESEM | RGB–thermal depth estimation | Edge Saliency Enhancement Module |
| eSEM | Materials informatics | eSEN-initialized single-task regressors |
| SEMs | Galaxy formation | Semi-empirical models |
| ESEm | Earth system science | Earth System Emulator |

In the neuroimaging usage, eSEM is a **multivariate, data-driven framework** that infers directed dependencies among ROI time series without imposing an a priori connectivity pattern. In the computer-vision usage, ESEM is not a standalone predictor but a **structure-aware component** inside the proposed Edge-Guided Fusion pipeline. In the materials-informatics usage, eSEM denotes **single-task regressors** obtained by frozen transfer learning from the pretrained eSEN-30M-OAM interatomic potential. By contrast, the galaxy-formation literature standardizes on **SEMs**, and Earth-system calibration uses **ESEm** as a software framework rather than a single statistical model [1602.06827], [2503.04821], [2508.20556], [2502.12764], [2108.10747].

This suggests that any encyclopedic treatment of eSEM models must be domain-specific. The acronym alone does not determine architecture, objective, or epistemic status.

## 2. Exploratory structural equation modeling in neuroimaging

In resting-state neuroimaging, eSEM is an **exploratory version of structural equation modeling** used to infer directed dependencies among fMRI signals and relate them to dMRI-derived structural connectivity. Traditional SEM specifies a hypothesized network structure in advance and estimates whether that prespecified model reproduces the observed covariance structure. The exploratory SEM used here **dispenses with a priori constraints on the connectivity pattern**: it is saturated and fully connected among all observed variables, estimating directed influences between every pair while controlling for the rest [1602.06827].

For the instantaneous formulation, the core structural equation is
$$
x = Bx + \varepsilon,
$$
with implied covariance
$$
\Sigma(B,\Psi) = (I-B)^{-1}\Psi\big[(I-B)^{-1}\big]^\top.
$$
Here the path matrix $B$ contains contemporaneous directed influences among the observed ROI signals. The model has no measurement model and no exogenous variables, and parameters are estimated by maximum likelihood in `lavaan`. In the reported application, the structural part is a saturated path model over **15 regions**, and the diagonal of the influence matrix is kept at zero in the reported results [1602.06827].

The lagged formulation augments the variable set with lagged ROI time series and removes contemporaneous inter-ROI regressions. Its generic form is
$$
x_t = B_0x_t + \sum_{\ell=1}^{p} B_\ell x_{t-\ell} + \varepsilon_t,
$$
with cross-lagged matrices $B_\ell$ encoding directed influences at lags $\ell=1,2,3$. The study defines **eSEM1**, **eSEM2**, and **eSEM3** by including lag sets $\{1\}$, $\{1,2\}$, and $\{1,2,3\}$ respectively, with observed-variable counts of **30**, **45**, and **60**. These lagged models are compared with conditioned multivariate Granger causality of orders **GC1–3** [1602.06827].

The empirical setting consists of **12 healthy adults** scanned on a **Philips Achieva 1.5T Nova scanner**. Structural connectivity is defined by **fiber counts between ROI pairs**, while functional or effective connectivity is estimated with eSEM, linear correlation, partial correlation, and lagged methods. The main quantitative result is that structural connectivity correlates significantly with each instantaneous estimate: **0.44** for eSEM, **0.43** for linear correlation, and **0.50** for partial correlation. On structurally connected pairs, eSEM is highly concordant with the undirected baselines, with **corr(eSEM, C) = 0.86** and **corr(eSEM, PC) = 0.93**. By contrast, lagged eSEM and Granger causality do **not** significantly distinguish connected from non-connected pairs, and their alignment with structural connectivity is markedly poorer [1602.06827].

The conceptual significance of this result is precise. Instantaneous methods probe interactions on time scales much shorter than the fMRI sampling interval, whereas lagged methods probe directed influences at **1–3 TRs**, corresponding here to **2.2–6.6 s**. The study concludes that **instantaneous functional influences** are much more related to structural connectivity than **slow directed influences**. A common misconception is to read the asymmetric instantaneous path coefficients as mechanistic causality. The source explicitly limits that interpretation: contemporaneous directed paths are **statistical, not mechanistic causality**, and the saturated instantaneous model can overfit noise [1602.06827].

## 3. Edge Saliency Enhancement Module in RGB–thermal depth estimation

In RGB–thermal depth estimation, **ESEM** stands for **Edge Saliency Enhancement Module**. It is the structure-aware component inside the proposed **Edge-Guided Fusion (EGFusion)** pipeline of **RTFusion**, a model that fuses RGB and thermal infrared modalities to produce robust depth maps in conditions such as nighttime, rain, fog, and glare. EGFusion contains two parts: **Mutual Complementary Attention (MCA)** for cross-modal feature alignment and **ESEM** for boundary-preserving fusion [2503.04821].

The architecture uses **dual feature extractors**, specifically **ConvNeXt-Tiny backbones with non-shared parameters**, to generate hierarchical RGB and THR features. MCA first produces a cross-attended RGB representation, after which ESEM computes an edge-weight map from thermal features and uses it to modulate the local thermal contribution before fusion. The explicit ESEM equations are
$$
E_L = \sigma\!\left(W_2 \cdot \mathrm{ReLU}(W_1 \cdot F_{THR})\right),
$$
$$
F_{\mathrm{enhanced}} = F_{\mathrm{cross}} + E_L \odot F_{THR},
$$
$$
F_{\mathrm{fused}} = \mathrm{FusionConv}(F_{\mathrm{enhanced}}).
$$
Here $W_1$ is a **3×3 convolution** and $W_2$ is a **1×1 convolution**. ReLU follows $W_1$, Sigmoid follows $W_2$, and the learned map $E_L \in [0,1]$ acts as an edge-aware gate [2503.04821].

The design motivation is structural. Object boundaries are where depth discontinuities occur, yet RGB edges become unreliable under over-exposure, under-exposure, and noise, while thermal imagery retains heat signatures but often has coarser spatial resolution and weaker fine texture. ESEM therefore uses **thermal gradients to control how much THR contributes locally**, boosting contribution in edge regions and suppressing it in homogeneous areas. This is a learned alternative to handcrafted edge masks such as Sobel or Canny [2503.04821].

Training combines depth regression with edge-aware smoothness:
$$
L_{1} = \frac{1}{N}\sum_{i=1}^{N} \left| D_{\mathrm{pred}(i)} - D_{\mathrm{gt}(i)} \right|,
$$
$$
L_{\mathrm{smooth}} = \sum_{i=1}^{N} \left\| \nabla D_{\mathrm{pred}(i)} \right\| \cdot \exp\!\left( - \left\| \nabla I(i) \right\| \right),
$$
$$
L_{\mathrm{total}} = \lambda_{1}L_{1} + \lambda_{2}L_{\mathrm{smooth}}.
$$
The paper does **not** add explicit edge supervision or cross-modal consistency losses beyond MCA and ESEM [2503.04821].

The ablation evidence isolates ESEM’s effect on the **MS2** dataset. The **without enhancement** setting yields **AbsRel 0.119, SqRel 1.055, RMSE 5.228, $\delta_1$ 0.869**; **Sobel-based** yields **AbsRel 0.111, SqRel 1.014, RMSE 5.083, $\delta_1$ 0.891**; **Canny-based** yields **AbsRel 0.115, SqRel 1.018, RMSE 5.175, $\delta_1$ 0.887**; and **ESEM** yields **AbsRel 0.104, SqRel 0.942, RMSE 4.997, $\delta_1$ 0.895**. The full **RTFusion (RGB+THR)** model on MS2 reports **AbsRel 0.104, SqRel 0.942, RMSE 4.997, RMSE(log) 0.186, $\delta_2$ 0.974, $\delta_3$ 0.991**, while on **ViViD++** it achieves **AbsRel 0.052** [2503.04821].

A recurrent misunderstanding is to treat ESEM as the entire model. It is not: ESEM is a **mid-level fusion module** placed after MCA at each scale, and its output is passed to a **UNet-like decoder**. The paper also notes clear limitations: if thermal contrast is weak or noisy, the edge map may be unreliable; if RGB and THR are misaligned, MCA and ESEM may propagate misregistered edge cues; and the paper does not report parameter counts, FLOPs, or latency [2503.04821].

## 4. eSEM as eSEN-initialized regressors for Heusler screening

In magnetic-materials discovery, **eSEM** refers to a family of **property-prediction models** built by transfer learning from the pretrained **eSEN-30M-OAM** machine-learning interatomic potential. The paper defines them conceptually as **expressively Smooth, eSEN-initialized Models**: single-task regressors that reuse learned chemical and structural representations in eSEN and add a task-specific output head. Their purpose is distinct from that of the underlying MLIP. **eSEN-30M-OAM** performs geometry optimization and predicts formation energy and distance to the convex hull; **eSEM** predicts higher-level physical properties from optimized structures [2508.20556].

The eSEM suite contains **four separate single-task models**. They predict **local magnetic moments $\{m_i\}$**, **minimum phonon frequency $\omega_{\min}$**, **magnetic critical temperature $T_c$**, and **magnetocrystalline anisotropy energy $E_{\mathrm{aniso}}$**. All are trained as continuous regressors and then post-processed with thresholds for screening: $\omega_{\min} > -10\ \mathrm{cm}^{-1}$, $T_c > 300\ \mathrm{K}$, and $|E_{\mathrm{aniso}}| > 1\ \mathrm{MJ/m^3}$. For magnetic identification, the workflow uses $\Sigma |m_i| > 0.1\ \mu_B/\mathrm{f.u.}$ [2508.20556].

The training base is the **DxMag Computational Heusler Database**. Reported dataset sizes are **27,864** entries for local magnetic moments, **8,198** for minimum phonon frequency, **2,106** for magnetic critical temperature, and **6,123** for magnetocrystalline anisotropy energy, with a random **8:1:1** train/validation/test split. Architecturally, the models use the eSEN graph message-passing network as a frozen feature extractor. The **embedding plus first seven message-passing layers** are kept fixed, while the **last three message-passing layers** and the **task-specific output head** are fine-tuned. Regression uses **mean squared error**, and the magnetic-moment model imposes **global spin-inversion invariance** by evaluating the loss on both the predicted moments and their sign-inverted counterpart and taking the smaller value [2508.20556].

The freezing-depth study is central. Performance improves as more layers are frozen, peaks at **TL-MLIP-7**, and then degrades when additional layers are frozen. Final reported test-set $R^2$ values with **TL-MLIP-7** are **0.990** for $\{m_i\}$, **0.986** for $m_{\mathrm{total}}$, **0.990** for $\Sigma |m_i|$, **0.750** for $\omega_{\min}$, **0.910** for $T_c$, and **0.680** for $E_{\mathrm{aniso}}$. Training from scratch yields consistently lower $R^2$, and a model initialized only from DxMag formation energies underperforms full eSEN initialization [2508.20556].

The models are embedded in a machine-learning-accelerated high-throughput workflow. **eSEN-30M-OAM** optimizes structures, computes $\Delta E$ and $\Delta H$, and screens tetragonality via $|c/a - 1| > 0.01$. The eSEM regressors then evaluate magnetism, dynamical stability, magnetic stability, and strong anisotropy. The required screening conditions are: **tetragonality**, **$\Delta E < 0\ \mathrm{eV/atom}$**, **$\Delta H < 0.22\ \mathrm{eV/atom}$**, **$\Sigma|m_i| > 0.1\ \mu_B/\mathrm{f.u.}$**, **$\omega_{\min} > -10\ \mathrm{cm}^{-1}$**, **$T_c > 300\ \mathrm{K}$**, and **$|E_{\mathrm{aniso}}| > 1\ \mathrm{MJ/m^3}$** [2508.20556].

The validation scale is large. The workflow screens **131,544 conventional quaternary** and **104,139 all-$d$ Heuslers**, yielding **366** and **924** candidates respectively. DFT validation confirms high precision: **100%** of ML-selected candidates remain tetragonal; **$\Delta E_{\mathrm{DFT}} < 0$** in **99.1%** of quaternary and **97.8%** of all-$d$ cases; **$\Delta H_{\mathrm{DFT}} < 0.22\ \mathrm{eV/atom}$** in **96.4%** and **98.8%**; **$\omega_{\min} > -10\ \mathrm{cm}^{-1}$** in **89.2%** and **93.1%**; **$T_c > 300\ \mathrm{K}$** in **81.7%** and **80.4%**; and strong MAE validation reaches **82.0%** and **68.2%** [2508.20556].

A critical distinction is often missed: **eSEM does not predict forces or optimize structures**. Those tasks belong to **eSEN-30M-OAM**. Another important limitation is domain shift. The paper reports reduced performance for **$E_{\mathrm{aniso}}$** on all-$d$ Heuslers because $Z$-site elements from the $d$ block were absent in training, altering SOC and electronic environments in ways that are harder to learn from limited data [2508.20556].

## 5. Related acronymic traditions: SEMs in galaxy formation and ESEm in Earth-system science

The wider literature contains nearby abbreviations that are conceptually distinct from the three eSEM usages above. In galaxy formation, the standard term is **semi-empirical models (SEMs)**, defined as **effective, data-driven frameworks** that connect the statistical evolution of galaxies to the growth of their dark-matter halos using empirically calibrated relations rather than *ab initio* baryonic physics. These models come in **interpretative**, **descriptive**, and **hybrid** forms, with exemplars such as **DECODE**, **EMERGE**, **UNIVERSEMACHINE**, and **TOPSEM** [2502.12764].

Their standard pipeline begins with a halo backbone, either an analytic halo mass function or a halo catalog and merger tree from an N-body simulation. Halos are then populated using abundance matching, occupancy models, or conditional stellar mass functions, and galaxy properties are evolved by linking star formation to halo growth, assigning quenching, and optionally partitioning bulges and disks. A representative stellar-to-halo mapping is
$$
M_\star(M_{\mathrm{h}}, z)=\frac{2\,\epsilon(z)\,M_{\mathrm{h}}}{\left(\frac{M_{\mathrm{h}}}{M_c(z)}\right)^{-\alpha(z)}+\left(\frac{M_{\mathrm{h}}}{M_c(z)}\right)^{\beta(z)}}.
$$
The review emphasizes high computational efficiency, interpretability, and empirical grounding, but also notes limited physical self-consistency, calibration dependence, and extrapolation risks [2502.12764].

In Earth-system science, **ESEm v1.0.0** denotes an **open-source Python framework** for emulation, validation, uncertainty quantification, and model calibration. It standardizes data preparation, design of experiments, emulator construction, and sampling using **Gaussian Processes**, **convolutional neural networks**, and **random forests**, together with **Approximate Bayesian Computation** and **Hamiltonian Monte Carlo**. The framework approximates an expensive simulator $f$ by an emulator $\hat f$ such that
$$
\hat{f}(\boldsymbol{\theta}) \approx f(\boldsymbol{\theta}),
$$
and propagates predictive, observational, representation, and structural uncertainty through the total variance
$$
\sigma^2=\sigma^2_{\text{emu}}+\sigma^2_{\text{obs}}+\sigma^2_{\text{rep}}+\sigma^2_{\text{str}}.
$$
Its case studies include calibration of **ECHAM6.3-HAM2.3**, a precipitation emulator for a cloud-resolving model, and a GP emulator for **CMIP6** scenario uncertainty [2108.10747].

These traditions matter because they show that eSEM-like nomenclature spans at least three levels: a **statistical model class**, a **neural module**, and a **software framework**. A plausible implication is that literature searches using the string “eSEM” require immediate domain disambiguation.

## 6. Common methodological themes, interpretation, and recurrent misconceptions

Despite their heterogeneity, the principal eSEM usages share a methodological pattern: each replaces or augments a more limited baseline with a structured, empirically anchored mechanism. In neuroimaging, eSEM augments correlation-based functional connectivity by providing **directionality** while remaining comparable to correlation and partial correlation in its alignment with structural connectivity [1602.06827]. In RGB–thermal fusion, ESEM augments naïve fusion and handcrafted edge preservation by learning a **THR-guided edge-weight map** that modulates cross-modal contribution at boundaries [2503.04821]. In materials informatics, eSEM augments direct training on small target datasets by reusing large-scale eSEN representations through **frozen transfer learning** [2508.20556].

The main interpretive cautions are domain-specific. In neuroimaging, the asymmetric path coefficients of instantaneous eSEM are often over-read as causal arrows, but the study explicitly characterizes them as **statistical** rather than mechanistic causality and shows that lagged analyses at fMRI sampling rates are much less aligned with structural connectivity [1602.06827]. In depth estimation, ESEM is sometimes described as if it were the full model, yet the source defines it as one component of **EGFusion**, downstream of MCA and upstream of the decoder [2503.04821]. In magnetic materials discovery, eSEM may be conflated with the eSEN interatomic potential, although the paper sharply separates **optimization and energetics** from **property prediction** [2508.20556].

A second recurring issue is calibration dependence. Galaxy-formation SEMs are explicitly calibrated to stellar mass functions, main sequences, quenched fractions, clustering, and lensing; their virtue is empirical grounding, but their weakness is dependence on the quality and consistency of those inputs [2502.12764]. ESEm in Earth-system science similarly emphasizes that underestimating structural uncertainty yields over-confident posteriors, and that collocation and summary-statistic design materially affect calibration results [2108.10747].

The most general lesson is therefore taxonomic. **eSEM models** are best understood not as one research program but as a family of acronymically related constructions whose commonality lies in empirical structure rather than shared formalism. In one field the term names a saturated SEM for directed functional dependencies; in another it names an edge-aware fusion gate; in another it names transfer-learned regressors built on a foundation MLIP. The surrounding literatures on SEMs and ESEm make that ambiguity even more pronounced [1602.06827], [2503.04821], [2508.20556], [2502.12764], [2108.10747].

Source: https://www.emergentmind.com/topics/esem-models