Residual Attention Physics-Informed Neural Networks for Robust Multiphysics Simulation of Steady-State Electrothermal Energy Systems

Abstract: Efficient thermal management and precise field prediction are critical for the design of advanced energy systems, including electrohydrodynamic transport, microfluidic energy harvesters, and electrically driven thermal regulators. However, the steady-state simulation of these electrothermal coupled multiphysics systems remains challenging for physics-informed neural computation due to strong nonlinear field coupling, temperature-dependent coefficient variability, and complex interface dynamics. This study proposes a Residual Attention Physics-Informed Neural Network (RA-PINN) framework for the unified solution of coupled velocity, pressure, electric-potential, and temperature fields. By integrating a unified five-field operator formulation with residual-connected feature propagation and attention-guided channel modulation, the proposed architecture effectively captures localized coupling structures and steep gradients. We evaluate RA-PINN across four representative energy-relevant benchmarks: constant-coefficient coupling, indirect pressure-gauge constraints, temperature-dependent transport, and oblique-interface consistency. Comparative analysis against Pure-MLP, LSTM-PINN, and pLSTM-PINN demonstrates that RA-PINN achieves superior accuracy, yielding the lowest MSE, RMSE, and relative $L_2$ errors across all scenarios. Notably, RA-PINN maintains high structural fidelity in interface-dominated and variable-coefficient settings where conventional PINN backbones often fail. These results establish RA-PINN as a robust and accurate computational framework for the high-fidelity modeling and optimization of complex electrothermal multiphysics in sustainable energy applications.

• The paper introduces RA-PINN, a five-field physics-informed neural network that combines residual blocks, attention-based feature gating, and residual-driven collocation for coupled velocity, pressure, electric-potential, and temperature fields.
• RA-PINN achieves the best case-averaged accuracy across four benchmarks, including a relative L2 error of 5.065×10⁻³ for temperature-dependent transport and 1.377×10⁻³ for an oblique-interface case, although it does not win every individual field.
• The method’s main practical trade-off is computational cost: training takes roughly 2–9 times longer than competing architectures, reaching 39.81 hours, while its performance beyond 2D steady synthetic benchmarks remains unverified.

Motivation and problem setting

Steady-state electrothermal coupled systems—spanning electrohydrodynamic transport, microfluidic energy harvesters, and electrically driven thermal regulators—require the simultaneous resolution of velocity, pressure, electric-potential, and temperature fields that feed back into one another through transport, forcing, and constitutive effects. The authors identify three persistent obstacles for physics-informed neural computation on such problems: strong nonlinear field coupling, temperature-dependent coefficient variability, and complex interface dynamics. Field magnitudes and gradient scales differ substantially across the five unknowns, which biases vanilla PINN optimization toward some fields at the expense of others; variable coefficients and indirect constraints stiffen the loss landscape; and interface-dominated solutions demand simultaneous representation of broad smooth transport and narrow local transitions.

The paper proposes a Residual Attention Physics-Informed Neural Network (RA-PINN) that combines two ingredients drawn from recent literature: a residual-connected deep backbone [(2603.23578); cf. (Thomasian, 2024)] and attention-guided channel modulation motivated by residual-based attention (RBA) and loss-attentional weighting schemes. The framework is evaluated against Pure-MLP, LSTM-PINN, and pLSTM-PINN backbones on four steady electrothermal benchmarks of increasing difficulty.

Architecture

The RA-PINN solver maps the spatial coordinate $(x,y)$ to a five-field prediction vector $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$. Each residual block computes a trunk feature $\bm{t}^{(\ell)}$ and an attention gate $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$, with block output

$$\bm{z}^{(\ell+1)}=\bm{z}^{(\ell)}+\left(\bm{1}+\bm{m}^{(\ell)}\right)\odot\bm{t}^{(\ell)}.$$

The residual pathway preserves gradient transmission and global background structure across depth, while the multiplicative $(\bm{1}+\bm{m})$ gate amplifies channels encoding steep transitions and interface-sensitive signatures. This is distinct from residual-driven collocation focusing: the same pointwise residual indicator $$r(x;\theta)=\|\mathcal{N}(\widehat{U}(x;\theta))\|_2$$ also drives adaptive resampling of interior and boundary collocation points toward persistently high-residual regions.

Training minimizes a composite objective combining interior PDE residuals, boundary residuals, an optional supervised data term, regularization, a pressure-gauge term, and interface continuity/flux-jump terms:

$$\mathcal{L}_{\mathrm{total}}=\lambda_{\mathrm{res}}\mathcal{L}_{\mathrm{res}}+\lambda_{\mathrm{b}}\mathcal{L}_{\mathrm{b}}+\lambda_{\mathrm{data}}\mathcal{L}_{\mathrm{data}}+\lambda_{\mathrm{reg}}\mathcal{L}_{\mathrm{reg}}+\lambda_{\mathrm{gauge}}\mathcal{L}_{\mathrm{gauge}}+\lambda_{\Gamma}\mathcal{L}_{\Gamma},$$

optimized by Adam-style gradient descent. A notable strength of the formulation is its uniformity: all four benchmarks share one PDE operator template, with case-specific constraints activated only when required.

Unified PDE formulation

The governing system is posed on the unit square with unknown vector $U=(u,v,p,\phi,T)^\top$: steady incompressibility $\nabla\cdot\bm{u}=0$, momentum balance with electrothermal body force $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$0, a conductivity equation $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$1, and an advection–diffusion temperature equation $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$2. The four cases instantiate this template as follows:

Case	Distinguishing feature
1	Constant coefficients, full Dirichlet conditions
2	Zero-mean pressure gauge $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$3 replaces direct pressure anchoring
3	Temperature-dependent coefficients $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$4, $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$5
4	Oblique interface $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$6 with field continuity and prescribed flux jumps

This unified construction permits a consistent comparison of error metrics (MSE, RMSE, MAE, relative $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$7) and training cost across cases and architectures.

Benchmark results

Case 1 (constant coefficients). RA-PINN attains the minimum error in every field and every metric. The averaged MSE improves from $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$8 (LSTM-PINN) to $$\widehat{U}(x;\theta)=[\hat u,\hat v,\hat p,\hat\phi,\hat T]^\top$$9, and averaged relative $\bm{t}^{(\ell)}$0 from $\bm{t}^{(\ell)}$1 to $\bm{t}^{(\ell)}$2—roughly a threefold reduction over the strongest baseline. Pure-MLP trails by two orders of magnitude in MSE ($\bm{t}^{(\ell)}$3). However, RA-PINN requires 24.01 h of training versus about 1 h for pLSTM-PINN and Pure-MLP, so the accuracy gain carries a substantial computational penalty even in the simplest regime.

Case 2 (pressure gauge). Replacing pointwise pressure prescription with the zero-mean gauge constraint degrades all solvers, but RA-PINN again achieves the minimum in all fields and metrics, with averaged RMSE reduced from $\bm{t}^{(\ell)}$4 to $\bm{t}^{(\ell)}$5 relative to LSTM-PINN. The authors note an "exact mismatch warning" on the benchmark canvas and accordingly restrict figure-based conclusions to structural comparison rather than absolute color-level reading—a candid caveat on the visual evidence. Training time remains the longest among the compared models (24.67 h).

Case 3 (temperature-dependent transport). Variable coefficients amplify inter-solver differences most strongly. RA-PINN's averaged relative $\bm{t}^{(\ell)}$6 is $\bm{t}^{(\ell)}$7 versus $\bm{t}^{(\ell)}$8 (LSTM-PINN), $\bm{t}^{(\ell)}$9 (Pure-MLP), and $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$0 (pLSTM-PINN)—the recurrent-parallel backbone fails almost completely under coefficient feedback. One nuance deserves emphasis: LSTM-PINN slightly outperforms RA-PINN on the $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$1 field, so the claim of universal per-field superiority does not hold here. This case also produces the longest runtime in the entire study at 39.81 h.

Case 4 (oblique interface). The margin narrows considerably. LSTM-PINN is slightly better on $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$2, $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$3, $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$4, and $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$5, while RA-PINN wins clearly on $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$6 and retains the best averages (relative $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$7 of $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$8 vs. $$\bm{m}^{(\ell)}=\sigma(\mathcal{M}^{(\ell)}(\bm{z}^{(\ell)}))$$9; MAE $$\bm{z}^{(\ell+1)}=\bm{z}^{(\ell)}+\left(\bm{1}+\bm{m}^{(\ell)}\right)\odot\bm{t}^{(\ell)}.$$0 vs. $$\bm{z}^{(\ell+1)}=\bm{z}^{(\ell)}+\left(\bm{1}+\bm{m}^{(\ell)}\right)\odot\bm{t}^{(\ell)}.$$1). The abstract's claim that RA-PINN yields "the lowest MSE, RMSE, and relative $$\bm{z}^{(\ell+1)}=\bm{z}^{(\ell)}+\left(\bm{1}+\bm{m}^{(\ell)}\right)\odot\bm{t}^{(\ell)}.$$2 errors across all scenarios" is therefore accurate only at the case-average level, not field-by-field. At 38.35 h versus 18.35 h for LSTM-PINN, the accuracy gain in the interface-dominated setting is modest relative to its cost.

Cross-case assessment

Three patterns emerge from the aggregate evidence. First, the residual-attention mechanism provides its largest benefit precisely where conventional backbones are weakest—variable-coefficient coupling and indirect gauge constraints—with improvements of up to several orders of magnitude over pLSTM-PINN. Second, the advantage shrinks as the solution becomes smoother or when a specialized recurrent backbone already handles the geometry well, indicating that part of RA-PINN's benefit stems from representational capacity rather than physics-specific inductive bias. Third, training cost scales unfavorably: RA-PINN is consistently the slowest model, requiring roughly 2–9 times the runtime of alternatives, which the authors acknowledge explicitly rather than obscuring.

Limitations and open questions

Several limitations qualify the reported results. All benchmarks are two-dimensional steady problems on the unit square; no transient, three-dimensional, or geometrically complex configuration is tested, so scalability of the adaptive sampling loop beyond this regime remains unverified. The evaluation relies on synthetically constructed benchmark fields with prescribed sources and coefficients rather than experimentally measured data, leaving generalization to real device geometries open. Hyperparameter sensitivity—the loss weights $$\bm{z}^{(\ell+1)}=\bm{z}^{(\ell)}+\left(\bm{1}+\bm{m}^{(\ell)}\right)\odot\bm{t}^{(\ell)}.$$3, sampling schedules, and network depth—is deferred to supplementary material without ablation in the main text, so the contribution of each component (residual connections versus attention gating versus adaptive sampling) cannot be disentangled from the reported aggregates. Finally, the substantial training-time overhead is conceded but not addressed; whether efficiency can be improved without sacrificing accuracy is an explicit open question left by the authors.

Conclusion

This paper presents RA-PINN, a residual-connected, attention-modulated PINN framework with adaptive residual-based collocation, applied within a unified five-field operator formulation for steady electrothermal multiphysics. Across four benchmarks it achieves the best case-averaged accuracy against Pure-MLP, LSTM-PINN, and pLSTM-PINN, with the largest gains under temperature-dependent coefficients and pressure-gauge constraints, and only narrow margins in the oblique-interface case where LSTM-PINN wins several individual fields. The consistent trade-off is computational cost, with runtimes up to 39.81 h. The framework is released as open-source code, providing a reproducible baseline for high-fidelity electrothermal simulation, though its value in practice will depend on resolving the training-efficiency question the study leaves open.