Physics-Reinforced Triple-LSTM (PhyLSTM³)

• The paper introduces PhyLSTM³, embedding physical constraints into a triple-LSTM architecture to model nonlinear dynamics from sparse data.
• It integrates governing equations, state dependencies, and hysteretic relationships via custom loss functions to enhance accuracy and interpretability.
• The framework outperforms traditional LSTMs, achieving high correlation (>0.9) and robust latent variable inference even under extreme excitation.

The Physics-Reinforced Triple-LSTM (PhyLSTM³) framework is an advanced physics-informed deep learning architecture designed for metamodeling nonlinear dynamical systems, particularly under conditions of sparse or limited data. By integrating explicit physical constraints—such as governing equations, state dependencies, and hysteretic relationships—directly into the sequential modeling process, PhyLSTM³ achieves robust, interpretable, and data-efficient predictions for critical latent variables and system responses.

1. Integration of Physical Laws in Deep Sequential Learning

The central principle of PhyLSTM³ is the incorporation of known physics into the training process of LSTM-based networks via custom loss functions. Rather than treating the model as purely data-driven, several physical relationships are directly enforced:

• Equation of Motion: The reduced motion equation for earthquake-excited structures is cast as

$$M \ddot{u} + C \dot{u} + \lambda K u + (1-\lambda) K r = -\Gamma a_g$$

which, after mass normalization, becomes

$\ddot{u} + g = -\Gamma a_g$

where $g(t)$ (restoring force) is a latent variable governed by system state $Z$.

• State Dependency: The displacement derivative must equal velocity, enforced by a loss term:

$$\mathcal{J}_e(\theta_1) = \sum_{i=1}^{n_c} \| \dot{z}_1^{(i)}(\theta_1) - z_2^{(i)}(\theta_1) \|_2^2$$

• Hysteretic Constitutive Relationship: For rate-dependent hysteresis (e.g., Bouc–Wen model),

$\dot{r} = f(\Phi)$

with physics-informed loss:

$$\mathcal{J}_h(\theta_1, \theta_3) = \sum_{i=1}^{n_c} \| \dot{z}_3^{(i)}(\theta_1) - \dot{r}^{(i)}(\theta_1, \theta_3) \|_2^2$$

These constraint losses—alongside the typical data loss—serve to restrict the network solution to physically admissible regimes, even for non-observable state variables.

2. Architecture: Triple-LSTM System Design

PhyLSTM³ extends the double-LSTM architecture (PhyLSTM²) to a triple-LSTM system to model complex hysteretic and dynamical phenomena. The architecture comprises:

• LSTM1: Maps external excitation (e.g., ground acceleration $a_g$) to the system state $Z = \{u, \dot{u}, r\}$, where $r$ is often latent.
• LSTM2: Predicts restoring force $g$ from the learned $Z$, serving as a surrogate for a complex constitutive law.
• LSTM3: Models the differential evolution of the hysteretic parameter $r$, taking system-derived features $\Phi$ as input.

A graph-based tensor differentiator interconnects these networks for derivative calculations, enforcing cross-variable relationships. Loss components are combined as a weighted sum, and training proceeds via standard optimizers with backpropagation.

Sub-network	Input(s)	Output(s)	Purpose
LSTM1	$a_g$	$u$, $\dot{u}$, $r$	State trajectory prediction
LSTM2	$Z$	$g$	Restoring force computation
LSTM3	$\Phi$	$\dot{r}$	Hysteretic rate modeling

3. Demonstrated Performance on Nonlinear Structural Systems

PhyLSTM³ is validated through seismic metamodeling of moment-resisting steel frames and single-degree-of-freedom models with rate-dependent hysteresis.

• When only displacement and velocity measurements are provided, PhyLSTM³ can infer latent hysteretic quantities and restore physically plausible nonlinear system responses.
• Regression metrics (correlation coefficient $\gamma > 0.9$) demonstrate that PhyLSTM³ maintains high predictive accuracy on unseen loadings, outperforming conventional LSTM architectures, notably in “worst-case” scenarios involving extreme nonlinearities.
• Overfitting is alleviated due to physics-driven regularization, enabling robust generalization from small, representative data sets.

Notably, PhyLSTM³ accurately captures residual drifts, post-yield deformations, and hysteresis curves, essential for assessing structural safety and reliability under dynamic excitation.

4. Comparative Advantages over Classical Neural Approaches

The physics-informed paradigm employed in PhyLSTM³ yields significant benefits compared to classical, non-physics-guided sequential models:

• Data Efficiency: High-accuracy predictions achievable with far fewer training samples, critical for real-world settings with limited measurements.
• Latent Variable Inference: The framework can estimate unobservable system components, such as restoring force $g$ and hysteretic parameter $r$, expanding the utility of metamodeling.
• Generalizability and Robustness: Models generalize across varying excitation types with lower susceptibility to overfitting.
• Interpretability: The architecture, constrained by well-understood physical laws, offers superior interpretability over “black-box” neural solutions.
• Computational Effectiveness: Faster network convergence is observed due to guided optimization within a physically feasible solution space.

5. Extension to Other Domains and Future Directions

The methodological insights from PhyLSTM³ are extensible beyond seismic structural metamodeling. Candidate domains include molecular dynamics, complex industrial process control, chaotic systems, and teleoperated robotics—where physically constrained sequential prediction is essential.

• Multistream architectures leveraging triple-LSTM designs, as described for teleoperated UGVs, suggest that reinforcement across predictors (e.g., for distinct control loops) could further enhance system fidelity in complex, delayed, or feedback-coupled environments.
• Physical constraints and prior-informed loss formulations hold promise for improved learning efficiency and model stability in domains such as molecular simulations and control systems with non-observable state variables.
• Combination with program-synthesis-inspired strategies or simulation-driven feedback (as investigated in LLMPhy) could provide new avenues for integrating high-level reasoning and low-level dynamics in future recurrent architectures.

A plausible implication is that the PhyLSTM³ design offers a robust template for physically constrained machine learning in scenarios where latent variable inference and generalization with limited data are essential.

6. Summary of Key Characteristics

• Explicit embedding of governing equations, state dependencies, and constitutive relationships within the LSTM framework via physics-informed loss components.
• Triple-network architecture enables modeling of complex state evolution, restoring forces, and rate-dependent hysteresis.
• Validated superior performance under sparse data, with substantial improvements in accuracy and robustness versus purely data-driven LSTMs.
• Extensible methodology showing potential for broader impact in physics-informed deep learning across dynamical systems, stochastic simulations, and adaptive control.

In conclusion, Physics-Reinforced Triple-LSTM (PhyLSTM³) represents an advanced approach to sequential modeling with embedded physical constraints, offering a path toward data-efficient, interpretable, and robust metamodeling of nonlinear systems. Its design and demonstrated effectiveness underline its potential relevance and applicability across diverse physics-governed domains (Zhang et al., 2020).