PhyLSTM²: Physics-Reinforced Double-LSTM

• The paper demonstrates that PhyLSTM² effectively incorporates physical laws via additional loss functions to enforce governing equations such as the Equation of Motion for improved accuracy.
• By combining two deep LSTM modules with graph-based differentiation, the method reliably predicts latent states including hysteretic dynamics in nonlinear structural systems.
• Empirical results show that PhyLSTM² achieves correlation coefficients above 0.9 on seismic benchmarks, markedly outperforming traditional LSTM models in data-scarce scenarios.

Physics-Reinforced Double-LSTM (PhyLSTM²) refers to a sequence modeling paradigm that integrates physically grounded constraints directly into the architecture and loss functions of recurrent neural networks, specifically leveraging coupled or hierarchical Long Short-Term Memory (LSTM) modules. The primary objective is to bridge the gap between purely data-driven modeling and physics-based simulation by ensuring that learned dynamics both fit observed data and intrinsically comply with governing physical laws. This approach is particularly relevant for the metamodeling of nonlinear dynamical systems—such as hysteretic structures—where data is scarce and robust generalization is required.

1. Integration of Physical Laws into LSTM Architectures

PhyLSTM² incorporates relevant physical laws not as post hoc corrections, but as intrinsic components of the network’s training process. Physical knowledge is encoded through additional loss terms that enforce consistency with established dynamic relationships:

• Equation of Motion (EOM): Constraints such as $M\ddot{u} + g = -\Gamma a_g$, where $g$ is the mass-normalized restoring force and $a_g$ is ground acceleration, are embedded into the loss function.
• State Dependency: Equality constraints enforce dependencies like $\frac{d}{dt}u = \dot{u}$; any deviation is penalized.
• Hysteretic Constitutive Relationships: For systems with rate-dependent or hysteretic behavior (especially in the PhyLSTM³ extension), one enforces additional differential constraints for internal variables representing hysteretic states.

The total loss function is a weighted sum of data loss ($J_d$), equality loss ($J_e$), governing loss ($J_g$), and possibly hysteretic loss ($J_h$), combined as

$$J(\theta_1, \theta_2, ... ) = \alpha J_d + \beta J_e + \gamma J_g [+ \eta J_h].$$

These terms ensure that the feasible solution space for optimization is defined not just by accuracy on observed data but by adherence to physical laws.

2. Network Structure and Computational Workflow

The canonical PhyLSTM² architecture consists of:

• Two deep LSTM modules:
  • LSTM1: Receives external driving signals (e.g., ground acceleration $a_g$) and outputs the predicted state vector $Z = [u, \dot{u}, r]^T$ (displacement, velocity, and latent hysteretic parameter).
  • LSTM2: Accepts $Z$ as input and predicts $g$, the (possibly latent) restoring force that governs structural behavior.
• Graph-based Differentiator: Uses finite-difference or built-in computational graphs to derive required time derivatives (e.g., $\dot{Z}$, $\ddot{u}$) for enforcing physical constraints.

In PhyLSTM³, a third LSTM is introduced to explicitly model hysteretic parameter evolution via the learned differential expression for $\dot{r} = f(\Phi)$, with $\Phi$ containing relevant system variables.

The architecture can be summarized by the interconnected flow:

a_g(t) → LSTM1 → Z(t) → Differentiator → [Z, Z', Z''] | | ↓ ↓ LSTM2(g) Physics Losses

3. Empirical Performance and Quantitative Results

PhyLSTM² and its extension PhyLSTM³ have been validated on complex nonlinear structural dynamics benchmarks:

• 3-story Steel Moment Frame (seismic loading):
  • Only displacement and velocity were measured. Latent quantities, including restoring force and hysteretic state, were predicted via physics guidance.
  • Correlation coefficients ($\gamma$) between predictions and ground truth for measured and latent variables were generally above 0.9, compared to $0.25$–$0.7$ for classical LSTM models (without physics constraints).
  • Physics-informed variants consistently maintained robust prediction quality even with limited training data and in regimes where classical LSTM performance degraded.
• Single Degree-of-Freedom Bouc-Wen Model:
  • In systems with rate-dependent hysteresis, PhyLSTM³’s explicit modeling of the hysteretic law further improved accuracy and stability.
  • Correlation coefficients for latent variables remained high for both states and restoring forces across challenging test scenarios.

These results underscore that embedded physics enables strong generalization, accurate latent state inference, and significant alleviation of overfitting risks under data scarcity.

4. Advantages, Limitations, and Calibration Considerations

Principal Benefits

• Interpretability: Physical constraints assure that outputs align with domain understanding, enhancing the model’s trustworthiness.
• Data Efficiency: The solution space is restricted to physically plausible regimes, reducing the data required to generalize.
• Prediction of Latent Quantities: The approach enables the inference of non-observable internal states (e.g., hysteretic variables) vital for monitoring and diagnostics.
• Reduced Overfitting: The regularizing effect of physics-based losses mitigates overfitting (a major concern in data-poor environments).

Challenges

• Model Complexity: Balancing multiple LSTM modules with graph-based differentiators increases implementation and computation burden.
• Hyperparameter Selection: Weighting of loss terms ($\alpha, \beta, \gamma, \eta$) is nontrivial and typically requires careful task-specific calibration to avoid undesirable tradeoffs between data fidelity and physics adherence.
• Portability: Extending the methodology to systems governed by different types of dynamics (e.g., fluid mechanics, biological systems) requires appropriate formulation of the physics losses and potentially new architectural motifs.

5. Connections to Related Physics-Informed Sequence Modeling Approaches

PhyLSTM² is positioned within a broader ecosystem of physics-informed sequential learning:

Model	Physics Integration	Application Domain
PhyLSTM² / PhyLSTM³	Loss terms for EOM, equality, hysteresis	Structural dynamics (nonlinear, hysteretic)
MC-LSTM (Hoedt et al., 2021)	Architecture enforces conservation laws (e.g. mass/energy) at each step	Traffic, hydrology, pendulum
PI-LSTM (Özalp et al., 2023)	Regularization based on differential equation residuals	Reconstruction of chaos (Lorenz-96)
PI-LSTM UGV (Abubakar et al., 26 Feb 2024)	Enforces delay differential equations via physics-informed loss	Teleoperated vehicle control
Physics-informed Reservoir (Bonas et al., 2023)	PDE penalty in loss, multi-reservoir layers	Boundary layer dynamics in fluids

The principal distinction is that PhyLSTM² enforces multiple types of physics—state dependency, EOM, hysteresis—simultaneously via differentiable loss terms, while other models may integrate conservation or PDE constraints directly within the architecture or as scalar regularizers.

6. Application Domains and Generalization

Although designed for the metamodeling of nonlinear structural systems under seismic loading or dynamic excitation, the underlying principles of PhyLSTM² extend to any context characterized by:

• Well-defined governing equations or state relationships (quantum, fluid, or biological domains)
• Scarce or incomplete observational data, particularly where latent state estimation is critical
• The necessity to prevent physically impossible solutions during inference (e.g., negative energies, nonconservative flows)

Potential application domains include:

• Fluid dynamics (embedding Navier–Stokes or shallow-water equations into recurrent frameworks)
• Real-time health monitoring (cardiovascular, neurodynamics where only surrogates are observable)
• Smart grid stability and load forecasting (where the underlying physics of energy or power flow must be strictly respected)
• Materials and constitutive modeling in computational mechanics

7. Outlook and Research Directions

The PhyLSTM² paradigm exemplifies a hybrid direction in sequential modeling: leveraging the representational power of deep recurrent architectures while enforcing the invariants and constraints dictated by physical theory. Future trajectories include:

• Adaptive or meta-learning methods for automated tuning of loss weightings and model structure to diverse physical regimes
• Integration with uncertainty quantification and probabilistic inference to robustly address noisy or partially observed systems
• Modular extension to multi-scale or hierarchical physics-informed networks (e.g., coupling PhyLSTM² with graph neural networks for spatially extended systems)
• Expansion to agent-based and multi-verifier frameworks, inspired by agentic LLM pipelines for reasoning validation (Siddique et al., 31 Jul 2025), where an ensemble of physics-aware modules assess and refine candidate predictions through internal critique and verification.

In aggregate, Physics-Reinforced Double-LSTM approaches—by embedding physical law as an explicit inductive bias—deliver improvements in accuracy, robustness, interpretability, and data efficiency for the modeling of complex dynamical systems where traditional black-box strategies fall short.