Variational LSTM with Augmented Inputs: Nonlinear Response History Metamodeling with Aleatoric and Epistemic Uncertainty

Abstract: Uncertainty propagation in high-dimensional nonlinear dynamic structural systems is pivotal in state-of-the-art performance-based design and risk assessment, where uncertainties from both excitations and structures, i.e., the aleatoric uncertainty, must be considered. This poses a significant challenge due to heavy computational demands. Machine learning techniques are thus introduced as metamodels to alleviate this burden. However, the "black box" nature of Machine learning models underscores the necessity of avoiding overly confident predictions, particularly when data and training efforts are insufficient. This creates a need, in addition to considering the aleatoric uncertainty, of estimating the uncertainty related to the prediction confidence, i.e., epistemic uncertainty, for machine learning-based metamodels. We developed a probabilistic metamodeling technique based on a variational long short-term memory (LSTM) with augmented inputs to simultaneously capture aleatoric and epistemic uncertainties. Key random system parameters are treated as augmented inputs alongside excitation series carrying record-to-record variability to capture the full range of aleatoric uncertainty. Meanwhile, epistemic uncertainty is effectively approximated via the Monte Carlo dropout scheme. Unlike computationally expensive full Bayesian approaches, this method incurs negligible additional training costs while enabling nearly cost-free uncertainty simulation. The proposed technique is demonstrated through multiple case studies involving stochastic seismic or wind excitations. Results show that the calibrated metamodels accurately reproduce nonlinear response time histories and provide confidence bounds indicating the associated epistemic uncertainty.

• The paper introduces a framework using variational LSTM with augmented inputs to capture both aleatoric and epistemic uncertainty in nonlinear structural systems.
• The methodology leverages POD-wavelet compression and Monte Carlo dropout for efficient, high-fidelity surrogate modeling of dynamic responses.
• Numerical validations across various structural models demonstrate scalability, accuracy, and calibrated risk assessment for safety-critical applications.

Variational LSTM with Augmented Inputs for Nonlinear Structural Metamodeling with Aleatoric and Epistemic Uncertainty

Introduction and Motivation

The paper "Variational LSTM with Augmented Inputs: Nonlinear Response History Metamodeling with Aleatoric and Epistemic Uncertainty" (2604.01587) synthesizes contemporary advancements in data-driven surrogate modeling of nonlinear high-dimensional structural systems under stochastic excitation. The authors address two salient challenges in probabilistic performance-based engineering: prohibitive computational cost of repeated nonlinear time-history analyses, and the necessity to capture both aleatoric (inherent variability) and epistemic (model-form and data-driven) uncertainty in surrogate predictions. The proposed framework leverages variational LSTM networks, augmented with explicit system parameters to encapsulate system uncertainty, and utilizes Monte Carlo dropout for efficient epistemic uncertainty propagation without the computational overhead of full Bayesian inference.

Methodological Framework

Capturing Aleatoric and Epistemic Uncertainty

Aleatoric uncertainty is bifurcated into record-to-record excitation variability and parametric system uncertainty. The architecture explicitly fuses random structural parameters as additional input features, recursively processed with stochastic excitation time series by the LSTM network. This enables modeling the joint response variability induced by both input uncertainties in complex nonlinear multi-degree-of-freedom systems.

Epistemic uncertainty, crucial for safety-critical prediction tasks, is quantified via a variational Bayesian approach employing Monte Carlo dropout [gal2015dropout]. The network maintains an ensemble of parameter realizations by random dropout at inference; the predictive variability across this ensemble approximates posterior model confidence. This mechanism is computationally lightweight compared to full MCMC or bootstrap ensembles.

Data Preparation and Model Order Reduction

For application to high-dimensional systems and long-duration time series, the paper deploys proper orthogonal decomposition (POD) for spatial dimensionality reduction, and wavelet-based transformation for temporal downsampling. Input excitations and output responses are projected onto the dominant POD bases, and compressed via discrete wavelet coefficients, drastically reducing memory and computational demand while retaining signal fidelity.

Training and Inference

The LSTM model is trained to minimize a regularized loss function equivalent to the evidence lower bound (ELBO), combining mean squared reconstruction error with (optional) L2 regularization stemming from the variational approximation. Dropout is applied during both training and inference, with the same random mask across all time steps to mitigate signal weakening in the recurrent setting [gal2016theoretically]. Post-training, uncertainty propagation is efficiently carried out by assembling ensembles of predictions with perturbed dropout masks.

Numerical Validation

The paper systematically validates the methodology on three canonical structural dynamics problems:

1. SDOF Bouc-Wen oscillator under stochastic seismic excitation: The network accurately reconstructed nonlinear hysteretic responses with tight prediction intervals. The 95% epistemic confidence bounds strongly reflected training/validation data limitations.
2. Six-story nonlinear shear building: With multiple sources of system uncertainty, the surrogate produced accurate time histories and hysteresis loops. The model generalized well across unseen excitation-parameter combinations. Epistemic intervals provided a calibrated assessment of surrogate reliability for both global (floor displacements) and local (hysteresis) responses.
3. 37-story steel frame under synthetic wind loads: For this high-dimensional, highly nonlinear system, the POD-wavelet compressed variational LSTM achieved high-fidelity time history reconstruction. Importantly, epistemic uncertainty was notably larger for spatially localized quantities (e.g., local fiber strains) than for global floor displacements, emphasizing the necessity of uncertainty awareness in downstream fragility or risk analyses.

Key Findings and Numerical Performance

• The augmented input LSTM architecture reproduces response time histories and peak responses with negligible bias and low mean squared error relative to high-fidelity direct integration solvers, across a wide range of dynamic response regimes and system uncertainties.
• Incorporating system parameters as input features (rather than mapping parametric uncertainty to model weights, as in NARX-plus-Kriging frameworks) retains scalability with increasing system dimension.
• Monte Carlo dropout-based estimation of epistemic uncertainty incurs negligible additional cost, requiring only standard dropout-enabled forward passes with no need for retraining or explicit sampling over network weights.
• Confidence intervals derived from the surrogate enable risk calibration and avoid overconfident predictions, addressing the recognized limitation of black-box neural surrogates in safety-critical simulations.

Implications, Limitations, and Outlook

The framework directly supports large-scale uncertainty propagation and reliability assessment tasks in structural and wind/seismic engineering, where ensemble-level response simulation is critical. Efficient surrogate-based uncertainty quantification is essential for enabling probabilistic design, regional risk assessment, and real-time forecasting. The demonstrated approach is modular—amenable to integration with alternative sequence modeling architectures (e.g., attention-based models, neural operators [GOSWAMI2025121284]) and generic post-processing of outputs for fragility or loss estimation.

Key limitations include:

• The accuracy and generalizability of the metamodel are bounded by the coverage and density of the training data in both excitation and system parameter space.
• POD-wavelet compression relies on the smoothness and regularity of structural responses; extreme localization (e.g., fracture or loss of continuity) may require alternative domain-specific bases.
• While Monte Carlo dropout is a practical epistemic uncertainty quantification tool, its theoretical calibration as a true Bayesian posterior approximation is limited for deep, highly nonlinear RNNs [osband2016risk]. Alternative approximate Bayesian inference schemes may be required for extrapolative predictive tasks.

Future development directions include end-to-end differentiable uncertainty propagation pipelines for full-building portfolios, integration with active learning strategies for data-efficient calibration, and extension to coupling with experimental or sensor-derived input-output data.

Conclusion

The study provides an operational surrogate modeling methodology that robustly addresses both aleatoric and epistemic uncertainty in the emulation of nonlinear high-dimensional structural dynamics. The fusion of variational LSTM networks with explicit input augmentation and Monte Carlo dropout yields a scalable, computationally efficient platform for uncertainty-aware, data-driven structural response simulation. This approach is well-positioned to support advances in performance-based engineering, risk analysis, and autonomous digital twins for resilient infrastructure systems.

---

References

• Y. Gal, Z. Ghahramani. "Dropout as a Bayesian approximation: Representing model uncertainty in deep learning." (Gal et al., 2015)
• GOSWAMI et al., "Neural operators for stochastic modeling of nonlinear structural system response to natural hazards." [GOSWAMI2025121284]
• I. Osband. "Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout." [NIPS workshop, 2016]