PREIG: Physics-Informed GRU for Forecasting

• PREIG is a deep learning framework that integrates physics-informed constraints, enforcing negative price-demand elasticity within a GRU architecture for commodity forecasting.
• It employs a custom composite loss combining data fidelity (MSE) with a physics loss that penalizes economic inconsistencies via automatic differentiation.
• The model utilizes a hybrid optimization strategy with population-based training, NAdam, and L-BFGS to achieve state-of-the-art accuracy on high-dimensional, nonlinear time series.

PREIG (Physics-informed and Reinforcement-driven Interpretable GRU) is a deep learning framework developed for commodity demand forecasting that integrates domain-specific economic constraints into a Gated Recurrent Unit (GRU) network via physics-informed principles and a hybrid, population-based optimization strategy. The central innovation is the enforcement of the negative price-demand elasticity—an economic law stating that demand should not increase with price—directly into the learning objective, ensuring interpretable and economically consistent predictions. PREIG is designed to handle high-dimensional, nonlinear, and non-stationary time series, and demonstrates improved predictive accuracy and interpretability relative to both traditional econometric models and standard deep learning baselines (Ma et al., 29 Jul 2025).

1. Core Motivation and Design Principles

PREIG addresses the challenge of accurate demand forecasting in commodity markets characterized by volatile dynamics and intricate nonlinear dependencies. Standard neural models may violate domain knowledge, such as the law of demand, leading to predictions that are plausible statistically but not economically interpretable. PREIG overcomes this by embedding a physics constraint—specifically, enforcing that the learned demand must not increase as price rises, mathematically encoded as $$\partial\,\mathrm{demand}/\partial\,\mathrm{price}\leq 0$$.

Two main architectural choices underpin PREIG:

• The use of a GRU recurrent network as its sequence modeling backbone, which enables the extraction of temporal features from high-dimensional inputs.
• The addition of a physics-informed custom loss, which penalizes any violation of negative price elasticity throughout training.

Furthermore, optimization is driven by a hybrid procedure: Population-Based Training (POP) orchestrates the joint evolution of model weights, learning rate, and physics loss weight, while each candidate is refined by both first-order (NAdam) and quasi-Newton (L-BFGS) methods.

2. Model Architecture and Physics-Informed Loss

GRU Backbone

Let $x_t\in\mathbb{R}^d$ denote the input vector at time $t$ (including features such as price and macro/mesoeconomic indicators), and $h_{t-1}\in\mathbb{R}^H$ the previous hidden state. The GRU cell update equations are

$$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$

where $W_*$, $U_*$, $b_*$ are learned parameters, $\sigma(\cdot)$ is the sigmoid, and $\odot$ is element-wise multiplication.

Physics-Informed Neural Network (PINN) Principle

The output $x_t\in\mathbb{R}^d$0 at time $x_t\in\mathbb{R}^d$1 represents predicted demand; $x_t\in\mathbb{R}^d$2 is the price component of $x_t\in\mathbb{R}^d$3. The model computes $x_t\in\mathbb{R}^d$4 through automatic differentiation. The economic constraint is $x_t\in\mathbb{R}^d$5 for all $x_t\in\mathbb{R}^d$6.

Composite Loss Function

The composite objective integrates two components:

1. Data loss (MSE):

$x_t\in\mathbb{R}^d$7

2. Physics loss (elasticity constraint):

$x_t\in\mathbb{R}^d$8

3. Total loss:

$x_t\in\mathbb{R}^d$9

$t$0 and $t$1 weight data fidelity vs. physics adherence.

3. Hybrid Optimization via Population-Based Training

PREIG employs a three-level optimization strategy:

3.1 Population-Based Training (POP)

A population of $t$2 candidate models, each parameterized by $t$3 ($t$4), is simultaneously optimized. Each round involves:

• Evaluation: Inner-loop optimization using NAdam and L-BFGS for a fixed budget; compute validation losses.
• Selection: Retain the top $t$5 models by validation performance.
• Perturbation: Clone and randomly perturb hyperparameters (model weights, learning rate $t$6, physics weight $t$7) of top candidates to replace the bottom $t$8.

3.2 NAdam First-Order Stage

Within each candidate, NAdam performs $t$9 steps: $h_{t-1}\in\mathbb{R}^H$0 $h_{t-1}\in\mathbb{R}^H$1.

3.3 L-BFGS Quasi-Newton Refinement

After NAdam, each candidate is refined with $h_{t-1}\in\mathbb{R}^H$2 L-BFGS steps, updating parameters using an approximate inverse Hessian constructed from recent gradients and steps: $h_{t-1}\in\mathbb{R}^H$3 $h_{t-1}\in\mathbb{R}^H$4 is built from histories $h_{t-1}\in\mathbb{R}^H$5 where $h_{t-1}\in\mathbb{R}^H$6 and $h_{t-1}\in\mathbb{R}^H$7.

4. Training Configuration and Datasets

The PREIG framework was benchmarked on monthly export volumes for Coal, Soybean, Crude Oil, and Iron Ore from December 2014 to December 2024 (121 months). The first 114 months were used for training; the final 6 for testing. Features included macroeconomic (exchange rate, commodity indices, GDP, etc.), mesoeconomic (energy/substitute prices, futures, shipping, inventories), and historical export lags (up to 64 months).

Key hyperparameters:

• GRU hidden size $h_{t-1}\in\mathbb{R}^H$8
• Input window $h_{t-1}\in\mathbb{R}^H$9
• Batch size 32; dropout 0.2
• Initial learning rate $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$0; evolved by POP
• NAdam parameters: $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$1, $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$2, $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$3
• Physics loss weight $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$4 (POP-evolved)
• Inner optimization: $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$5 NAdam steps, $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$6 L-BFGS steps
• POP population $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$7, $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$8 rounds

Convergence typically occurs in approximately 2200 steps, with total training on a single NVIDIA RTX 2080 Ti taking 25–30 minutes per POP sweep.

5. Empirical Performance and Interpretability

5.1 Quantitative Results

Test-set performance for the four commodities (lower is better):

Model	Soybean (RMSE)	Coal (RMSE)	Crude Oil (RMSE)	Iron Ore (RMSE)
ARIMA	300.20	3.34	774.86	8336.04
GARCH	366.80	1.87	478.50	5453.20
BPNN	276.74	1.46	240.79	2701.40
RNN	197.37	0.51	164.82	922.49
GRU	143.82	0.24	89.53	627.95
PREIG	154.55	0.22	104.94	685.23

PREIG reduces coal RMSE to 0.22 (vs. 0.24 for GRU), and coal MAPE to 0.52% (vs. 0.55% for GRU), achieving top-tier accuracy across all series.

5.2 Physics Consistency

For every test time step $$\begin{aligned} z_t &= \sigma(W_z x_t + U_z h_{t-1} + b_z) \ r_t &= \sigma(W_r x_t + U_r h_{t-1} + b_r) \ \tilde{h}_t &= \tanh(W_h x_t + U_h (r_t \odot h_{t-1}) + b_h) \ h_t &= (1 - z_t) \odot h_{t-1} + z_t \odot \tilde{h}_t \end{aligned}$$9, the condition $W_*$0 holds, guaranteeing negative elasticity. In coal, $W_*$1 averaged $W_*$2 (std 0.003) across test months, resulting in a zero physics-loss penalty—demonstrating that outputs are by construction aligned with the economic constraint.

6. Scalability, Limitations, and Future Extensions

Scalability

The additional backward pass to compute the PINN gradient $W_*$3 scales linearly with GRU hidden size $W_*$4 and sequence length $W_*$5. POP candidates are independent and suitable for parallelization on multiple GPUs. The architecture accommodates inputs with $W_*$6 features and look-back windows of $W_*$7 with modest cost increases.

Limitations

Currently, PREIG enforces only univariate price elasticity; cross-price and income elasticities are not modeled. POP introduces computational overhead due to multiple concurrent candidates, potentially impeding real-time adaptation. Storing gradient histories for L-BFGS elevates memory requirements.

Potential Extensions

• Imposition of multivariate PDE-style constraints, e.g., cross-elasticity matrices $W_*$8
• Replacement of L-BFGS with stochastic quasi-Newton methods (e.g., oLBFGS) for lower algorithmic overhead
• Integration of attention mechanisms to dynamically weight features (macro, meso, historical)
• Extension to multivariate outputs for joint forecasting of multiple commodities with shared physics constraints

By embedding a hard economic rule within a recurrent neural forecasting architecture and leveraging a hybridized optimization scheme, PREIG delivers both interpretability and state-of-the-art predictive performance for high-dimensional commodity demand forecasting (Ma et al., 29 Jul 2025).