Phys-JEPA: Physics-Informed Latent World Models for Multivariate Time-Series Forecasting

Abstract: Multivariate forecasting in physical systems requires models that predict coupled temporal variables while preserving meaningful state evolution. Deep forecasters can fit temporal correlations, and physics-informed models can regularize predictions with scientific constraints, but these directions are often connected only at the decoded-output level. As a result, the hidden predictive state that generates future trajectories may remain statistically useful but physically unstructured. We introduce Phys-JEPA, a physics-informed joint-embedding predictive architecture for multivariate time-series forecasting. Phys-JEPA learns a latent world model in which predictive states are decomposed into physical and residual components, and physical consistency is imposed directly on latent states and latent transitions rather than only on decoded forecasts. This formulation uses known physical variables to organize the representation space while retaining residual capacity for unresolved dynamics. On Jena Climate 2009--2016, Phys-JEPA reduces aggregate MSE from 0.12482 to 0.12273 and temperature MSE from 0.01892 to 0.01831 at H=24. On Traffic, full Phys-JEPA improves aggregate MSE over the supervised baseline across all tested horizons, reducing H=192 MSE from 0.800784 to 0.773873. On Electricity, the best variant depends on horizon: static latent consistency is strongest at H=24 and H=48, while full Phys-JEPA gives the best aggregate and target-variable MSE at H=192. These initial results suggest that moving physics-informed learning from output space to latent predictive state space is a promising direction for interpretable temporal world models.

• The paper introduces latent-space physics constraints that enforce physical consistency for more interpretable and plausible forecasts.
• It decomposes the latent state into physical and residual components to integrate domain knowledge with data-driven dynamics.
• Experiments on climate, electricity, and traffic datasets demonstrate improved MSE and robustness compared to traditional methods.

Phys-JEPA: Physics-Informed Latent World Models for Multivariate Time-Series Forecasting

Introduction and Motivation

Time-series forecasting in domains governed by underlying physical phenomena—such as meteorology, energy, or transportation—poses specific challenges for predictive modeling. Despite the predictive capabilities of deep learning forecasters, their latent states often lack physical interpretability and may diverge from physically valid system evolution, especially for long-horizon predictions. Classical physics-informed neural networks (PINNs) typically impose physical constraints in the output space by penalizing physically inconsistent predictions after decoding, which constrains only visible errors and leaves latent dynamics largely unstructured.

Phys-JEPA, a physics-informed joint-embedding predictive architecture, directly targets this limitation by moving physical consistency constraints into the latent predictive state space. With a physical–residual decomposition, Phys-JEPA imposes both static (state) and dynamic (transition) physical consistency in the latent space prior to decoding, aimed at learning compact, interpretable, and physically plausible world models.

Figure 1: Motivation of Phys-JEPA—conventional physics-informed forecasting applies losses after decoding, whereas Phys-JEPA enforces physical structure directly in the latent predictive state, shaping hidden dynamics.

Architectural Formulation

Phys-JEPA’s framework is an instantiation of a JEPA (Joint Embedding Predictive Architecture) for temporal forecasting, with a two-branch design: one latent branch is regularized for physical interpretability, while the residual branch captures dynamics not explicitly described by physical variables. The process begins by encoding historical sequences into a predictive latent state, which is then decomposed into physical and residual subspaces. A predictor forecasts the future latent state, which is subsequently decoded to yield the forecasted trajectory. The physical latent branch is supervised using both static and dynamic consistency via a physics projector, and the residual component is regularized to prevent collapse (SIGReg).

Figure 2: Phys-JEPA framework: historical multivariate time series are encoded and decomposed into physical/residual latent features; the future is predicted and physical consistency enforced directly on latent physical states and their transitions.

This formulation enables explicit alignment of representations with domain knowledge, providing a mechanism to enforce not only the magnitude but also the evolution of encoded physical states.

Physical-Latent Decomposition and Constraints

The critical innovation is the latent decomposition:

• Physical component ($\bm{z}^{phy}$): Subvector aligned to known physical variables via a trainable projector, supervised for static and dynamic consistency.
• Residual component ($\bm{z}^{res}$): Captures incompletely described or stochastic aspects, regularized for non-trivial representation with SIGReg.

Static consistency loss enforces agreement between latent physical projections and observed variables at both context and prediction times; dynamic consistency loss enforces consistency between latent transitions and observed transitions across the prediction interval.

Experimental Protocol and Datasets

Benchmarks include Jena Climate (physical meteorological variables), Electricity (multi-client electricity load; weak-domain descriptors), and Traffic (road occupancy; weak-domain descriptors). Metrics are aggregate and per-target mean squared error (MSE) across horizons $H \in \{24,48,96,192\}$, with a primary focus on $H=24$ for ablation.

Numerical Results

H=24 Benchmark Results

Phys-JEPA demonstrates variable improvements depending on descriptor quality and dataset. On Jena Climate, Phys-JEPA reduces aggregate MSE from 0.12482 to 0.12273 and temperature MSE from 0.01892 to 0.01831. On Traffic, Phys-JEPA reduces aggregate MSE from 0.7035 to 0.6846 (H=24), with consistent improvements across all horizons.

Figure 3: Core $H=24$ aggregate MSE comparison on Electricity and Traffic—annotating the differential impact of latent-physics and baseline constraints for infrastructure data.

Feature-level analysis on Jena Climate reveals improvements predominantly on thermodynamic variables; only wind direction remains challenging, indicating a need for proper handling of angular variables.

Figure 4: Per-variable MSE improvement of Phys-JEPA over supervised baseline on Jena Climate at $H=24$, highlighting enhancements on most meteorological components except wind direction.

Horizon Robustness

Phys-JEPA exhibits significant robustness on Traffic—consistently outperforming baselines across all horizons, with the strongest gains at longer horizons ($H=192$). Electricity benefits mainly from static latent constraints at short horizons; only at $H=192$ does the full objective yield the best results.

Figure 5: Completed horizon analysis for aggregate MSE, showing Phys-JEPA’s improvements are most consistent and horizon-stable on Traffic.

Ablation Analysis

Ablation experiments reveal output-level physics provides minimal or no benefit. Static latent physical regularization is most robust on Electricity, whereas on Traffic the full (static+dynamic) latent-physics objective yields the largest improvements, reducing aggregate MSE by 2.69% and target MSE by 5.49% relative to the baseline for $H=24$.

Figure 6: Ablation at $H=24$: only latent constraints, especially static+dynamic, deliver substantial improvements—output-level physical penalties are weak under these benchmarks.

Latent Representation Diagnostics

Latent diagnostics (effective rank, covariance rank) show that imposing physical constraints increases the dimensionality and variability of latent representations. This confirms that the proposed losses impact representation geometry, not just decoded outputs.

Figure 7: Effect of latent physics on representation geometry: effective rank and covariance rank are increased, reflecting higher latent expressivity and structure.

Robustness and Baseline Comparison

Repeated-seed compact experiments confirm the variance-sensitivity of the dynamic loss and the stability of static regularization depending on dataset and optimization regime.

Figure 8: Compact $\bm{z}^{res}$0 repeated-seed robustness check, demonstrating variance and degradation in dynamic-heavy regimes.

Comparison to DLinear, a strong statistical baseline, highlights that Phys-JEPA’s primary contribution is in representational structure and interpretability; DLinear performs competitively or better on certain metrics, indicating essential future work in benchmarking against state-of-the-art forecasting backbones.

Figure 9: Compact DLinear comparison—motivates further suite-based evaluations for claims of architectural superiority.

Hyperparameter Sensitivity

A sweep over latent physical dimension and loss weights demonstrates the anticipated trade-off: stronger dynamic loss improves transition alignment but can harm state alignment or prediction accuracy with poor descriptors.

Figure 10: Sensitivity analysis—optimization trade-offs between accuracy and latent transition alignment, particularly when descriptors only partially encode underlying dynamics.

Implications and Perspectives

Phys-JEPA establishes a methodology for infusing physical interpretability into latent representations and their transitions, rather than treating physical constraints as post hoc output corrections. This improves physical plausibility and stability of forecasts in datasets where descriptors capture meaningful system evolution, and supports theoretical development toward latent world models handshaped by physical priors.

Empirically, benefits are contingent on descriptor quality; weak or aggregate descriptors can render dynamic constraints over-restrictive. The architectural separation into physical/residual subspaces allows targeted regularization and mitigates representation collapse, a frequent challenge in self-supervised predictive learning.

From a practical standpoint, Phys-JEPA’s principle provides a unifying framework for integrating physics in deep sequence models, complementing advances in JEPA and temporal representation learning. The approach is particularly suitable in scientific or operational domains where interpretable, physically structured prediction is critical.

Future Directions

Phys-JEPA’s results underscore several priorities for continued development:

• Expansion to datasets with explicit, high-fidelity physical laws for more direct evaluation of latent physical alignment.
• Design of more expressive or domain-specific physics projectors.
• Comprehensive repeated-seed and strong-baseline comparisons, including architectures such as PatchTST, TimesNet, or iTransformer.
• Systematic ablation of descriptor quality to quantify sensitivity of latent dynamics to physical projection fidelity.

Conclusion

Phys-JEPA advances the organization of latent predictive states in multivariate time-series forecasting by moving physical regularization from output to latent space and introducing static and dynamic latent consistency. Experiments on meteorological and infrastructure benchmarks substantiate improvements in predictive accuracy and physical interpretability, particularly when physical descriptors encode meaningful temporal evolution. Output-space-only physics is less effective, confirming that internal world model regularization is critical. These results support future expansion of physics-informed latent modeling and hybrid neural–physical architectures for interpretable sequence prediction in complex systems.

(2606.16076)