Physical Autoregressive Model (PAR)

• Physical Autoregressive Model (PAR) is a framework that sequentially factors joint probability distributions to model spatial, temporal, and interaction-based physical systems.
• PAR employs various architectures such as RNNs, transformers, and diffusion-based decoders to predict system dynamics and optimize learning of correlations.
• By integrating physical tokenization and modality fusion, PAR enhances the modeling of multi-agent interactions, periodic processes, and lattice-based systems.

A Physical Autoregressive Model (PAR) is a general formalism in which the joint dynamics of physical systems—comprising spatially extended configurations, temporal trajectories, or interacting agents—are modeled by sequentially predicting each element conditioned on prior elements according to a prescribed factorization. PARs unify classical time series models in physics with modern machine learning approaches for multi-dimensional, spatiotemporal, and control-augmented data. The concept is realized via various instantiations across statistical physics, robotics, time series analysis, and multi-agent prediction, linked by their autoregressive decomposition and conditioning structure.

1. Theoretical Foundations and Formal Definitions

A PAR is fundamentally characterized by its sequential factorization of the joint probability distribution. For a configuration $\sigma = (\sigma_1, \ldots, \sigma_N)$—such as a 2D spin lattice or a trajectory over a temporal or spatiotemporal domain—the probability is written as

$$P(\sigma) = \prod_{i=1}^N P(\sigma_i \mid \sigma_{<i})$$

where the index ordering $<i$ is a model design choice and may represent a temporal order, a path through a lattice or graph, or a composite order over modalities (states, actions, observations). In periodic and multivariate settings, the system evolves according to

$$Y_t = \sum_{i=1}^{p} \phi_i(t) Y_{t-i} + \varepsilon_t$$

with potentially time-dependent, periodic coefficients $\phi_i(t)$ and possibly additive noise $Z_t$, yielding

$Y_t = X_t + Z_t,$

where $X_t$ is a periodic AR process and $Z_t$ is noise (Żuławiński et al., 2023, Bracher et al., 2017).

In physical and robotic domains, "physical tokens"—concatenated encodings of observations and action sequences—become the sequential elements, with the entire system's evolution encoded as an autoregressive product over token streams (Song et al., 13 Aug 2025).

2. Architectures and Inference Schemes

PARs are instantiated via a range of architecture choices, from recurrent neural networks (RNNs, LSTMs, GRUs) to transformers and diffusion-based decoders. Key points include:

• Recurrent and Transformer Models: In statistical physics, models such as GRUs (hidden size $N_h=16$) or transformers (2 masked-attention layers, 32 embedding dimensions, feedforward width 512, 4 heads) predict each spin or element given the sequence to that point, with training via cross-entropy loss per element (Teoh et al., 2024).
• Physical Tokenization: In robotics, a PAR streams jointly encoded visual frames and actions (with dense, continuous tokenization using pretrained VAEs and small MLPs), aligned to a frozen instruction context (Song et al., 13 Aug 2025).
• Parallelization and Masking: Teacher-forced parallel training, domain-specific causal or bidirectional attention masking, and inference mechanisms such as KV-cache enhance both convergence speed and online predictivity.

Advanced schemes integrate diffusion models (DiT decoders) to sample continuous-valued frame and action tokens via reverse diffusion, mitigating discretization artifacts and better capturing uncertainty.

3. Index Ordering, Path Dependence, and Conditioning Strategies

A foundational modeling choice in any PAR is the selection of index ordering—how one traverses the system as a sequence:

• Statistical Physics: For lattice systems (e.g., Ising model), imposing a 1D path (zigzag, snake, Hilbert, Morton) on a 2D structure has nontrivial effects on model learning. Long contiguous runs (e.g., zigzag, snake) accelerate learning of long-range, power-law correlations even if they break local neighbor relations; locality-preserving paths (Hilbert, Morton) encode geometric neighborhoods more faithfully but converge more slowly for critical phenomena. The sequence design thus mediates between fast propagation of correlations and preservation of physical locality (Teoh et al., 2024).
• Multi-Agent and Spatiotemporal Systems: In poly-autoregressive frameworks for agent interactions, the index is factored over both agents and time. Conditioning for each agent's next state leverages the past of all agents, with agent-specific token identities and next-timestep prediction heads, as opposed to generic next-token factorization (Thakkar et al., 12 Feb 2025).
• Robotics: PARs combine frame and action tokens, conditioning predicted actions not just on past actions, but also on past and predicted frames, realizing implicit inverse kinematics via the attention pattern (Song et al., 13 Aug 2025).

The index ordering reflects the modeling priorities—speed of correlation learning, preservation of physical context, or computational tractability.

4. Model Selection, Validation, and Fit Assessment

Model order (autoregressive lag), period (for periodic processes), and coefficient estimation are critical in PAR fitting. In the context of periodic AR processes with noise:

• Order and Period Selection: The Bayesian Information Criterion (BIC) is generalized for block-dependent residuals. The parameter count includes both periodic AR coefficients and variance components, with likelihoods derived from block empirical characteristic functions (CFs), and residual blocks validated for independence. For Gaussian settings, closed-form multivariate PDFs enable direct BIC calculation; non-Gaussian residuals require multidimensional FFT inversion (Żuławiński et al., 2023).
• Residual Analysis: Independence and fit are tested by comparing empirical and theoretical CFs over residual vectors, with parametric bootstrap used to obtain $$P(\sigma) = \prod_{i=1}^N P(\sigma_i \mid \sigma_{<i})$$0-values.
• Practical Workflow: Data preprocessing (detrending, standardization), fitting via modified errors-in-variables or Yule-Walker solvers, and residual block validation are standard.

For physical generative models, performance metrics include negative log-likelihood per element, power-law correlation reproduction, and downstream task success rates (e.g., task completion in robotics (Song et al., 13 Aug 2025), predictive error in agent trajectories (Thakkar et al., 12 Feb 2025)).

5. Applications and Domain-Specific Instantiations

PARs are realized in numerous domains:

• Lattice Statistical Mechanics: Modeling full spin configurations, capturing high-order spatial correlations near critical points, and exploring path dependence of learning and sampling efficiency (Teoh et al., 2024).
• Robotic Manipulation: Extending autoregressive video world models to robotics by integrating action streams as physical tokens enables transfer to tasks with little or no action-pretraining. Diffusion-based de-tokenization and causal/inverse-kinematics-aware attention yield sample-efficient, successful manipulation policies (Song et al., 13 Aug 2025).
• Time Series with Periodicity/Noise: In classical settings, periodic AR processes with external or seasonal driving and additive noise model periodicity in engineering and condition monitoring, with multivariate extensions enabling epidemic surveillance and environmental monitoring (Żuławiński et al., 2023, Bracher et al., 2017).
• Multi-Agent Interaction: Poly-autoregressive factorization enables accurate prediction of physical and social agent trajectories, outperforming single-agent AR in highly interactive, physically coupled environments (Thakkar et al., 12 Feb 2025).

6. Principal Findings and Broader Lessons

Research demonstrates several cross-domain generalizations and insights:

• Choice of Sequence Path: For non-1D or spatially extended physical systems, autoregressive path design is critical. Aligning the sequence with principal correlations or optimizing the path as a trainable parameter can enhance learning efficiency and model fit (Teoh et al., 2024).
• Compositional Tokenization and Multimodal Fusion: In robotics and multi-agent settings, representing states, actions, and observations as multiplexed tokens in a joint LATEX space allows seamless transfer from vision/LLMs to physical systems (Song et al., 13 Aug 2025, Thakkar et al., 12 Feb 2025).
• Effect of Model Components: Ablation studies indicate that both the autoregressive backbone and continuous, diffusion-based decoders are essential for optimal prediction and control; agent-ID and time-aware position encodings are crucial for agent interaction modeling (Thakkar et al., 12 Feb 2025, Song et al., 13 Aug 2025).
• Validation under Physical Constraints: Empirical/functional criteria tailored to the joint dependency structure yield more robust model selection than classical scalar AIC/BIC, while periodic, multivariate recursion relations allow analytic computation of long-run means, variances, and cross-correlations for surveillance and anomaly detection (Bracher et al., 2017, Żuławiński et al., 2023).

A plausible implication is that future PARs will increasingly incorporate path or ordering optimization, heterogeneous modality fusion, and physically grounded conditioning patterns to integrate domain structure with scalable neural architectures.