Short-Horizon Autocorrelation

• Short-horizon autocorrelation is a measure of the dependency structure at minimal time lags, universally exhibiting a quadratic decay in various systems.
• It finds applications in quantum physics, financial econometrics, and event-driven modeling by revealing memory effects and predictability of dynamic processes.
• Robust estimation techniques, such as bin-counting and bias-corrected methods, address challenges related to finite data and non-stationary influences.

Short-horizon autocorrelation quantifies the dependence structure of a stochastic process or time series at small temporal (or discrete-event) lags. Formally, it is typically operationalized via the autocorrelation function (ACF) evaluated at short times or small separations, serving as a central tool in characterizing the memory, predictability, and microdynamic organization in domains as varied as quantum many-body physics, financial econometrics, and high-frequency event timing. The universal and system-specific features of short-horizon autocorrelation, as well as the associated estimation and modeling methodologies, have profound implications for both theory and applied analysis.

1. Mathematical Foundations and Universal Laws

Short-horizon autocorrelation is defined, for a stationary process $X(t)$, as %%%%1%%%%, with normalization for zero mean. In quantum many-body systems, for any (traceless) operator $O$ and Hamiltonian $H$, the normalized autocorrelation function is $C(t) = \langle O(t)O(0)\rangle$, with $O(t) = e^{iHt} O e^{-iHt}$ and expectation with respect to a suitable state (e.g., infinite temperature) (Zhang et al., 2023). The short-time expansion reads: $$C(t) = C(0) - \mu_2 t^2/2! + \mu_4 t^4/4! - \ldots,$$ where $\mu_2 = \langle [H, O]^2 \rangle$ and higher moments involve successive commutators.

A universal law emerges at short times: after rescaling $\tau = b_1 t$, where $b_1$ is the first Krylov (Lanczos) coefficient, every normalized autocorrelation function satisfies to $O(\tau^2)$: $C(\tau) \simeq 1 - \tau^2/2 + O(\tau^4),$ regardless of the detailed system or operator. The next-order correction includes the ratio $(b_2/b_1)^2$ but the coefficient of $\tau^2$ is universally $\frac{1}{2}$ (Zhang et al., 2023). In Langevin systems, similar quadratic short-time forms arise, e.g., for the second-order Langevin equation $$d^2\alpha/dt^2 + a\,d\alpha/dt + b^2\alpha = \epsilon(t)$$: the normalized current autocorrelation expands as $c(t) = 1 - (b^2/2)t^2 + O(t^3)$, and $c'(0)=0$—a result not achieved by first-order stochastic dynamics (Belousov et al., 2016).

2. Short-Horizon Autocorrelation in Stochastic and Event-Driven Systems

For discrete-event processes, the ACF at short lags is acutely sensitive to both the law of inter-event times (IETs) and correlations between them. For a point process with inter-event law $P(\tau)$ (mean $\mu$) and weak pairwise correlation coefficient $M$, the autocorrelation at lag $n$ reads (Jo, 2019): $A_M(n) = A_0(n) + M \Delta A(n) + O(M^2),$ where $A_0(n)$ is the renewal (uncorrelated) part and $\Delta A(n)$ is a correction term decaying faster than $A_0(n)$. For exponential IETs, $$A_M(n) \approx e^{-n/\mu} + \frac{4M}{\mu-1} n e^{-2n/\mu}$$, so the $M$-term is a positive, short-lived bump. Heavy-tailed $P(\tau)\propto \tau^{-\alpha}$ shift the observed short-lag scaling exponents upward; the correction $\propto n^{-\alpha}$ dominates at the very shortest lags before more rapid decay sets in.

Higher-order correlations—such as correlation between burst sizes in sequences of events—also directly modulate short-horizon autocorrelation. In time series generated with both exponential IETs and exponentially distributed (but correlated) burst sizes $b$, the initial decay slope of the ACF is linearly and negatively shifted by the copula parameter $\rho$, indicating that positive consecutive burst-size correlations slow the initial falloff of autocorrelation; this is analytically exact up to $O(\rho)$ (Yu et al., 16 Feb 2025).

3. Short-Horizon Autocorrelation in High-Frequency Financial Time Series

Ultra-high-frequency financial returns exhibit short-horizon autocorrelation determined primarily by the long-memory in order directions. The modified Mike–Farmer (MMF) model decomposes the determinants of the return Hurst exponent $H_r$ as a linear function,

$$H_r = 0.25 - 0.02\,\alpha_x - 0.08\,H_x + 0.52\,H_s,$$

where $H_s$ is the Hurst index of order signs, $H_x$ and $\alpha_x$ describe order price depth and heavy-tailedness, respectively (Zhou et al., 2014). Empirical data validate this decomposition: short-horizon autocorrelation in returns is dominated by $H_s$ (herding), while depth and price distributional characteristics play minor roles. Frictional microstructure (discreteness, transaction costs, market rules) typically nullifies arbitrage opportunities even in the presence of persistent $H_r\ne0.5$.

4. Effects of Non-Stationarity, Seasonality, and Market Microstructure

Time-varying patterns, such as intra-day seasonality (e.g., U-shaped activity in equities), systematically bias short-horizon autocorrelation estimators. For a time series $Y(t)$ observed in calendar time with nonstationary intervals $\theta(t)$, the relation between the observed autocorrelation $C_Y(\Delta t)$ and the stationary underlying process $X(\tau)$ is an exact weighted average (Gubiec et al., 2014): $$C_Y(\Delta t) = \int_{\Delta\tau_{\rm min}}^{\Delta\tau_{\rm max}} \rho_{\Delta t}(\Delta\tau) C_X(\Delta\tau) d\Delta\tau,$$ where $\tau$ parametrizes intrinsic (de-seasonalized) time. Concavity arguments show that intra-day seasonality always stretches observed memory: for returns, negative autocorrelation is accentuated; for volatility proxies, decay is slowed by 5–20%. Stationarity-restoring time transforms or weighted estimators are mandatory for unbiased measurement and model calibration at short horizons.

Program trading and algorithmic liquidity provision have shortened the empirical time scale of non-Pearson (nonlinear) volatility autocorrelations from $\sim30$ days to $\sim7$ days, as revealed by block-shuffling and inverse-statistics methodologies (Sándor et al., 2016). These short, non-linear memory horizons underpin phenomena such as gain–loss asymmetry and the leverage effect.

5. Methodologies for Estimating and Interpreting Short-Horizon Autocorrelation

Accurate estimation of short-horizon autocorrelation from finite realizations is nontrivial, particularly for processes with slow decay or heavy tails. In stationary stochastic processes, a robust "bin-counting" estimator computes the two-point joint probability $N(\tau,g_\mu,g_\nu)$ and aggregates the autocovariance as $\sum_{\mu,\nu} x_\mu x_\nu N(\tau,g_\mu,g_\nu)/T$ (Miccichè, 2022). For practical convergence, series length $T$ must be sufficient to suppress tail errors, which decay exponentially for Gaussian-type processes and algebraically for power-law processes. For Markov Monte Carlo chains, bias-corrected logarithmic binning permits on-the-fly recovery of short-lag $\rho(\tau)$ up to any desired horizon with negligible systematic error, using an $O(N)$ time and $O(\log N)$ memory protocol (Wallerberger, 2018).

In empirical high-frequency settings (e.g., electricity markets), short-horizon autocorrelation in both the endogenous outcome and available instruments can induce severe estimation biases in regression or instrumental variable (IV) approaches. Specifically, when both the instrument and the error exhibit AR(1) autocorrelation, the IV estimator for elasticity is inflated by a factor $1/(1-\rho_W\rho_\varepsilon)$. Extended approaches such as conditional IV (with lagged instruments) and nuisance IV (with lagged dependent variables) restore identification and consistency in the presence of pronounced short-horizon autocorrelation (Tiedemann et al., 2023).

6. Control Parameters, Universality, and System Classification

A central insight from quantum dynamics is the role of Krylov–Lanczos complexity: once time is rescaled by the first Lanczos coefficient $b_1$, autocorrelation functions across different operators and systems collapse universally for $\tau = b_1 t \lesssim 1$ (Zhang et al., 2023). The characteristic parameter $\eta = b'(1)/b(1)$, representing the relative growth rate of the Lanczos sequence, organizes the intermediate-time collapse and controls the transition between oscillatory and monotonic autocorrelation decay. Systems with $\eta < 0.5$ display damped oscillations at short horizon, while those with $\eta > 0.5$ exhibit monotonic decay. These observations provide a unifying framework for universality across quantum many-body, classical, and stochastic systems.

---

Short-horizon autocorrelation thus encodes fundamental information about microdynamics, predictability, and memory. Its universal functional forms, dependence on operator complexity, sensitivity to inter-event statistics and system non-stationarities, and practical estimation techniques are now rigorously characterized across domains, with broad implications for both theoretical understanding and applied inference (Zhang et al., 2023, Belousov et al., 2016, Jo, 2019, Yu et al., 16 Feb 2025, Choi et al., 2016, Miccichè, 2022, Wallerberger, 2018, Zhou et al., 2014, Gubiec et al., 2014, Sándor et al., 2016, Tiedemann et al., 2023).