Simultaneous Inference for Covariance and Precision Matrices of Long-Range Dependent Time Series

Abstract: For time series with long-range temporal dependence, inference for covariance and precision matrices is non-trivial. We propose a Berry-Esseen type Gaussian approximation result that gives a finite-sample bound for the Kolmogorov distance between the infinity norms of the estimation error of sample covariance matrix and the corresponding Gaussian approximation. The method utilizes martingale and m-dependent approximation and relies on constructing triadic blocks. We also establish a bootstrapping result with block sampling method, which preserves validity despite strong temporal dependence. Our results on covariance allow ultra-high-dimensional settings where the dimension of time series can grow sub-exponentially with sample size. Similar results can be built for precision matrix under low-dimensional settings. No assumption is required on the structure of covariance and precision matrices.

• The paper develops non-asymptotic Gaussian approximation for maximum error estimation in dense covariance and precision matrices under long-range dependence.
• It introduces a block bootstrap method validated for ultra-high dimensions, enabling simultaneous inference without relying on structural sparsity.
• Empirical results, including fMRI applications, confirm the method’s efficacy in detecting connectomic differences in neuroimaging studies.

Simultaneous Inference for Covariance and Precision Matrices of Long-Range Dependent Time Series

Introduction and Contributions

"Simultaneous Inference for Covariance and Precision Matrices of Long-Range Dependent Time Series" (2604.16219) advances the theory and practical methodology for high-dimensional inference in temporally dependent data, specifically time series with long-range (long-memory) dependence. The work addresses the significant challenge of developing non-asymptotic, high-dimensional Gaussian approximation and valid bootstrap methods for simultaneous inference on both covariance and precision matrices, with minimal structural assumptions on the population matrices.

The framework considers $\{X_i\}_{i=1}^n$, a stationary vector-valued Gaussian linear process in $\mathbb{R}^p$, modeled as

$X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$

where $\epsilon_t$ are i.i.d. innovations and the coefficients $A_t$ decay as $t^{-\beta}$ for some $\beta>3/4$. The decay rate $\beta$ governs the degree of temporal dependence; $\beta<1$ yields long-memory. The principal focus is on the high-dimensional regime with $p \gg n$ and where the covariance and precision matrices can be dense and unstructured.

The core contributions are as follows:

1. Berry–Esseen-type non-asymptotic Gaussian approximation: Provides finite-sample Kolmogorov bounds for the maximum deviation ($\mathbb{R}^p$0) between the empirical covariance (and, in certain regimes, the empirical precision) matrix entries and their Gaussian approximations in regimes of strong long-range dependence. The dimension $\mathbb{R}^p$1 is permitted to grow sub-exponentially with $\mathbb{R}^p$2.
2. High-dimensional block bootstrap validity without short-memory assumptions: Delivers a finite-sample block bootstrap guarantee via triadic block constructions and $\mathbb{R}^p$3-dependent approximations, applicable even when the process exhibits strong (long-range) temporal dependence.
3. Simultaneous inference in ultra-high dimensions without structural assumptions: Critically, both the covariance and (where invertible) precision matrix results require no sparsity or banding assumptions, contrasting existing literature that relies heavily on such constraints.
4. Empirical validation and neurological application: Simulations cover both low- and high-dimensional ($\mathbb{R}^p$4 up to several thousand) regimes, with practical demonstrations using functional MRI (fMRI) data to delineate connectomic differences between autistic and neurotypical populations.

Main Theoretical Results

1. Non-Asymptotic Gaussian Approximation

For the sample covariance $\mathbb{R}^p$5, the maximum norm estimation error $\mathbb{R}^p$6 is the focus of inference. Let $\mathbb{R}^p$7 be a zero-mean Gaussian random vector with covariance matching the appropriately scaled vectorization of $\mathbb{R}^p$8. Define the Kolmogorov distance between the distributions of $\mathbb{R}^p$9 and $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$0: $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$1

The primary upper bound is: $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$2 for $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$3, under the requirement $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$4.

This result allows for dimensions $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$5 that grow sub-exponentially with $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$6, generalizing prior results which required much slower ($X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$7) growth or short-memory. The proof employs martingale deviation bounds, $X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$8-dependent truncation, and triadic block constructions enabling decoupling, followed by an application of modern high-dimensional CLT tools.

Figure 1: QQ-plots for the covariance matrix, low-dimensional regime ($X_i = \sum_{t=0}^\infty A_t \epsilon_{i-t},$9, short memory), $\epsilon_t$0 versus the Gaussian approximation $\epsilon_t$1.

2. Block Bootstrap Consistency under Long-Range Dependence

For high-dimensional settings where direct estimation of the long-run covariance for Gaussian approximation is infeasible, the block bootstrap is developed. An overlapping block empirical distribution function is constructed as: $\epsilon_t$2 where each block $\epsilon_t$3. The block length $\epsilon_t$4 is tuned according to the dependence structure (larger for stronger memory).

The finite sample bootstrap validity guarantee is: $\epsilon_t$5 where $\epsilon_t$6 mirrors the structure of $\epsilon_t$7 in the Gaussian approximation and the Kolmogorov distance between the bootstrap empirical distribution and the true distribution of the maximum absolute entry error converges to zero as $\epsilon_t$8 under the same dimensional restrictions. This establishes legitimate simultaneous confidence regions for all entries of $\epsilon_t$9.

Figure 2: QQ-plots for covariance, $A_t$0 (ultra-long memory), showcasing breakdown of Gaussian approximation and bootstrap.

3. Precision Matrix Inference (Low-Dimensional $A_t$1 Case)

In requirements $A_t$2 (so $A_t$3 invertible), the sample precision matrix $A_t$4 is considered. The work supplies non-asymptotic Kolmogorov bounds for the distribution of $A_t$5 compared to its Gaussian (and bootstrap) approximations. The rate now depends crucially on $A_t$6, allowing for polynomial growth but requiring control over matrix norms. No sparsity, banding, or other regularity is assumed for the precision matrix.

Figure 3: QQ-plot for precision matrix error $A_t$7 and Gaussian approximation, $A_t$8.

4. Empirical and Applied Implications

Simulations demonstrate validity of the theory for both covariance and precision matrix inference in realistic $A_t$9 settings, with clear performance degradation when temporal dependence is sufficiently strong (e.g., $t^{-\beta}$0) as predicted by the theory.

An applied analysis is conducted on fMRI data from the ABIDE project, distinguishing the connectomic patterns of individuals with autism spectrum disorder (ASD) compared to neurotypical controls. Using block bootstrap-based inference of the brain connectivity (conditional independence) graphs, the study detects known neurofunctional differences (e.g., alterations in connectivity of the frontal and posterior regions in ASD).

Figure 5: Conditional independence graph (CIG) recovered by block bootstrap–based inference in brain fMRI data for ASD.

Methodological Innovations

• Triadic block decomposition: Enables transformation of high-dimensional dependent time series into a setting where high-dimensional CLT and Gaussian approximation results for independent data can be leveraged.
• Martingale and $t^{-\beta}$1-dependent approximation analysis: Provides tight exponential bounds even without underlying sparsity.
• Non-reliance on structural assumptions: Results apply to general dense covariance/precision matrices.
• Data-driven confidence regions: Simultaneous, high-dimensional, and valid under strong dependence.

Practical and Theoretical Implications

• High-dimensional time series analysis: The methodology supports valid simultaneous inference in high-$t^{-\beta}$2 time series, a pervasive demand in econometrics, finance, neuroscience, and genomics.
• Inference for dense graphs: Robust hypothesis testing and uncertainty quantification are enabled in graphical models, without sensitivity to sparse prior structures.
• Foundation for robust structure learning: The results provide the base for rigorously justified feature/structure selection in graphical models arising from dependent data.
• Guidance for block bootstrap practice: The work yields theoretically-optimal block size scaling for inference in strong dependence, with implications for empirical researchers applying bootstrap in such contexts.

Limitations and Future Directions

The Gaussian and bootstrap approximation for the precision matrix strictly holds in the $t^{-\beta}$3 setting. In high-dimensional ($t^{-\beta}$4) precision estimation, extensions to regularized (CLIME, graphical Lasso) estimators and characterization of their inference properties under long-range dependence are important open problems.

The practical block size selection remains an empirical art, although the theory gives asymptotic scaling. Future advances may allow adaptive, data-driven optimal block-length setting even in ultra-high-dimensional or ultra-long-memory regimes.

Extending Gaussian approximation techniques to handle non-Gaussian sub-exponential innovations or to richer settings (functional, random field, or nonstationary data) is another natural trajectory, as is the development of post-selection inference and debiased estimators in high-dimensional dependent models.

Conclusion

This paper provides a comprehensive non-asymptotic theory for high-dimensional, strongly dependent (long-memory) time series, establishing both Gaussian approximation rates and block bootstrap valid inference for covariance and (in $t^{-\beta}$5) precision matrices independent of sparsity or structure. The simulation and applied sections document its empirical value. The results are fundamental for modern statistics on high-dimensional dependent data, with methodological and application impact across disciplines such as finance and neuroscience.