Functional PCA: Concepts & Extensions

• Functional PCA (FPCA) is a method that decomposes infinite-dimensional functions using spectral analysis to reveal dominant modes of variation.
• It employs the Karhunen–Loève expansion to estimate mean functions, eigenfunctions, and component scores for efficient dimension reduction.
• Extensions of FPCA address challenges such as sparse sampling, non-Gaussian noise, and high-dimensionality, broadening its applications across diverse fields.

Functional Principal Component Analysis (FPCA) is a central technique in the analysis of infinite-dimensional data where each observation is modeled as a function. By decomposing the primary modes of variation through spectral analysis of the covariance operator, FPCA provides parsimonious representations for functional data, supports dimension reduction, and underpins diverse inference and prediction tasks across the physical, biomedical, financial, and engineering sciences.

1. Core Concepts and Theoretical Foundation

FPCA models each square-integrable random function $X(t): \mathcal{T} \to \mathbb{R}$ via the (truncated) Karhunen–Loève expansion: $X(t) = \mu(t) + \sum_{k=1}^K \xi_k \phi_k(t)$ where $\mu(t)$ is the mean function, $\{\phi_k\}$ are orthonormal eigenfunctions of the covariance operator $C(s, t)$, and $$\xi_k = \int_{\mathcal{T}} (X(t) - \mu(t)) \phi_k(t) dt$$ are uncorrelated principal component scores with $E[\xi_k^2]=\lambda_k$. This framework generalizes multivariate principal component analysis to infinite-dimensional $L^2$ or reproducing kernel Hilbert spaces (RKHS).

The eigendecomposition of the covariance operator $C: L^2(\mathcal{T}) \to L^2(\mathcal{T})$, defined as $(Cf)(t) = \int_{\mathcal{T}} C(s, t) f(s) ds$, yields eigenvalues $\lambda_1 \geq \lambda_2 \geq \cdots \geq 0$ and corresponding eigenfunctions, which determine the dominant directions of functional variation. For high-dimensional or replicated settings, Mercer’s theorem and Sobolev norm convergence control the behavior of these eigenfunctions and the approximation quality of finite truncations (Zhou et al., 2022).

FPCA extends to settings where the functions are only partially observed, contaminated by outliers, non-Gaussian, exhibit structured missingness, involve covariates, are observed as point processes, are truncated, or require joint modeling across multiple functional variates or with hierarchical/Bayesian regularization.

2. FPCA with Sampling, Sparsity, and Structural Constraints

Observing full trajectories is often impractical; typically, only noisy, finite samples of the underlying functions are available. In the RKHS framework, the sampling operator $\Phi: \mathcal{H} \to \mathbb{R}^m$, modeled as $$\Phi f = [\langle \phi_1, f \rangle_{\mathcal{H}},\ldots,\langle \phi_m, f \rangle_{\mathcal{H}}]^T$$, transfers smoothness from the infinite-dimensional space to finite-dimensional sampled vectors (Amini et al., 2011). Time sampling and frequency (basis) truncation arise as special cases through judicious basis choices, such as kernel sections or eigenfunctions of the RKHS kernel operator.

To recover the leading $r$-dimensional subspace, FPCA under sampling constraints solves the constrained M-estimator: $$\widehat{Z} \in \argmax_{Z \in \mathcal{V}_r(\mathbb{R}^m): \langle K^{-1}, ZZ^\top \rangle \leq 2r\rho^2} \langle \widehat{\Sigma}_n, ZZ^\top \rangle,$$ where $\mathcal{V}_r(\mathbb{R}^m)$ is the Stiefel manifold and $K$ is the sampling Gram matrix (Amini et al., 2011). Smoothness constraints ensure that the estimated principal components do not overfit sampling noise. After estimation in $\mathbb{R}^m$, functions are reconstructed in the infinite-dimensional space by $\widehat{f}_j = \Phi^* K^{-1} \widehat{z}_j$. This construction is minimax optimal; that is, the estimator achieves the best possible rate under the given sampling regime.

High-dimensional functional data (e.g., neuroimaging arrays, EEG grids) present statistical and computational challenges when the number of processes $p$ is large relative to sample size $n$. The sparse FPCA framework exploits weak $\ell_q$ sparsity, whereby the energy vector across coordinates follows $V_{(j)} \leq C j^{-2/q}$ for $q \in (0,2)$. A thresholding rule on basis coefficient variances (via a preselected or estimated basis $\{b_l\}$) efficiently screens out negligible coefficients, yielding a parsimony–accuracy trade-off. The approach realizes lower computational complexity and exhibits improved performance in comparison to standard multivariate FPCA or separate univariate FPCAs, especially evident in applications to simulated and EEG data (Hu et al., 2020).

3. Extensions for Covariates, Non-Gaussianity, and Robustness

FPCA has been extended to accommodate covariate-dependent mean and covariance structures, as in Covariate Dependent FPCA (CD-FPCA), where both $\mu(t, z)$ and $G(t, s|z)$ vary smoothly with covariates $z$ via tensor-product spline bases. The covariance is modeled as

$$G(t,s|z) \approx b(t)^\top \Gamma(z) b(s), \quad \Gamma(z) = C(z)C(z)^\top,$$

enforcing positive semi-definiteness for every $z$ and allowing flexible, smooth variation of eigenspaces and variances (Ding et al., 2020). Estimation leverages penalized likelihood, roughness penalties on mean/covariance, and matrix determinant lemma/Sherman-Morrison-Woodbury for computational tractability.

For data exhibiting non-Gaussianity or contamination, robust FPCA approaches alter the definition of the covariance operator. The pairwise spatial sign (PASS) FPCA replaces standard covariance with

$$K(s, t) = E[(X(s)-\tilde{X}(s))(X(t)-\tilde{X}(t))/\|X-\tilde{X}\|^2],$$

delivering robustness to outliers and heavy tails under weakly functional coordinate symmetric (wFCS) distributional assumptions (Wang et al., 2021). Eigenfunctions of the PASS operator coincide with those of the true covariance under mild conditions, ensuring interpretable results even under significant asymmetry or contamination.

Another robustification replaces covariance with a functional Kendall’s $\tau$: $K(s, t) = E[(X(s)-X'(s))(X(t)-X'(t))/\|X-X'\|^2],$ with the notable property that its eigenfunctions are identical to those of the covariance operator, ensuring correct principal modes of variation irrespective of underlying noise distribution (Zhong et al., 2021). Asymptotic consistency is guaranteed for both operator and eigenfunction estimates.

4. Inference for Irregular, Truncated, and Sparse Data

When functional data are irregularly or sparsely sampled, e.g., in longitudinal biomedical studies, adaptation of FPCA relies on reconstructing the mean and covariance surfaces using local smoothing. In the truncated data setting—where instrument error or measurement bounds truncate values to an interval $[a,b]$—naive FPCA is biased. Maximum kernel-weighted local likelihood estimation, adjusting for truncation at each grid point, recovers unbiased smooth mean and covariance estimates; the resulting covariance estimator remains positive semi-definite without post-hoc correction (Murphy et al., 8 Jul 2024). FPC scores are then predicted using Monte Carlo sampling from the conditional truncated normal, supporting regression or classification in generalized functional linear models. Theoretical convergence rates that directly reflect the effect of truncation and the smoothness of the estimators are established.

For sparse repeated measures and missing data, FPCA implemented using PACE (conditional expectation projection under a Gaussian process model) robustly estimates principal components and subject scores, provided the missing data mechanism is Missing at Random (MAR) (Ségalas et al., 16 Feb 2024). Under MNAR or heavily structured dropout, both FPCA and mixed-effects models fail without explicit modeling of the missingness process.

5. Specialized Extensions: Point Processes, Densities, and High-Dimensionality

For replicated point processes, such as earthquake or neurospike event data, functional PCA is defined via the cumulative mass function $F_\mu(t) = \mu([0, t])$ of the associated centered random measure. The covariance kernel $K_\Delta(s, t)$ admits a Mercer expansion, and the Karhunen–Loève expansion of the centered measure $\Delta = \Pi - W$ (with principal measures $\mu_j$ defined as distributional derivatives of the eigenfunctions) provides a direct series representation suited for the analysis of variability in point pattern data (Picard et al., 30 Apr 2024). This approach achieves parametric convergence rates for the eigenelements and is supported by software such as “pppca”.

For density-valued data (data on Bayes spaces), compositional constraints motivate robust FPCA based on an infinite-dimensional extension of the Mahalanobis distance and regularized whitening. The robust density PCA (RDPCA) uses a trimmed estimator on the Bayes space covariance, reducing the influence of outlying or anomalous densities (Oguamalam et al., 26 Dec 2024).

High-dimensional regime-specific approaches, including factor-guided FPCA (FaFPCA), decompose observations via a factor model before applying FPCA to the latent factors, exploiting both temporal and cross-sectional correlation structures. This approach is computationally tractable via closed-form moment equations and avoids iterative rotations, with statistical properties controlled through large $p$ asymptotics (Wen et al., 2022).

6. Bayesian and Multilevel FPCA

Recent development of fully Bayesian FPCA (e.g., Fast BayesFPCA, MSFAST) models the eigenfunctions as parameters with a uniform Stiefel manifold prior, projected onto an orthonormal spline basis. The polar decomposition $X (X^\top X)^{-1/2}$ of a matrix $X$ with i.i.d. standard normal entries ensures a uniform prior on the manifold, facilitating computational stability, uncertainty quantification, and scalable posterior inference. Variability of principal components is fully propagated into interval predictions and inferences (Sartini et al., 15 Dec 2024, Sartini et al., 3 Sep 2025). For multivariate, sparsely observed functional data, extensions such as MSFAST model multiple functional outcomes, standardize each variable for stability, structure mean/eigenfunctions as spline expansions, align posteriors via Procrustes, and employ parallel computation for scalability. Dynamic prediction and unparalleled inference reliability in low signal settings are achieved in contemporary child growth and clinical studies (Sartini et al., 3 Sep 2025).

7. Applications and Impact

FPCA underlies functional regression (Jiang et al., 2022), time series forecasting (Jasiak et al., 26 May 2025), classification, and clustering, and is crucial in domains including neuroimaging, smart grid energy load analysis (Beretta et al., 2019), epidemiology, and spectroscopy. In time series, the KL dynamic factor model enables forecasting of functional series like intraday financial returns, with FPCA eigenscores modeled via VAR or GARCH, often outperforming classical ARMA and machine learning approaches in both interval and directional forecasting.

Localized FPCA (L-FPCA) exploits block structure in the covariance to yield eigenfunctions supported locally on subdomains, directly mirroring the underlying stochastic process’s sparsity and resolving interpretation and over-regularization concerns associated with penalized approaches (Battagliola et al., 3 Jun 2025).

Robust, nonparametric, and computationally scalable extensions of FPCA have substantially extended its relevance and practicality, supporting sophisticated inference and decision-making in modern high-dimensional and heterogeneous functional data environments.

---

This synthesis reflects the breadth and technical depth of FPCA, its diverse modeling generalizations, and the rigorous statistical theory supporting modern applications. For implementation and application in specific domains or under bespoke data regimes, see the cited arXiv contributions for methodological and theoretical details.