Nonstationary Linear Discriminant Analysis

• NSLDA is a family of methods that extends classical LDA to account for evolving class distributions through dynamic estimation techniques.
• The methodology employs state-space and exposure-indexed models to update class means, covariances, and discriminant directions over time.
• Empirical results highlight improved classification accuracy under drift, noise, and sparse data conditions compared to static LDA.

Nonstationary Linear Discriminant Analysis (NSLDA) denotes a class of discriminant methods in which the class-conditional distributions are allowed to evolve with time or another observed index variable, so the separating hyperplane is itself nonstationary. In the state-space formulation introduced in "A State-Space Approach to Nonstationary Discriminant Analysis" (Xie et al., 22 Aug 2025), NSLDA is the homoskedastic Gaussian special case of nonstationary discriminant analysis: class priors, class means, and the common covariance are indexed by time and estimated by smoothing, after which the usual LDA rule is applied at the target time. Closely related exposure-indexed formulations treat nonstationarity through a smoothly varying Bayes direction (Bao et al., 2022) or through a locally estimated discriminative subspace followed by LDA (Ouyang et al., 2024).

1. Formal statistical setting

The central departure from stationary LDA is that the class-conditional density is indexed by time or exposure. In the state-space NSLDA framework, the nonstationary Bayes classifier at time $k$ is defined through

$$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$

with decision rule

$$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$

Under a Gaussian model,

$$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$

so the discriminant takes the standard quadratic form in the time-indexed parameters (Xie et al., 22 Aug 2025).

NSLDA is obtained by imposing a time-wise equal-covariance condition,

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$

The classifier then remains linear at each time $k$, even though the coefficients depend on $k$. This preserves the algebraic structure of classical LDA while abandoning its stationarity assumption.

A related exposure-indexed formulation replaces time $k$ by an observed variable $U\in[0,1]$ and assumes

$$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$

The Bayes discriminant direction is then

$$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$0

which makes explicit that the optimal separating direction is a function of $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$1 rather than a fixed vector (Bao et al., 2022).

This formulation is intended for temporal distribution shift, where $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$2 changes with time. The state-space paper distinguishes this from covariate shift by emphasizing that the conditional class distributions themselves evolve, so the optimal boundary is time-dependent rather than merely requiring reweighting of a fixed rule (Xie et al., 22 Aug 2025).

2. State-space construction of NSLDA

In the linear-Gaussian state-space model, each class has a latent state $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$3 evolving as

$$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$4

where $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$5, $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$6, and $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$7. Here $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$8 is the latent class centroid or state at time $$f_k^j(x)=p(x\mid Y_k=j), \qquad \pi_k^j=\mathbb P(Y_k=j),$$9, $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$0 is the dynamics matrix, $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$1 is process noise, and $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$2 is measurement noise (Xie et al., 22 Aug 2025).

These assumptions induce time-varying Gaussian observation moments,

$$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$3

and

$$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$4

Accordingly, nonstationarity is represented as latent dynamical drift in the class centroids and, more generally, in the class-conditional second moments.

A notable technical feature is that the model is adapted to multiple samples at the same time step. Rather than assuming a single observation per time, the paper introduces

$$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$5

so that each time point may contain several measurements from class $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$6. The resulting smoother performs one state prediction from $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$7 to $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$8 and then processes the $$\psi_k^*(x)=\arg\max_j D_k^{*,j}(x), \qquad D_k^{*,j}(x)=\log \pi_k^j+\log f_k^j(x).$$9 observations through sequential measurement updates at the same time index (Xie et al., 22 Aug 2025).

This differs materially from fitting independent time-slice LDAs. Independent slices ignore the dynamical prior and therefore do not borrow strength across neighboring times; the state-space construction was introduced precisely to use all historical data while retaining a time-specific rule.

3. Time-indexed discriminant rule

Under the equal-covariance assumption, the nonstationary discriminant at time $$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$0 is

$$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$1

and the NSLDA classifier is

$$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$2

For two classes, the decision can be written in the familiar linear form

$$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$3

$$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$4

with decision rule: decide class $$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$5 iff $$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$6 (Xie et al., 22 Aug 2025).

This representation clarifies the meaning of NSLDA. The classifier is still a plug-in LDA rule, but the plug-in quantities are time-indexed and obtained from a dynamical model rather than from pooled sample averages. The boundary is linear conditionally on time, not globally linear over the full dataset.

In the exposure-indexed varying-coefficient formulation, the oracle rule is written as

$$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$7

where $$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$8 and $$x_k\mid (Y_k=j)\sim \mathcal N(\mu_k^j,\Sigma_k^j),$$9 is aligned with the Bayes direction up to a positive scalar factor:

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$0

The sign of the discriminant score is therefore preserved even though the target is estimated through a least-squares characterization rather than direct inversion of a local covariance matrix (Bao et al., 2022).

A closely related dynamic subspace formulation defines

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$1

and introduces the total covariance

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$2

The top eigenvectors of this matrix define a $\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$3-dependent discriminative subspace, and ordinary LDA is then applied after projection (Ouyang et al., 2024).

4. Estimation procedures

For linear-Gaussian drift, NSLDA uses a modified Kalman smoother. Initialization is

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$4

The forward prediction step is

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$5

For each observation $\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$6 at time $\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$7, sequential measurement updates are applied:

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$8

$\Sigma_k^0=\Sigma_k^1\equiv \Sigma_k.$9

After all samples at that time have been processed,

$k$0

Backward smoothing then uses the Rauch–Tung–Striebel recursion

$k$1

$k$2

with the corresponding covariance update (Xie et al., 22 Aug 2025).

When system parameters are unknown, the same paper uses expectation-maximization. The E-step computes

$k$3

using smoothed moments $k$4, $k$5, and $k$6. The M-step then admits closed-form updates such as

$k$7

$k$8

If $k$9 is also unknown,

$k$0

The same framework also addresses missing or uncertain time labels through a GMM–Kalman approximation. Each time index is treated as a Gaussian mixture component with propagated moments

$k$1

and each sample is assigned a hard time label by

$k$2

For nonlinear or non-Gaussian drift, particle smoothing is used to estimate time-varying class centroids, yielding fully nonstationary discriminant rules (Xie et al., 22 Aug 2025).

5. Related nonstationary LDA formulations

Two recent lines of work are directly relevant to NSLDA even though they do not use that exact acronym.

The first is varying coefficient linear discriminant analysis (VCLDA), which models the Bayes direction as a smooth function of an exposure variable. Its estimation strategy is based on the least-squares population problem

$k$3

where

$k$4

Each coefficient function is approximated by a B-spline basis $k$5 through

$k$6

In low dimension, the estimator has the closed form

$k$7

while in high dimension it is replaced by the group-lasso-type program

$k$8

The paper emphasizes that this is more computationally efficient than the dynamic linear programming rule because it avoids repeated pointwise optimization at each new $k$9 (Bao et al., 2022).

The second is dynamic supervised principal component analysis (DSPCA), which produces a nonstationary LDA-like rule by local kernel smoothing and supervised low-rank projection. For a target index $k$0, class-specific means are estimated by the Nadaraya–Watson estimator

$k$1

with pooled covariance estimate

$k$2

and total covariance

$k$3

After spectral decomposition, the top $k$4 eigenvectors define a local discriminative subspace, and standard LDA is run on the projected data. A computational shortcut replaces direct eigendecomposition of the $k$5 matrix by an equivalent $k$6 matrix when $k$7 (Ouyang et al., 2024).

These approaches illuminate two distinct conceptions of nonstationarity. VCLDA estimates a smoothly varying discriminant direction directly; DSPCA estimates a smoothly varying discriminative subspace and then classifies within that subspace; the state-space NSLDA model estimates time-indexed class moments through latent dynamics and then plugs them into the LDA rule. All three are responses to the same failure mode of stationary LDA: pooled means and covariances become misspecified when class structure drifts.

6. Theory, empirical behavior, and limitations

The theoretical emphasis differs across formulations. In the state-space NSLDA paper, the primary contribution is a unified model-based framework with extensive simulations rather than a direct asymptotic classifier theorem (Xie et al., 22 Aug 2025). In the exposure-indexed literature, stronger formal guarantees are available. For VCLDA, the low-dimensional spline estimator satisfies

$k$8

and with

$k$9

the rate becomes

$U\in[0,1]$0

The excess misclassification risk satisfies

$U\in[0,1]$1

with corresponding $U\in[0,1]$2 rates in both low- and high-dimensional regimes (Bao et al., 2022).

For DSPCA-based dynamic LDA, theory is built around the spiked covariance model

$U\in[0,1]$3

If

$U\in[0,1]$4

the discriminant direction lies in the span of the first $U\in[0,1]$5 eigenvectors of $U\in[0,1]$6. The estimated dynamic subspace obeys the uniform bound

$U\in[0,1]$7

and, under additional scaling assumptions,

$U\in[0,1]$8

where

$U\in[0,1]$9

These results justify nonstationary dimension reduction rather than direct estimation of a time-varying normal vector (Ouyang et al., 2024).

Empirically, the state-space NSLDA paper reports lower misclassification error than stationary LDA, QDA, and SVM baselines, together with robustness to noise, missing data, sparse time slices, and class imbalance; its advantage grows as drift increases, and the nonlinear version based on particle smoothing remains stable when naive LDA deteriorates under higher noise (Xie et al., 22 Aug 2025). The VCLDA paper reports that in dynamic settings classical LDA deteriorates sharply, often close to random guessing, whereas VCLDA consistently outperforms classical LDA and DLPD; on diffuse large B-cell lymphoma data using age as the exposure, the reported average misclassification rates were $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$0 for LPD excluding age, $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$1 for LPD including age as a covariate, $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$2 for DLPD with age as exposure, and $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$3 for VCLDA with age as exposure (Bao et al., 2022). The DSPCA paper reports that, in synthetic examples where class means and covariances vary with $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$4, DSPCALDA and especially DSPCAQDA often outperform static methods such as LPD, POTD, SVM, and KNN, and are competitive with or close to the oracle classifier; on breast-cancer datasets, dynamic methods generally outperform static ones when the index variable is informative (Ouyang et al., 2024).

The limitations are correspondingly model-specific. State-space NSLDA presumes a meaningful temporal index and relies on a linear-Gaussian model unless Kalman smoothing is replaced by particle smoothing (Xie et al., 22 Aug 2025). VCLDA assumes smooth coefficient functions, bounded covariance eigenvalues, and in the high-dimensional theory a common active set across $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$5 (Bao et al., 2022). DSPCA depends on tuning the bandwidth $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$6, the total-covariance weight $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$7, and the reduced dimension $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$8; its guarantees rely on smoothness and spiked covariance assumptions, and although it is computationally much cheaper than DLPD, it still requires kernel smoothing and eigendecomposition at each target $$X\mid (Y=1,U=u)\sim \mathcal N(\mu_1(u),\Sigma(u)),\qquad X\mid (Y=0,U=u)\sim \mathcal N(\mu_2(u),\Sigma(u)).$$9 (Ouyang et al., 2024).

Taken together, these formulations establish NSLDA not as a single algorithm but as a family of time-aware or exposure-aware LDA procedures. What unifies them is the replacement of one global stationary discriminant rule by a collection of local or smoothed rules indexed by time or exposure, each designed to track drifting class distributions without collapsing all observations into a single pooled regime.