---
title: Preferential Subspace Identification (PSID)
url: https://www.emergentmind.com/topics/preferential-subspace-identification-psid
type: topic
---

# Preferential Subspace Identification (PSID)

Preferential Subspace Identification (PSID) is a linear system identification methodology designed for multivariate time series, particularly where a primary signal (such as neural activity) is used to predict a secondary signal (such as behavior). PSID constructs a state-space model that extracts latent dynamical structure from a "primary" observation sequence to achieve optimal prediction of a "secondary" sequence. Originally developed for optimal prediction given past information ("innovations form"), recent advances have extended PSID to incorporate optimal filtering (using concurrent data) and forward–backward smoothing (using all available data), leveraging both primary and secondary signals for enhanced dynamic state estimation and decoding [2507.15288].

## 1. State-Space Model and Core PSID Algorithm

In PSID, the underlying dynamics are represented by a forward stochastic state-space model:
\[
\begin{cases}
x_{k+1} = A x_k + w_k,\\
y_k = C_y x_k + v_k,\\
z_k = C_z x_k + \eta_k,
\end{cases}
\]
where \(y_k \in \R^{n_y}\) (primary), \(z_k \in \R^{n_z}\) (secondary), and \(x_k \in \R^{n_x}\) is the latent state. The process, observation, and secondary-signal noises \((w_k, v_k, \eta_k)\) are zero-mean Gaussian, with covariance structure:
\[
\E\left[
\begin{smallmatrix} w_k \\ v_k \end{smallmatrix}
\right]
\left[
\begin{smallmatrix} w_k \\ v_k \end{smallmatrix}
\right]^T =
\begin{pmatrix} Q & S \\ S^T & R \end{pmatrix}, \quad \E[\eta_k \eta_k^T]=R_z
\]
and \(A, C_y, C_z\) are system, primary observation, and secondary observation matrices, respectively.

The original PSID aims to estimate a model in the Kalman predictor ("innovation") form, determining (a) the predictor for the dynamics of \(y_k\) and (b) the low-dimensional shared subspace enabling optimal prediction of \(z_k\) from the history of \(y_k\). This is achieved via a two-stage subspace identification process:

- **Stage 1 (shared subspace extraction):** SVD of the projection of the future block Hankel matrix of \(z_k\) onto the row-space of the past block Hankel matrix of \(y_k\), selecting the top \(n_1\) modes and reconstructing the corresponding latent states.
- **Stage 2 (residual latent extraction):** Remove shared subspace influence from future blocks of primary data, perform SVD on reprojected components to extract the remaining \(n_x - n_1\) latent dimensions.
- **System parameter estimation:** Stack estimated latent trajectories, then estimate \(A, C_y, C_z, K\) (Kalman predictor gain) using ordinary least squares. Specifically,
\[
A \approx \hat X_{\rm shift} \hat X^\dagger, \quad 
C_y \approx Y \hat X^\dagger, \quad
C_z \approx Z \hat X^\dagger,\quad
K \approx (\hat X^+_{k+1|k} - A \hat X_{k|k-1})e^\dagger.
\]

## 2. Filter Gain Identification and Reduced-Rank Regression

While the predictor-form gain \(K\) is uniquely determined by the observed data, the Kalman filter update gain \(K_f\) generally is not when the state and observation noises are correlated (\(S \neq 0\)). With access to the secondary signal during training, PSID uniquely identifies \(K_f\) by directly minimizing the filtering mean-squared error (MSE) on \(z_k\):
\[
\arg\min_{K_f} \sum_{k=1}^N \|z_k - C_z x_{k|k}\|_2^2, \qquad x_{k|k} = x_{k|k-1} + K_f e_k.
\]
The problem is formulated as a reduced-rank regression (RRR):
\[
M = \arg\min_{M: \operatorname{rank} M \leq r} \|Z - C_z X^- - M E\|_F^2
\]
where \(E = [e_1, \ldots, e_N]\), \(Z = [z_1, \ldots, z_N]\), \(X^- = [x_{1|0}, ..., x_{N|N-1}]\). The optimal \(C_z K_f = M\) is computed, and under observability, \(K_f\) itself can be recovered if the concatenated observability matrix of \(z_k\) (\(\Gamma_z\)) has a left-inverse.

A key property is that access to the secondary signal resolves non-uniqueness, uniquely determining \(K_f\) within the equivalence class of observable stochastic-form models, as formalized by Faurré’s theorem. In the limit of large sample size, the estimated \(C_z K_f\) converges to the true value [2507.15288].

## 3. PSID with Filtering: Algorithmic Workflow

The PSID + filtering workflow involves the following steps:

1. **Learn innovation-form model:** Apply original PSID (SVD-based) to estimate \(A, C_y, C_z, K, \Sigma_e\).
2. **Compute innovations:** Calculate one-step-ahead innovations \(e_k = y_k - C_y x_{k|k-1}\) and predicted states.
3. **Form regression data matrices:** \(E = [e_1, ..., e_N]\), \(Z' = [z_1 - z_{1|0}, ..., z_N - z_{N|N-1}]\).
4. **Reduced-rank regression:** Solve \(M=\arg\min_{\operatorname{rank} M \leq n_x}\|Z' - M E\|_F^2\), yielding \(C_z K_f = M\).
5. **(Optional) Gain recovery:** If observability matrix \(\Gamma_z\) is invertible, recover \(K_f\) directly.
6. **Filter application:** For new data, apply
\[
x_{k|k} = x_{k|k-1} + K_f e_k, \quad
\hat z_{k|k} = C_z x_{k|k}.
\]

This extension yields state estimates and filtered predictions of the secondary signal with optimal MSE performance [2507.15288].

## 4. Forward–Backward PSID Smoothing

Building on classical two-filter Kalman smoothers, PSID is extended to a forward–backward smoothing algorithm targeting the secondary signal:

- **Forward pass:** Run PSID + filtering on the entire sequence \(\{y_{1:N}, z_{1:N}\}\), yielding filtered secondary estimates \(\hat z_{k|k}^f\) and residuals \(r_k = z_k - \hat z_{k|k}^f\).
- **Backward pass:** Reverse and process \(\tilde y_\ell = y_{N-\ell+1}\), \(\tilde z_\ell = r_{N-\ell+1}\) using PSID + filtering, producing backward-filtered residuals \(\tilde r_{\ell|\ell}\), then reverse time to obtain \(\hat z_{k|k}^b = \tilde r_{N-k+1|N-k+1}\).
- **Smoothing combination:** The final smoothed secondary estimate is given by
\[
\hat z_{k|N} = \hat z_{k|k}^f + \hat z_{k|k}^b
\]
which achieves the minimum MSE for the secondary signal over all available data, matching the theoretical optimum from a Rauch–Tung–Striebel (RTS) smoother [2507.15288].

## 5. Identifiability and Theoretical Guarantees

PSID’s identifiability properties differ from classical single-signal subspace identification:

- **Single-signal case (\(z_k \equiv y_k\)):** The filter gain \(K_f\) is non-identifiable; filtering yields the observation itself.
- **Two-signal case (\(y_k\) primary, \(z_k\) secondary):** The joint statistics of \((y_k, z_k)\) uniquely determine the optimal filter \(K_f\) minimizing the filtering MSE for \(z_k\). This secondary signal removes the stochastic-form equivariance present in the single-signal case.
- **Parameter identifiability:** The predictor-form parameters \(\{A, C_y, C_z, K, \Sigma_e\}\) are generically identifiable up to similarity transforms. The RRR step identifies the external characteristic \(C_z K_f\); under observability conditions, \(K_f\) is also identified.
- **Consistency:** As the number of samples \(N \to \infty\), least-squares and SVD estimates converge (in Frobenius norm) to ground truth system parameters; the RRR solution for \(C_z K_f\) converges as well [2507.15288].

## 6. Empirical Validation and Performance

PSID + filtering and smoothing methods were empirically validated on simulated state-space models, with random choices of model dimensions \((n_x, n_y, n_z)\) and training samples up to \(10^6\):

- **Parameter recovery:** Model parameters \(\{A, C_y, C_z, K, \Sigma_e, C_z K_f\}\) were recovered with normalized Frobenius error below 1% for identifiable parameters. As expected, non-identifiable \(K_f\) did not converge in the single-signal case.
- **Decomposition performance:** On held-out data, one-step-ahead prediction \(\hat z_{k|k-1}\), filtered estimate \(\hat z_{k|k}\), and smoothed estimate \(\hat z_{k|N}\) achieved coefficient of determination (\(R^2\)) values matching the performance of predictions from the true underlying model, including ideal filter and RTS smoother [2507.15288].

A plausible implication is that PSID + filtering provides a principled and practical means of identifying dynamic models and constructing optimal decoders for two-signal systems, particularly in the analysis of neural and behavioral data streams.

## 7. Significance and Applications

PSID and its extensions address the broader challenge of inferring dynamical latent representations from multivariate time series where joint prediction, filtering, and smoothing of secondary variables are essential—for example, neural decoding of behavior. The unique identifiability and empirical optimality guarantees position PSID as a rigorous tool for probing dynamic interactions and constructing evidence-based decoders in neuroengineering and related disciplines [2507.15288].

Source: https://www.emergentmind.com/topics/preferential-subspace-identification-psid