Eigenspace-Based Clustering for Personalized System Identification

Abstract: We study the problem of system identification in heterogeneous settings, where different systems may follow distinct underlying dynamics. Existing clustered system identification approaches often rely on iterative training-based cluster assignment, which can be sensitive to learning uncertainty and model initialization. In contrast, we propose a one-shot, training-free clustering method that identifies similar systems using the structure of their locally observed data. Specifically, each system estimates a local state covariance matrix, and cluster identities are inferred by measuring the alignment between the leading covariance eigenspaces of different systems. We provide a mathematical interpretation of the proposed similarity score and develop a finite-sample analysis that characterizes how covariance estimation error induces eigenspace perturbations in terms of the underlying system dynamics. We then derive a probability bound for pairwise false merges and a global clustering success guarantee. Numerical experiments demonstrate that the proposed eigenspace-based clustering method effectively identifies systems with shared dynamics, leading to lower personalized model-estimation error compared with training-based clustering and non-clustered baselines.

• The paper introduces a one-shot, training-free eigenspace-based clustering mechanism that uses local state covariance matrices to pre-assign clusters in heterogeneous LTI systems.
• The method decouples clustering from model training, reducing communication overhead and sensitivity to initialization while maintaining privacy.
• Finite-sample theoretical bounds and empirical validation demonstrate improved identification accuracy, reduced misclassification rates, and scalable performance.

Eigenspace-Based Clustering for Personalized System Identification

Problem Formulation and Motivation

The paper addresses system identification in distributed heterogeneous environments, particularly for linear time-invariant (LTI) systems whose local dynamics differ across participants. Traditional clustered identification strategies in personalized federated learning (PFL) often employ iterative cluster assignment intertwined with model training, resulting in sensitivity to initialization, high communication costs, and vulnerability to learning instability. The paper proposes an alternative: a one-shot, training-free eigenspace-based clustering mechanism that leverages the structure of local state covariance matrices to infer system similarity and subsequently assign clusters prior to collaborative model estimation.

The motivation stems from the limitations of prevailing approaches, e.g., alternating minimization techniques [toso2023learning] and regularized incremental clustering [kecceci2024novel], both of which depend on intermediate model estimates and require careful initialization. The proposed methodology breaks the coupling between clustering and model training, enabling efficient cluster identity determination solely from data statistics.

Eigenspace-Based Clustering Algorithm

Each participating system computes the empirical state covariance matrix from terminal states across multiple rollouts, performs local eigendecomposition, and extracts the leading $r$ eigenvectors and eigenvalues. Cluster identity estimation proceeds in three main steps:

1. Local eigenspace extraction: Systems compute the leading eigenspace of their state covariance, encapsulating principal directions of state variation.
2. Eigenspace alignment scoring: Systems exchange leading eigenvectors and compute alignment scores via the normed response of their local covariance to received eigenvectors, normalized to produce a symmetric pairwise similarity matrix. The geometric mean across $r$ directions yields a robust scalar score emphasizing consistent alignment.
3. Spectral clustering: The server applies spectral clustering to the symmetric affinity matrix, inferring cluster assignments without dependence on training or initialization.

This procedure incurs minimal communication overhead—systems share only principal eigenvectors rather than raw data—thus ensuring privacy and scalability. The affinity scores are mathematically interpreted: a similarity score of 1 between two systems implies exact alignment of their dominant eigenspace directions, formalized through variational spectral theory.

Theoretical Analysis

The paper develops rigorous finite-sample analyses quantifying the propagation of trajectory-level stochasticity into covariance estimation error, eigenspace perturbation, and clustering reliability:

• Covariance concentration: Non-asymptotic spectral norm bounds for Gaussian-distributed trajectory data are given, linking estimation error to the number of rollouts, state dimension, system stability, and dynamics.
• Eigenspace perturbation: Davis–Kahan-type bounds translate covariance estimation error into leading eigenspace misalignment, with explicit dependence on the eigengap and sample complexity.
• Cluster error probability: The probability of false merges (systems from different clusters being incorrectly clustered) is bounded as a function of inter-cluster eigenspace separation, eigengap, and trajectory count. The global clustering success probability is shown to grow rapidly with increased sample count, separation margin, and eigengap.

Critically, for stable systems, the sample complexity saturates with time horizon, while unstable dynamics entail exponential growth in trajectory requirements—highlighting system-specific trade-offs in identification reliability.

Empirical Validation

Extensive experiments on 50 heterogeneous LTI systems, grouped intrinsically into three clusters, are conducted. The methodology is compared against alternating minimization and non-clustered global baselines. Results demonstrate:

• Strong numerical performance: Eigenspace clustering achieves consistently lower cluster-wise identification error than both baselines. In contrast, alternating minimization collapses into a single cluster under random initialization, matching global model error and failing to personalize, confirming theoretical sensitivity predictions.
• Collaboration benefits: Collaborative learning within eigenspace-identified clusters outperforms individual model estimation, exploiting data-sharing to boost parameter accuracy.
• Sample complexity scaling: The clustering misclassification rate decreases monotonically with increasing trajectory count, aligning with finite-sample theory and validating the predicted trade-offs.

Implications and Future Directions

Practically, the method supports efficient, privacy-conscious personalization in federated control, robotics, and learning-based decision-making centers. It enables decoupled clustering and model identification, circumventing common initialization and training instability traps encountered in iterative methods. Theoretically, the paper establishes precise sample complexity laws for distributed identification under statistical heterogeneity, highlighting the importance of eigengap and inter-cluster separation. The approach is robust to the number of clusters, number of participants, and system-dependent dynamics.

Future research will extend the framework to partially observed systems, incorporate privacy-preserving variants for eigenspace sharing, and explore hybrid clustering-initialization schemes. Further theoretical developments may analyze the impact of adversarial data, non-Gaussian perturbations, and longitudinal trajectory evolution.

Conclusion

The paper introduces a one-shot, eigenspace-based clustering algorithm for personalized federated system identification, underpinned by rigorous finite-sample theory and empirical validation. The strategy improves identification accuracy, reduces misclassification rates, and mitigates the drawbacks of iterative clustering approaches, with practical implications for scalable, robust, and personalized distributed learning across heterogeneous dynamical systems (2606.20811).