---
title: Invariant Coordinate Selection (ICS)
url: https://www.emergentmind.com/topics/invariant-coordinate-selection-ics
type: topic
---

# Invariant Coordinate Selection (ICS)

Searching arXiv for recent and foundational papers on Invariant Coordinate Selection.
arxiv_search(query="Invariant Coordinate Selection", max_results=10, sort_by="submittedDate")
Invariant Coordinate Selection (ICS) is a multivariate exploratory, feature-extraction, and dimension-reduction method based on the joint diagonalization of two affine equivariant scatter matrices. Rather than relying solely on variance, ICS seeks directions in which two notions of scatter disagree most strongly, so that departures from elliptical normal structure—especially clustering, outlyingness, heavy tails, or other non-Gaussian geometry—become concentrated in a small number of invariant coordinates [2606.24521][1501.07240]. In recent work, ICS appears both as a general unsupervised transformation and as a task-specific preprocessing method for clustering, outlier detection, independent component analysis, anonymization, and complex-data analysis; its theoretical justification is closely tied to affine invariance, generalized kurtosis, and, in mixture models, the Fisher discriminant subspace [2409.17631][1612.06118].

## 1. Definition, scope, and relation to neighboring methods

ICS compares two scatter matrices instead of diagonalizing only one. In the standard formulation, these scatters are affine equivariant and positive definite, and the method constructs a coordinate system in which one scatter becomes the identity and the other becomes diagonal [2606.24521][2212.06108]. This already distinguishes ICS from principal component analysis (PCA): PCA “relies solely on variance,” whereas ICS is built to reveal structure that is not primarily variance-driven, including clustering and anomaly structure [2606.24521].

The basic intuition is that, for elliptical distributions, affine equivariant scatter matrices are proportional at the population level, so their comparison adds little information; when the data are not elliptical, different scatter matrices respond differently to tails, mixtures, contamination, or local structure, and ICS exploits exactly this discrepancy [2207.02131][2409.17631]. Several papers therefore place ICS in a broader family of methods for non-Gaussian structure discovery, including outlier detection, cluster identification, independent component analysis (ICA), and non-Gaussian component analysis (NGCA) [2207.02131][2502.04046].

A recurring comparison in the literature is with projection pursuit (PP). PP is a one-dimensional version of ICS: ICS works with multivariate scatter matrices and an eigen-decomposition, whereas PP searches numerically over one-dimensional projections and evaluates a ratio of univariate spread measures [1501.07240]. Later work uses ICS precisely as a structured initializer for localized PP refinement, rather than as a competing framework [2112.11998].

The method has also been generalized beyond ordinary multivariate vectors. A coordinate-free definition on finite-dimensional Euclidean spaces extends ICS to functional and distributional data after preprocessing into a finite-dimensional subspace, so that the same simultaneous-diagonalization principle can be applied to complex data objects [2505.19403].

## 2. Mathematical formulation and invariance properties

A standard ICS formulation seeks a matrix \(B\) such that
\[
B S_1 B^\top = I_p, \qquad B S_2 B^\top = D,
\]
where \(S_1\) and \(S_2\) are scatter matrices and \(D\) is diagonal [2606.24521][2212.06108]. Equivalently, if \(v_j\) denotes a generalized eigenvector and \(\lambda_j\) the associated generalized eigenvalue, ICS solves
\[
S_2 v_j = \lambda_j S_1 v_j,
\]
or, when \(S_1\) is invertible,
\[
S_1^{-1}S_2 v_j = \lambda_j v_j.
\]
The transformed variables are the invariant coordinates,
\[
z = Bx, \qquad Z = X B^\top,
\]
up to orientation conventions [2606.24521].

The literature represented here uses two equivalent ratio viewpoints. One writes ICS through the generalized eigenproblem above and interprets \(\lambda_j\) as generalized kurtosis values [2606.24521][2212.06108]. Another writes a Rayleigh quotient such as
\[
\kappa_{\mathrm{ICS}}(a)=\frac{a^\top S_1 a}{a^\top S_2 a},
\]
with extrema attained at generalized eigenvectors [1501.07240]. The difference is a matter of scatter ordering and convention. Across papers, the practical rule is stable: directions with **extreme** generalized eigenvalues—largest, smallest, or both, depending on the pair and the application—are the informative ones [1501.07240][2212.06108].

Affine invariance is one of the defining properties of ICS. If the data are transformed by an affine map \(x \mapsto Ax+b\), the scatter matrices transform equivariantly, and the invariant coordinates are unchanged up to sign or equivalent basis choices [1612.06118][2606.24521]. This is stronger than PCA’s orthogonal invariance and explains why ICS is repeatedly advocated when scale choices or linear reparameterizations should not change the relevant subspace [2606.24521].

For outlier detection, the geometry of the ICS transformation is especially explicit. If all invariant components are retained, Euclidean distance in ICS space is exactly Mahalanobis distance in the original space relative to the first scatter:
\[
z_i^\top z_i = (x_i-m)^\top V_1^{-1}(x_i-m).
\]
This gives ICS a whitening-plus-rotation interpretation: selecting only a subset of invariant coordinates yields a reduced-space analogue of Mahalanobis distance that is not swamped by irrelevant dimensions [1612.06118].

## 3. Scatter matrices, robustness contrasts, and the role of location

ICS is not tied to a single pair of scatter matrices. Recent work emphasizes a broad family of scatters, including classical covariance, fourth-moment scatter, winsorized covariance, principal-axis-type scatters, pairwise scatters, \(t\)-based \(M\)-estimators, minimum covariance determinant (MCD), and minimum volume ellipsoid (MVE) variants [2606.24521][1501.07240]. Their practical role is application-dependent: some behave as global scatters, some as local or within-cluster scatters, and some as robust core-structure estimators [2212.06108].

For clustering, the decisive practical principle is to pair one scatter that captures **within-cluster/local structure** with another that captures **global structure**. This is why local shape or pairwise scatters are repeatedly recommended, and why MCD with a carefully chosen subset size smaller than usual can be effective: the goal is not robustness alone, but approximation of a within-cluster covariance [2212.06108][2606.24521]. For outlier detection under a small contamination rate, by contrast, the pair \(\mathrm{COV}-\mathrm{COV}_4\) is recommended because it combines simplicity, analytical tractability, and good empirical performance [1612.06118].

A central refinement of ICS methodology concerns **location**. Scatter matrices carry an implicit or explicit location measure, and ICS can behave counter-intuitively when the two scatters are not centered at the same point [1501.07240]. The paper on common location measures shows that this is not a minor implementation detail. In a two-cluster mixture, a highly robust scatter such as MVE or MCD may “home in” on the larger cluster, while a non-robust scatter such as covariance remains centered at the overall mean. Then the generalized eigen-analysis no longer compares two estimates of the same central structure.

The resulting pathology can be severe. In a balanced bivariate normal mixture standardized to covariance \(I_2\), unconstrained \(\mathrm{ICS}:\mathrm{var}:\mathrm{mve}\) selects the direction \((0,1)^\top\), even though the true clustering direction is \((1,0)^\top\); with a common mean imposed as location, the method recovers the correct axis [1501.07240]. The practical recommendation is correspondingly direct: use the same location measure for both scatter matrices, often the sample mean. The same issue affects PP, since PP is the one-dimensional analogue of ICS [1501.07240].

This discussion also clarifies a common misconception: “robust versus non-robust” is not the only relevant contrast. Location compatibility is essential. A robust scatter pair can be misleading if the two scatters are centered differently, while a non-robust pair can be theoretically and practically useful when the contamination regime is small and the application matches the underlying model [1501.07240][1612.06118].

## 4. Clustering, outlier detection, discriminant structure, and ICA

The strongest theoretical link between ICS and clustering is its relation to the Fisher discriminant subspace (FDS). For mixtures of elliptical distributions with common within-group scatter and group centers spanning a \(q\)-dimensional affine subspace, the ICS matrix \(V_1^{-1}V_2\) has a repeated “noise” eigenvalue of multiplicity at least \(p-q\); if that multiplicity is exactly \(p-q\), the remaining eigenvectors span the FDS [2409.17631]. Recent work extends this perspective beyond the two-group case and concludes that ICS is suitable for recovering the FDS under very general settings, with failures appearing rare [2409.17631].

This theoretical picture motivates tandem clustering with ICS. In that setting, ICS replaces PCA as the preliminary dimension-reduction step before a clustering algorithm. The empirical conclusion is unambiguous: tandem clustering with ICS “clearly outperforms the PCA-based approach,” particularly when scatter pairs contrast local/within-cluster and global structure [2212.06108]. The best-performing pairs are \(\tcov-\cov\) and \(\lcov-\cov\), with \(\tcov-\ucov\) as a robust alternative; informative coordinates may be among the first, the last, or both ends of the eigenvalue spectrum [2212.06108].

For multivariate outlier detection, ICS is positioned as an affine-invariant alternative to ordinary Mahalanobis-distance-based procedures and to PCA. The key point is that outliers often lie in a low-dimensional subspace. Mahalanobis distance uses all dimensions at once, so irrelevant directions inflate variance and reduce discrimination as dimension grows; ICS alleviates this by selecting only the invariant coordinates carrying the outlying structure [1612.06118]. The proposed outlier-detection workflow uses three steps: compute invariant coordinates, select relevant components, then apply a distance rule in the selected ICS subspace. In that literature, \(\mathrm{COV}-\mathrm{COV}_4\) is the recommended pair, with parallel analysis and D’Agostino’s skewness test as practical component-selection tools [1612.06118].

ICS also has an ICA interpretation. When both scatter matrices have the independence property, the invariant coordinates coincide with independent components up to sign, so ICA becomes a special case of ICS [2502.04046]. This observation motivates sparse and robust extensions: Sparse Invariant Coordinate Selection (SICS) rewrites ICS as a least-squares problem and adds an \(\ell_1\)-penalty to promote sparse unmixing vectors, while robustness is controlled through the choice of robust scatter matrices [2502.04046]. The paper proves a consistency result for the single-component case under vanishing penalization and separated generalized eigenvalues [2502.04046].

The same transformation has been adapted to privacy-preserving data release. In ICSA, the PCA step of spectral anonymization is replaced by ICS, and robustness is tuned through the location and scatter choices. The paper proves that spectral anonymization fails under a sufficiently strong outlier and reports that robust ICS-based anonymization gives stronger privacy protection while often maintaining comparable utility [2605.04833].

## 5. Computation, numerical stability, and software implementations

Classical ICS implementations can be numerically fragile because they directly compute scatter matrices, inverse square roots, and eigen-decompositions. This becomes problematic when data are ill-conditioned, nearly rank deficient, or exactly singular [2207.02131][2409.02258]. The numerical literature therefore treats computation itself as part of ICS methodology, not merely as an implementation detail.

For the class of scatter pairs \(\cov\)–\(\cov_w\), a QR-based implementation avoids direct formation of \(\cov^{-1}\) and \(\cov^{-1/2}\). The method starts from a pivoted QR factorization of the centered data matrix, computes Mahalanobis distances from leverage scores,
\[
D^2(x_i)=(n-1)\|Q(i,:)\|_2^2,
\]
and obtains the ICS eigenstructure from a small matrix built from \(Q\) rather than from explicit covariance inversion [2207.02131]. This stabilizes the computation in ill-conditioned settings and, with rank-revealing pivoting, allows ICS to be computed even when the scatter matrices are not full rank after an estimated-rank reduction [2207.02131].

A separate line of work generalizes ICS to positive semi-definite scatter matrices. Three approaches are examined: a Moore–Penrose pseudo-inverse implementation, a preliminary dimension reduction, and a generalized singular value decomposition (GSVD) [2409.02258]. The pseudo-inverse approach changes the optimization problem and preserves only orthogonal invariance; the dimension-reduction approach depends critically on rank estimation and can discard informative low-variance directions. The GSVD-based method is judged the most promising because it preserves the symmetry of the generalized eigenproblem and retains affine invariance of the invariant coordinates, although it restricts the admissible scatter matrices to those expressible as cross-products [2409.02258].

Software support has expanded markedly. ICSpyLab is presented as the first Python package implementing ICS; it provides eight scatter matrices,
\[
\texttt{cov},\ \texttt{covW},\ \texttt{cov4},\ \texttt{covAxis},\ \texttt{mcd},\ \texttt{tcov},\ \texttt{tcovAxis},\ \texttt{tM},
\]
four algorithms (`eigh`, `standard`, `whiten`, `QR`), and three component-selection criteria (median, normal, unimodality), all within a scikit-learn-style estimator interface [2606.24521]. For complex data, the coordinate-free extension is implemented in the R package **ICSFun** [2505.19403]. Earlier applied work also relies on the R ecosystem around `ICS`, `ICSOutlier`, `ICSNP`, `rrcov`, `robustbase`, and related packages [1612.06118][2605.04833].

These developments sharpen another practical point: there is no single “ICS algorithm.” The mathematical object is fixed by the scatter pair, but numerical route, rank handling, sign conventions, and component-selection rules all matter materially in applications [2207.02131][2606.24521].

## 6. Extensions, refinements, and current directions

A prominent refinement treats ICS as a strong initializer rather than a terminal answer. Local projection pursuit begins from a promising ICS projection and performs gradient descent on estimated differential entropy over the projection manifold [2112.11998]. The motivation is geometric: ICS often finds projections that are close to genuinely interesting ones, but small angular errors can blur clusters or other latent structure. Local PP preserves the value of ICS while refining the projection toward lower estimated entropy [2112.11998].

Another major extension moves ICS beyond ordinary multivariate vectors. A coordinate-free formulation on finite-dimensional Euclidean spaces allows ICS to be transported under isometries and then applied to functional and density-valued data after finite-dimensional preprocessing [2505.19403]. For distributional data, the method is developed in Bayes Hilbert spaces. Densities are mapped through the centered log-ratio transform, smoothed into compositional spline functions by Maximum Penalised Likelihood, and then analyzed by ICS; the resulting procedure is used for outlier detection in yearly distributions of daily maximum temperatures across provinces of Northern Vietnam [2505.19403].

The projector-averaging literature places ICS inside a broader ensemble view of dimension reduction. In that framework, selected ICS directions define an orthogonal projector, and this projector can be averaged with projectors from PCA, SIR, SAVE, PHD, or IRE through weighted distances between subspaces of possibly different dimensions [1210.2575]. The role of ICS there is not methodological replacement but complementary subspace estimation.

Across these developments, two challenges remain central. The first is **scatter-pair choice**: effective use of ICS depends strongly on whether the pair captures the structural contrast relevant to the task [2212.06108][2606.24521]. The second is **component selection**: informative coordinates may occur among the first, the last, or both tails of the generalized eigenvalue spectrum, and current rules—median, normal, unimodality, parallel analysis, skewness tests, scree plots—remain context-dependent [1612.06118][2606.24521]. Recent software and theory make these choices more explicit, but they do not remove them.

Taken together, the contemporary literature presents ICS as a family of affine-invariant relative-eigenanalysis methods centered on a simple principle: meaningful multivariate structure is often revealed not by absolute variance, but by disagreement between carefully chosen scatter operators. The strength of the method lies in that contrast; so do its main practical demands.

Source: https://www.emergentmind.com/topics/invariant-coordinate-selection-ics