Feature Identifiability in Sparse Factor Analysis

• Feature identifiability is the ability to uniquely determine the factor loading matrix from observed data by leveraging sparsity patterns.
• The matching criterion facilitates a local, recursive check of sign-identifiability using polynomial-time algorithms, enhancing model validation.
• This approach improves the interpretability and reliability of latent factors in domains like psychology, genetics, and econometrics.

Feature identifiability concerns the capacity to uniquely determine structural characteristics of a factor analysis model—specifically, the factor loading matrix—from observed data. In sparse factor analysis, where the loading matrix exhibits patterns of zeros reflecting interpretability or scientific constraints, identifiability becomes central to both statistical inference and domain interpretation.

1. Identifiability in Sparse Factor Analysis

In the classical factor analysis model, the observed covariance matrix $\Sigma$ is modeled as

$\Sigma = \Lambda \Lambda^\top + \Omega,$

where $\Lambda$ is the $n \times k$ loading matrix and $\Omega$ is diagonal. Without further constraints, $\Lambda$ is only determined up to an orthogonal transformation: for any orthogonal $Q$, $\Lambda Q$ yields the same $\Sigma$. This rotational non-identifiability means the recovered factors are ambiguous and undermine the interpretability of learned features.

In sparse factor analysis, zeros are imposed on $\Lambda$ according to domain knowledge or inferred via penalization, effectively reducing the class of allowable transformations. In many common patterns, this improves identifiability to column signs: the nonzero structure of $\Lambda$ restricts the set of allowable orthogonal matrices $Q$ in $\Lambda Q$ to signed permutation matrices. Thus, $\Lambda$ is (generically) uniquely determined up to simultaneous column sign changes—a stronger and more useful guarantee for structure and interpretation.

The identifiability question in sparse factor models becomes: Given the imposed zero-pattern in $\Lambda$, under what graphical or algebraic conditions does the model admit this improved (sign) identifiability?

2. Structural Representation: Bipartite Graphs and Sparsity

Sparse factor models naturally correspond to bipartite graphs $G = (V \cup H, D)$, where $V$ denotes observed variables, $H$ latent factors, and $(h, v) \in D$ if the loading $\lambda_{vh}$ is not restricted to zero. This graph encodes the support of the loading matrix.

This representation forms the basis for graphical analysis of identifiability. The children of $h \in H$ are $ch(h)$, those $v \in V$ with nonzero loading; parents of $v$ are $pa(v)$. The sparsity pattern informs how much structural restriction is imposed on the loading matrix, and thereby on feature identifiability.

3. Existing and Novel Graphical Criteria

Initial approaches to identifiability in sparse models rely on global properties, such as AR-identifiability or the zero upper-triangular assumption (ZUTA), which require a particular structure for the loading matrix (e.g., upper-triangular zeros under some ordering), or sufficient rank conditions on the support.

While such criteria provide easy checks for some patterns, they are neither sharp nor local: many interpretable or learned structures in practice violate these global rules, leaving identifiability unresolved.

The matching criterion introduced in this work provides a finer, locally checkable, and more broadly sufficient graphical criterion for generic sign-identifiability. This criterion depends on finding, for each column of $\Lambda$, certain intersection-free matchings in the bipartite graph. Formally (simplified):

Given a latent node $h$, the existence of a specific configuration—a matching of observed nodes $W$ and $U$ through distinct factors not intersecting a solved set $S$—certifies that the corresponding loading parameters are uniquely determined up to column sign.

Mathematically, this is encoded via local determinants of principal submatrices of $\Sigma$, providing explicit formulas (see Theorem 4.3 and Section 4.1).

4. The Matching Criterion and Algorithmic Verification

The matching criterion operates recursively and locally: for each column, it checks for recoverability conditional on already identified structure. If, at each stage, the criterion is satisfied, all entries of $\Lambda$ can be uniquely recovered up to column sign.

A central advance is that, under bounded-size search (i.e., restricting attention to small subsets of nodes), the matching criterion can be checked in polynomial time in the model’s size. The key insight is reducing the search for intersection-free matchings to a maximum flow problem, allowing the use of efficient combinatorial algorithms.

This scalability marks a practical improvement over earlier algebraic methods, which suffer from double-exponential complexity, and over earlier graphical criteria, which were often conservative or failed to cover relevant real-world sparsity patterns.

5. Implications for Scientific and Statistical Practice

The matching criterion enables practitioners in psychology, genetics, econometrics, and other sciences to guarantee feature identifiability of sparse factor models before or after model fitting. Important consequences include:

• Interpretability: Ensures that estimated factors correspond to substantively meaningful latent traits, as the model structure uniquely picks out features (up to sign).
• Validity of Inference: Validates confidence interval construction, hypothesis testing, and model selection for the entries of $\Lambda$.
• Flexible Model Design: Allows confirmation of identifiability for highly structured or locally sparse models that fall outside the reach of earlier global criteria, including those produced by modern statistical learning procedures.

6. Comparison with Previous Approaches

Earlier results (AR-identifiability, ZUTA, BB criteria) imposed global constraints—e.g., full column sparsity or certain triangular structures—that, while easy to check, failed for many local or irregular patterns. The matching criterion covers all AR-identifiable models and many additional cases, strictly generalizing these older results.

Moreover, the recursive and local nature of the criterion aligns well with both theoretical analysis and algorithmic implementation, providing a foundation for automated identifiability checking in high-dimensional confirmatory and exploratory factor analysis, particularly as model sizes and complexities increase.

7. Summary Table: Identifiability Criteria in Sparse Factor Models

Criterion	Scope	Computational Complexity
Classical (AR, ZUTA, BB)	Global, rigid	Low
Matching Criterion (this work)	Local, flexible	Polynomial (for bounded set size)

The matching criterion thus marks a significant step forward in certifying generic feature identifiability in sparse factor models and offering practical guidance for users of factor analysis in scientific research.