Overfitting rank consistency

Prove that, under the assumptions of the underfitting consistency theorem—including fixed idiosyncratic variance [?] no, avoid control. Need ensure statement: Establish that posterior probability of effective ranks greater than q0 vanishes for DP model under assumptions.

Background

The paper proves only the underfitting direction: with the idiosyncratic variance fixed at its true value, posterior mass on effective ranks below the true rank q0 vanishes. The complementary claim that ranks above q0 are ruled out is not proved and is explicitly formulated as a conjecture.

The authors explain that the usual Bayes-factor route fails because overfitted loading configurations can reproduce the true covariance exactly or arbitrarily closely. In particular, a free diagonal idiosyncratic variance can absorb an additional rank-one loading contribution, while even with the idiosyncratic variance fixed, an arbitrarily small loading in a direction outside the true loading span yields an overfitted model with covariance arbitrarily close to the truth.

References

The complementary direction --- that the posterior does not concentrate on ranks exceeding $q_0$ --- is substantially harder. We state it as a conjecture.

— Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity  (2609.34546 - Bhattacharya et al., 28 Sep 2026) in Conjecture 1 (Conjecture~\ref{conj:overfit}), Section 8.3, “Overfitting: A Conjecture”

Whether such constructions can be excluded by the shell alone, or whether an additional geometric constraint on the columns of $F$ (for example, a lower bound on the angle between any two distinct columns) is necessary, is a question that we have not been able to settle. We therefore present the shell as a natural first step towards restoring KL separation, whose sufficiency for Conjecture~\ref{conj:overfit} is left as an open problem.

— Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity  (2609.34546 - Bhattacharya et al., 28 Sep 2026) in Section 8.4, paragraph “Attempts to restore KL separation via a shell prior”

Making this argument rigorous requires a careful analysis of the local geometry of the factor manifold near the nested point and a verification that the Laplace expansion underlying the Savage--Dickey formula is valid under the spike-and-slab prior. Both are substantial technical undertakings that we leave as an open problem.

— Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity  (2609.34546 - Bhattacharya et al., 28 Sep 2026) in Section 8.4, paragraph “Alternative: Savage--Dickey analysis”

A rigorous quantification of this observation remains an open problem and is, in our view, one of the more interesting theoretical questions raised by the present work.

— Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity  (2609.34546 - Bhattacharya et al., 28 Sep 2026) in Section 8.5, “Discussion,” final paragraph