Hierarchical calibration of the slab variance

Develop and analyze a fully hierarchical treatment of the global slab variance $\tau_0^2$ that controls its substantial effect on the estimated number of factors and improves variance calibration without undermining the model’s sparsity and column-clustering properties.

Background

The sensitivity analysis shows that the posterior mode of the number of factors changes from 8 to 13 to 25 as the slab variance decreases from 1.0 to 0.3 to 0.1. This indicates that rank estimation is materially dependent on a user-specified global scale.

The authors also find that the global slab overestimates marginal variances on standardized data and suggest hierarchical treatment of the slab scale as a way to improve reliability. They explicitly identify this as an open problem because it directly affects the method’s principal goal of rank estimation.

References

We regard this last point as the most important open problem, because it speaks directly to the reliability of the rank estimate that the method is designed to produce.

— Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity  (2609.34546 - Bhattacharya et al., 28 Sep 2026) in Section 9, “Future work,” final paragraph