Identifiable and interpretable nonparametric factor analysis

Abstract: Factor models are widely used to reduce dimensionality in modeling high-dimensional data. However, there remains a need for models that can be reliably fit in modest sample sizes and are identifiable, interpretable, and flexible. To address this gap, we propose a NIFTY model that uses a linear factor structure with Gaussian residuals, but with a novel latent variable modeling structure. In particular, we model each latent variable as a one-dimensional nonlinear mapping of a uniform latent location. A key innovation is allowing different latent variables to be transformations of the same latent locations, accommodating intrinsic lower-dimensional nonlinear structures. Leveraging on pre-trained data obtained by diffusion maps and post-processing of MCMC samples, we obtain model identifiability. In addition, we softly constrain the empirical distribution of the latent locations to be close to uniform to address a latent posterior shift problem, which is common in factor models and can lead to substantial bias in parameter inferences, predictions, and generative modeling. We show good performance in density estimation and data visualization in simulations, and apply NIFTY to bird song data in an environmental monitoring application.