Global convergence analysis of mixtures of Exponential densities

Abstract: The theoretical foundations of the EM algorithm are often thought of in the context of Gaussian mixture models, However, the practical use cases of the EM algorithm span beyond Gaussian models. This paper establishes the first step towards understanding the behavior of the EM algorithm under mixtures of non-Gaussian densities. We show that a mixture of two Exponential distributions can be approximated by the EM algorithm at the sub-Exponential rate of convergence in at most $\log(n)$ iterations. The results here show that extending away from Gaussian mixture models does not affect the statistical performance of the EM algorithm. Furthermore, we present generalizations of typical assumptions in the Gaussian setting like minimum mean-separation and signal-to-noise ratio to the sub-Exponential setting. A simulation study is used to highlight the empirical performance of EM for mixtures of exponentials with promising results for the extension of existing theory to a larger class of mixture models.