Interaction effects on almost flat surface bands in topological insulators

Abstract: We consider ferromagnetic instabilities of two-dimensional helical Dirac fermions hosted on the surface of three-dimensional topological insulators. We investigate ways to increase the role of interactions by means of modifying the bulk properties which in turn changes both the surface Dirac theory and the screening of interactions. We discuss both the long-ranged part of the Coulomb interactions controlled by the dimensionless coupling constant $\alpha = e^{2}/(\hbar \epsilon v_{F}^{\mathrm{surf}})$ as well as the effects of local interactions parametrized by the ratio $U_{\mathrm{surf}}/D_{\mathrm{surf}}$ of a local interaction on the surface, $U_{\mathrm{surf}}$, and the surface bandwidth, $D_{\mathrm{surf}}$. If large compared to 1, both mechanisms can induce spontaneously surface ferromagnetism, thereby gapping the surface Dirac metal and inducing an anomalous quantum Hall effect. We investigate two mechanisms which can naturally lead to small Fermi velocities $v_{F}^{\mathrm{surf}}$ and a corresponding small bandwidth $D_{\mathrm{surf}}$ at the surface when the bulk band gap is reduced. The same mechanisms can, however, also lead to an enhanced screening of surface interactions. While in all considered cases the long-ranged part of the Coulomb interaction is screened efficiently, $\alpha \lesssim 1$, we discuss situations, where $U_{\mathrm{surf}}/D_{\mathrm{surf}}$ becomes \emph{parametrically}\ large compared to 1, thus inducing surface magnetism.