Maximally Localized Wannier Orbitals, Interaction Models and Fractional Quantum Anomalous Hall Effect in Twisted Bilayer MoTe2

Abstract: We investigate the moir\'e band structures and the strong correlation effects in twisted bilayer MoTe$_2$ for a wide range of twist angles, employing a combination of various techniques. Using large-scale first principles calculations, we pinpoint realistic continuum modeling parameters, subsequently deriving the maximally localized Wannier functions for the top three moir\'e bands. Simplifying our model with reasonable assumptions, we obtain a minimal two-band model, encompassing Coulomb repulsion, correlated hopping, and spin exchange. Our minimal interaction models pave the way for further exploration of the rich many-body physics in twisted MoTe$_2$. Furthermore, we explore the phase diagrams of the system through Hartree-Fock approximation and exact diagonalization. Our two-band exact diagonalization analysis underscores significant band-mixing effects in this system, which enlarge the optimal twist angle for fractional quantum anomalous Hall states.