Emergence of a nematic paramagnet via quantum order-by-disorder and pseudo-Goldstone modes in Kitaev magnets

Abstract: The appearance of nontrivial phases in Kitaev materials exposed to an external magnetic field has recently been a subject of intensive studies. Here, we elucidate the relation between the field-induced ground states of the classical and quantum spin models proposed for such materials, by using the infinite density matrix renormalization group (iDMRG) and the linear spin wave theory (LSWT). We consider the $K \Gamma \Gamma'$ model, where $\Gamma$ and $\Gamma'$ are off-diagonal spin exchanges on top of the dominant Kitaev interaction $K$. Focusing on the magnetic field along the $[111]$ direction, we explain the origin of the nematic paramagnet, which breaks the lattice-rotational symmetry and exists in an extended window of magnetic field, in the quantum model. This phenomenon can be understood as the effect of quantum order-by-disorder in the frustrated ferromagnet with a continuous manifold of degenerate ground states discovered in the corresponding classical model. We compute the dynamical spin structure factors using a matrix operator based time evolution and compare them with the predictions from LSWT. We, thus, provide predictions for future inelastic neutron scattering experiments on Kitaev materials in an external magnetic field along the $[111]$ direction. In particular, the nematic paramagnet exhibits a characteristic pseudo-Goldstone mode which results from the lifting of a continuous degeneracy via quantum fluctuations.