Quantum Phase Transition in Heisenberg-Kitaev Model

Abstract: We explore the nature of the quantum phase transition between a magnetically ordered state with collinear spin pattern and a gapless $Z_2$ spin liquid in the Heisenberg-Kitaev model. We construct a slave particle mean field theory for the Heisenberg-Kitaev model in terms of complex fermionic spinons. It is shown that this theory, formulated in the appropriate basis, is capable of describing the Kitaev spin liquid as well as the transition between the gapless $Z_2$ spin liquid and the so-called stripy antiferromagnet. In particular, within a mean field theory, we have a discontinuous transition from the $Z_2$ spin liquid to the stripy antiferromagnet. We argue, however, that subtle spinon confinement effects, associated with the instability of gapped U(1) spin liquid in two spatial dimensions, are playing an important role at the transition. The possibility of an exotic continuous transition is briefly addressed.