Ground state phase diagram of Kitaev-Heisenberg model on honeycomb-triangular lattice

Abstract: The Kitaev-Heisenberg model defined on both honeycomb and triangular lattices has been studied intensively in recent years as a possible model to describe spin-orbital physics in iridium oxides. In the model, there are many phases characteristic for each lattice. However, there is no study how the phases in the two lattices merge each other when geometry changes from honeycomb lattice to triangular lattice. We investigate the ground state of the Kitaev-Heisenberg model defined on the system connecting the honeycomb and triangular lattices, named a honeycomb-triangular lattice. We obtain a ground state phase diagram of this model with classical spins by using the Luttinger-Tisza method and classical Monte Carlo simulation. In addition to known phases in the honeycomb and triangular lattices, we find coexisting phases consisting of the known phases. Based on the exact diagonalization and density-matrix renormalization group calculations for the model with quantum spins, we find that quantum fluctuations less affect on the phase diagram.