Entangled spin-orbital phases in the bilayer Kugel-Khomskii model

Abstract: We derive the Kugel-Khomskii spin-orbital (SO) model for a bilayer and investigate its phase diagram depending on Hund's exchange $J_H$ and the $e_g$ orbital splitting $E_z$. In the (classical) mean-field approach with on-site spin $<S_i^z>$ and orbital $<\tau_i^z>$ order parameters and factorized spin-and-orbital degrees of freedom, we demonstrate a competition between the phases with either $G$-type or $A$-type antiferromagnetic (AF) or ferromagnetic long-range order. Next we develop a Bethe-Peierls-Weiss method with a Lanczos exact diagonalization of a cube coupled to its neighbors in $ab$ planes by the mean-field terms --- this approach captures quantum fluctuations on the bonds which decide about the nature of disordered phases in the highly frustrated regime near the orbital degeneracy. We show that the long-range spin order is unstable in a large part of the phase diagram which contains then six phases, including also the valence-bond phase with interlayer spin singlets stabilized by holes in $3z^2-r^2$ orbitals (VB$z$ phase), a disordered plaquette valence-bond (PVB) phase and a crossover phase between the VB$z$ and the $A$-type AF phase. When on-site SO coupling is also included by the $<S_i^z\tau_i^z>$ order parameter, we discover in addition two entangled phases which compete with $A$-type AF phase and another crossover phase in between the $G$-AF phase with occupied $x^2-y^2$ orbitals and the PVB phase. Thus, the present bilayer model provides an example of SO entanglement which generates novel disordered phases. We analyze the order parameters in all phases and identify situations where SO entanglement is crucial and factorization of the spins and orbitals leads to qualitatively incorrect results. We point out that SO entanglement may play a role in a bilayer fluoride K$_3$Cu$_2$F$_7$ which is a realization of the VB$z$ phase.