Magnetic order and anisotropic interactions induced by mixing between the $J=1/2$ and $3/2$ sectors in spin-orbit coupled honeycomb-lattice compounds

Abstract: Novel magnetic ordering on the honeycomb lattice due to emergent weak anisotropic interactions generated by the mixing between the $J=1/2$ sector and the magnetically inactive 3/2 sector is investigated in a three-orbital interacting electron model in the absence of Hund's coupling. Self-consistent determination of magnetic order yields anisotropic N\'{e}el and zigzag orders for different parameter regimes, highlighting the effect of the emergent single-ion anisotropy. Study of magnon excitations shows extremely small magnon energy scale compared to the hopping energy scale, and enhancement of anisotropy effects for smaller spin-orbit coupling. These results account for several features of the honeycomb lattice compounds such as $\rm Na_2 Ir O_3$ and $\rm Ru Cl_3$, where the leading order anisotropic interactions within the magnetically active $J=1/2$ sector are completely quenched due to the edge-sharing octahedra.