Quantum disordered ground state in the spin-orbit coupled Jeff = 1/2 distorted honeycomb magnet BiYbGeO5

Abstract: We delineate quantum magnetism in the strongly spin-orbit coupled, distorted honeycomb-lattice antiferromagnet BiYbGeO${5}$. Our magnetization and heat capacity measurements reveal that its low-temperature behavior is well described by an effective $J{\rm eff}=1/2$ Kramers doublet of Yb$^{3+}$. The ground state is nonmagnetic with a tiny spin gap. Temperature-dependent magnetic susceptibility, magnetization isotherm, and heat capacity could be modeled well assuming isolated spin dimers with anisotropic exchange interactions $J_{\rm Z} \simeq 2.6$~K and $J_{\rm XY} \simeq 1.3$~K. Heat capacity measurements backed by muon spin relaxation suggest the absence of magnetic long-range order down to at least 80\,mK both in zero field and in applied fields. This sets BiYbGeO$_5$ apart from Yb$_2$Si$_2$O$_7$ with its unusual regime of magnon Bose-Einstein condensation and suggests negligible interdimer couplings, despite only a weak structural deformation of the honeycomb lattice.