Renormalization group analysis of competing quantum phases in the J1-J2 Heisenberg model on the kagome lattice

Abstract: Recent discoveries in neutron scattering experiments for Kapellasite and Herbertsmithite as well as theoretical calculations of possible spin liquid phases have revived interest in magnetic phenomena on the kagome lattice. We study the quantum phase diagram of the S=1/2 Heisenberg kagome model as a function of nearest neighbor coupling J1 and second neighbor coupling J2. Employing the pseudofermion functional renormalization group, we find four types of magnetic quantum order (q=0 order, cuboc order, ferromagnetic order, and Sqrt{3}x\Sqrt{3} order) as well as extended magnetically disordered regions by which we specify the possible parameter regime for Kapellasite. In the disordered regime J2/J1<<1, the flatness of the magnetic susceptibility at the zone boundary which is observed for Herbertsmithite can be reconciled with the presence of small J2>0 coupling. In particular, we analyze the dimer susceptibilities related to different valence bond crystal (VBC) patterns, which are strongly inhomogeneous indicating the rejection of VBC order in the RG flow.