Phase diagram and magnetic excitations of $J_1$-$J_3$ Heisenberg model on the square lattice

Abstract: We study the phase diagram and the dynamical spin structure factor of the spin-1/2 J1-J3 Heisenberg model on the square lattice using density matrix renormalization group, exact diagonalization (ED), and cluster perturbation theory (CPT). By extrapolating the order parameters and studying the level crossings of the low-lying energy and entanglement spectra, we obtain the phase diagram of this model and identify a narrow region of quantum spin liquid (QSL) phase followed by a plaquette valence-bond solid (PVBS) state in the intermediate region, whose nature has been controversial for many years. More importantly, we use CPT and ED to study the dynamical spin structure factor in the QSL and the PVBS phase. In the QSL phase, the high-energy magnon mode completely turns into some dispersive weak excitations around the X and M points. For the PVBS phase, the low-energy spectrum is characterized by a gapped triplet excitation, and at the high energy, we find another branch of dispersive excitation with broad continua, which is unlike the plaquette phase in the 2x2 checkerboard model. In the latter case, the second branch of excitation is nearly flat due to the weak effective interactions between the local excitations of the plaquettes. And in the J1-J3 Heisenberg model, the uniform interactions and the spontaneously translational symmetry breaking of the PVBS phase make the difference in the excitation spectra.