Incommensurate order with translationally invariant projected entangled-pair states: Spiral states and quantum spin liquid on the anisotropic triangular lattice

Abstract: Simulating strongly correlated systems with incommensurate order poses significant challenges for traditional finite-size-based approaches. Confining such a phase to a finite-size geometry can induce spurious frustration, with spin spirals in frustrated magnets being a typical example. Here, we introduce an Ansatz based on infinite projected entangled-pair states (iPEPS) which overcomes these limitations and enables the direct search for the optimal spiral in the thermodynamic limit, with a computational cost that is independent of the spiral's wavelength. Leveraging this method, we simulate the Heisenberg model on the anisotropic triangular lattice, which interpolates between the square and isotropic triangular lattice limits. Besides accurately reproducing the magnetically ordered phases with arbitrary wavelength, the simulations reveal a quantum spin liquid phase emerging between the N\'eel and spin spiral phases.