Extraordinary transition at the edge of a correlated topological insulator

Abstract: The interplay of topology and correlations defines a new playground to study boundary criticality in quantum systems. We employ large scale auxiliary field quantum Monte Carlo simulations to study a two-dimensional Kane-Mele-Hubbard model on the honeycomb lattice with zig-zag edges and the Hubbard U-term tuned to the three-dimensional XY bulk critical point. Upon varying the Hubbard-U term on the edge we observe a boundary phase transition from an ordinary phase with a helical Luttinger liquid edge decoupled from the critical bulk to an extraordinary-log phase characterized by a logarithmically diverging spin stiffness. We find that the spectral functions exhibit distinct features in the two phases giving potential experimental signatures.