Two-dimensional Weyl nodal-line semimetal and antihelical edge states in a modified Kane-Mele model

Abstract: The Kane-Mele model has been modified to achieve versatile topological phases. Previous work [Phys. Rev. Lett. 120, 156402 (2018)] introduced a staggered intrinsic spin-orbit coupling effect to generate pseudohelical edge states, with Rashba spin-orbit coupling facilitating spin flips in alternating sublattices. Our study demonstrates that, in the absence of Rashba spin-orbit coupling, the modified Kane-Mele model with staggered intrinsic spin-orbit coupling evolves into a $Z_{2}$ class topological metal, specifically a two-dimensional Weyl nodal-line semimetal. In a nanoribbon geometry, we predict the emergence of antihelical edge states, which support spin-polarized currents flowing in the same direction along parallel boundaries. Unlike pseudohelical edge states, antihelical edge states can be viewed as a superposition of two antichiral edge states related by time-reversal symmetry. However, the spin Hall conductance from antihelical edge states is not quantized due to the presence of gapless bulk states. Additionally, we examine the robustness of helical, pseudohelical, and antihelical edge states in the presence of nonmagnetic disorders, highlighting the particular fragility of antihelical edge states. Our findings enhance the understanding of the modified Kane-Mele model, providing new insights into its topological properties.