Interplay between surface Dirac and Rashba states specific for topologically nontrivial van der Waals superlattices

Abstract: Here we show that, in contrast to the observed surface states in well studied pnictogen chalcogenide van der Waals (vdW) topological insulators (TIs) with quintuple layer (QL) or septuple layer~(SL) structure, in superlattices, comprising the alternating QL and SL vdW blocks, the Dirac state becomes accompanied by emergent spin-polarized states of the Rashba type. This specific feature is caused by an inequivalence of the surface and subsurface structural blocks and an electrostatic potential bending near the surface. Within density functional theory~(DFT) and $\emph{ab-initio}$ tight-binding~(TB) calculations we analyze peculiarities of these states depending on the surface termination, structural parameters and chemical composition. It is found that their possible hybridization with the Dirac state significantly affects its dispersion and spatial localization. We analyze the influence of intrinsic magnetism on behavior of the termination-dependent surface states for magnetic QL/SL superlattices. These findings provide a better understanding of the existing experimental observations of such QL/SL alternating superlattices.