Robustness of topological states to lattice instability in non-symmorphic topological insulator KHgSb

Abstract: We report a polarized Raman scattering study of non-symmorphic topological insulator KHgSb with hourglass-like electronic dispersion. Supported by theoretical calculations, we show that the lattice of the previously assigned space group $P6_3/mmc$ (No. 194) is unstable in KHgSb. While we observe one of two calculated Raman active E${2g}$ phonons of space group $P6_3/mmc$ at room temperature, an additional A${1g}$ peak appears at 99.5 ~cm$^{-1}$ upon cooling below $T^*$ = 150 K, which suggests a lattice distortion. Several weak peaks associated with two-phonon excitations emerge with this lattice instability. We also show that the sample is very sensitive to high temperature and high laser power, conditions under which it quickly decomposes, leading to the formation of Sb. Our first-principles calculations indicate that space group $P6_3mc$ (No. 186), corresponding to a vertical displacement of the Sb atoms with respect to the Hg atoms that breaks the inversion symmetry, is lower in energy than the presumed $P6_3/mmc$ structure and preserves the glide plane symmetry necessary to the formation of hourglass fermions.