Topological phase transition with nanoscale inhomogeneity in (Bi$_{1-x}$In$_{x}$)$_{2}$Se$_{3}$

Abstract: Topological insulators are a class of band insulators with non-trivial topology, a result of band inversion due to the strong spin-orbit coupling. The transition between topological and normal insulator can be realized by tuning the spin-orbit coupling strength, and has been observed experimentally. However, the impact of chemical disorders on the topological phase transition was not addressed in previous studies. Herein, we report a systematic scanning tunneling microscopy/spectroscopy and first-principles study of the topological phase transition in single crystals of In doped Bi$_2$Se$_3$. Surprisingly, no band gap closure was observed across the transition. Furthermore, our spectroscopic-imaging results reveal that In defects are extremely effective "suppressors" of the band inversion, which leads to microscopic phase separation of topological-insulator-like and normal-insulator-like nano regions across the "transition". The observed topological electronic inhomogeneity demonstrates the significant impact of chemical disorders in topological materials, shedding new light on the fundamental understanding of topological phase transition.