Unconventional domain tessellations in moiré-of-moiré lattices (2402.15760v2)

Abstract: Imposing incommensurable periodicity on the periodic atomic lattice can lead to complex structural phases consisting of locally periodic structure bounded by topological defects. Twisted trilayer graphene (TTG) is an ideal material platform to study the interplay between different atomic periodicities, which can be tuned by twist angles between the layers, leading to moir\'e-of-moir\'e lattices. Interlayer and intralayer interactions between two interfaces in TTG transform this moir\'e-of-moir\'e lattice into an intricate network of domain structures at small twist angles, which can harbor exotic electronic behaviors. Here we report a complete structural phase diagram of TTG with atomic scale lattice reconstruction. Using transmission electron microscopy combined with a new interatomic potential simulation, we show several large-scale moir\'e lattices, including triangular, kagome, and a corner-shared hexagram-shaped domain pattern. Each domain is bounded by a two-dimensional network of domain wall lattices. In the limit of small twist angles, two competing structural orders-rhombohedral and Bernal stackings-with a slight energy difference, cause unconventional lattice reconstruction with spontaneous symmetry breaking and nematic instability, highlighting the importance of long-range interlayer interactions across entire van der Waals layers. The diverse tessellation of distinct domains, whose topological network can be tuned by the adjustment of the twist angles, establishes TTG as a platform for exploring the interplay between emerging quantum properties and controllable nontrivial lattices.