Identification and Structural Characterization of Twisted Atomically Thin Bilayer Materials by Deep Learning

Abstract: Two-dimensional materials are expected to play an important role in next-generation electronics and optoelectronic devices. Recently, twisted bilayer graphene and transition metal dichalcogenides have attracted significant attention due to their unique physical properties and potential applications. In this study we describe the use of optical microscopy to collect the color space of chemical vapor deposition (CVD) molybdenum disulfide ($\mbox{MoS}_2$), and the application of a semantic segmentation convolutional neural network (CNN) to accurately and rapidly identify thicknesses of $\mbox{MoS}_2$ flakes. A second CNN model is trained to provide precise predictions on the twist angle of CVD-grown bilayer flakes. This model harnessed a dataset comprising over 10,000 synthetic images, encompassing geometries spanning from hexagonal to triangular shapes. Subsequent validation of the deep learning predictions on twist angles was executed through the second harmonic generation and Raman spectroscopy. Our results introduce a scalable methodology for automated inspection of twisted atomically thin CVD-grown bilayer.

• The paper introduces a novel deep learning framework that integrates semantic segmentation for thickness classification with a regression model for twist angle estimation.
• It leverages synthetic image datasets to train the model and validates predictions against optical spectroscopy, outperforming conventional image analysis techniques.
• The method enables scalable, high-throughput characterization of 2D materials, potentially accelerating advances in moiré physics and materials design.

Deep Learning-Based Characterization of Twisted Bilayer Atomically Thin Materials

Introduction

The paper "Identification and Structural Characterization of Twisted Atomically Thin Bilayer Materials by Deep Learning" (2604.15960) presents an integrated deep learning (DL) framework for the automated optical inspection and structural analysis of chemical vapor deposition (CVD)-grown twisted bilayer transition metal dichalcogenides (TMDs) and related van der Waals two-dimensional materials. The crucial role of twist angle in determining the physical properties of bilayer materials is underscored, with emphasis on the need for scalable, high-throughput optical characterization techniques beyond conventional and resource-intensive methods such as AFM, Raman, SHG, and TEM/STM. The authors address this by combining semantic segmentation-based flake thickness identification and regression-based twist angle determination within an extensible machine learning pipeline.

Methodology: Dataset Construction, Model Training, and Segmentation

A critical bottleneck for DL approaches in this context is the lack of annotated experimental data, especially for twist angle characterization. The authors overcome this by building a two-stage pipeline:

1. Semantic Segmentation for Thickness Classification: Optical micrographs of CVD-grown TMDs, particularly MoS$_2$, are labeled to distinguish monolayer, bilayer, and thicker domains. Four segmentation architectures were evaluated—DeepLabV3, Fully Convolutional Networks (FCN), LR-ASPP, and U-Net—revealing U-Net's superiority in capturing morphological features and boundaries in irregularly shaped flakes.

   Figure 1: Workflow for thickness identification and twist angle prediction of CVD-grown bilayer TMDs using deep learning.

   

   Figure 2: Comparative performance of DeepLabV3, FCN, LR-ASPP, and U-Net models for segmentation of flake thickness in optical micrographs.

2. Regression for Twist Angle Estimation: As direct annotation of twist angles in experimental data is impractical, the authors generated over 10,000 synthetic images of bilayer polygons (triangles/hexagons) with controlled twist angles and realistic edge perturbations, serving as ground truth for supervised training of a ResNet-based regression CNN. The network is optimized to predict continuous twist angles from cropped bilayer micrographs, both in synthetic and experimental settings.

   Figure 3: Schematic of synthetic dataset generation and ResNet-based regression for twist angle estimation.

Numerical Evaluation and Verification

Robust experimental validation is provided by comparing DL-predicted twist angles with OpenCV-based geometric analysis and, primarily, with optical SHG (second harmonic generation) and low-wavenumber Raman spectroscopy. The results demonstrate:

• The regression CNN generalizes well to real micrographs, displaying resilience to shape irregularities and outperforming conventional image processing, which often failed on irregular flakes.
• Histogram analysis of the twist angles in CVD MoS$_2$ bilayers reveals a distribution reflective of the underlying growth dynamics.
• SHG and Raman measurements correlate well with the model's predictions, confirming the reliability of optical micrograph-based twist angle inference.

  Figure 4: Comparison of CNN-predicted twist angles with OpenCV methods and SHG verification; distribution of measured angles across bilayer MoS$_2$ samples.

Characterization of Moiré Phonons

The methodology is leveraged to further study twist-angle-dependent moiré phonons using Raman spectroscopy. Distinct moiré phonon branches (folded TA, LA, and $A_1'$ modes) are observed and tracked as a function of twist angle, with experimental data matching first-principles calculations. Some deviations near intermediate twist angles are attributed to strain and structural inhomogeneity, demonstrating the system's sensitivity to local sample conditions.

Figure 5: Raman spectra of moiré phonons in bilayer MoS$_2$, showing twist-angle-dependent frequency shifts and comparison between experimental and calculated phonon modes.

Discussion and Implications

This work introduces several significant innovations:

• Automated, High-Throughput Optical Structural Analysis: The pipeline supports large-scale, unbiased inspection of 2D samples, facilitating statistical studies of twist angle distributions and their correlation with emergent physical properties.
• Synthetic Data as Ground Truth: The synthetic dataset approach enables training of regression models without the need for laborious manual annotation, and is extensible to heterostructures and various shapes.
• Model Generalizability: The authors claim effective transfer of methods to CVD-grown graphene, h-BN, and their heterostructures, opening the way for cross-material analysis in 2D electronics and optoelectronics.

Bold claims in the paper include the assertion that this is the first fully automated, DL-based method for twist angle determination in 2D bilayer materials using only optical micrographs, with accuracy approaching that of specialized spectroscopic methods. Performance metrics such as global accuracy and mean intersection over union are cited (albeit in supplementary material) to underline U-Net's superiority in segmentation. The method's robustness to sample irregularities and its ability to process large datasets are emphasized.

Practical implications include acceleration of research in moiré physics, enabling rapid candidate screening for correlated phenomena (e.g., superconductivity, correlated insulators) and optimization of CVD synthesis protocols. Theoretically, the framework could catalyze the coupling of large-scale structure-property datasets with generative modeling and inverse materials design. The open-source code and datasets are likely to serve as a benchmark for future studies on autonomous laboratories and computer vision-driven materials discovery.

Future Directions

Anticipated extensions include:

• Incorporation of multi-modal data, such as photoluminescence or cathodoluminescence imaging, into the pipeline for richer property mapping.
• Deployment in autonomous or semi-autonomous growth-monitoring setups.
• Extending the approach to more complex heterostructures, few-layer systems, and non-TMD 2D crystals.
• Integration with generative AI models for inverse design and defect prediction.

Conclusion

The paper establishes a robust machine learning approach for the optical characterization of twisted bilayer TMDs, combining semantic segmentation and regression architectures for thickness and twist angle identification, respectively. The approach bypasses labor-intensive spectroscopy and geometrically robustifies twist angle predictions using synthetically generated datasets, validated against experimental optical measurements. This framework substantially enhances the scalability of structural analysis in 2D materials and provides a foundation for further developments in automated materials characterization and discovery.