Hysteresis without coexistence: disorder-rounded first-order transitions in a van der Waals magnet

Abstract: Quenched disorder can profoundly modify phase transitions. In low-dimensional systems, theory predicts that even weak quenched disorder can round the thermodynamic discontinuities associated with a first-order phase transition. Here, we employ time-domain terahertz spectroscopy to investigate the quasi-two-dimensional trimerized kagome van der Waals magnet family Nb$<em>3$Cl$</em>{8-x}$Br$<em>x$ ($x=0$, 1 and 8). We observe the emergence of an additional phonon branch upon Br substitution, whose spectral weight increases and frequency softens with increasing Br concentration. The temperature evolution of the phonon frequencies reveals a clean first-order transition in Nb$_3$Cl$_8$ characterized by macroscopic phase coexistence and thermal hysteresis. In contrast, the transition in the substitutionally disordered compound Nb$_3$Cl$_7$Br retains its hysteresis while exhibiting a substantially broadened transition with no resolvable macroscopic phase coexistence. These observations reveal disorder-induced fragmentation of the transition into locally favored domains instead of well-defined bulk phases separated by stable phase boundaries. The behavior is consistent with the Imry-Wortis and the Aizenman-Wehr scenarios for the effect of quenched disorder in low-dimensional systems, which destabilizes macroscopic phase coexistence and rounds the thermodynamic discontinuities associated with first-order transitions. Thermal hysteresis persists in the disordered compound despite the lack of resolvable coexistence, indicating that the two features often treated as a single hallmark of first-order character arise distinctly and can be separated by disorder. Moreover, our results establish Nb$_3$Cl$</em>{8-x}$Br$_x$ as a promising platform for investigating the effects of disorder on first-order transitions in low-dimensional systems.

• The paper demonstrates that disorder in a kagome magnet (Nb3Cl8-xBrx-(x= 0-8)) can decouple thermal hysteresis from phase coexistence, two observables typically associated with first-order transitions, due to one element substitution.
• Using TDTS, the research documents different phonon activation and softening mechanisms as consequential for each substitution.xlsx[x=0-8], largely dependent upon the spectroscopic phase of the overall compation.
• This behavior is consistent with the theories of Imry-Wortis fragmentation and the Aizenman-Wehr rounding scenario i.e. discontinuities blur upon introducing disorder whereas kinetic barriers and memory-dependent switching still exist for 2D systems.

Overview

This paper reports time-domain terahertz spectroscopy (TDTS) measurements on the quasi-two-dimensional trimerized kagome van der Waals magnet family Nb$_3$Cl$_{8-x}$Br$_x$ ($x = 0, 1, 8$), using the infrared-active $E_u$ phonons as a local probe of the coupled structural and magnetic first-order transition. The central finding is a clean separation of two signatures that are conventionally treated as inseparable hallmarks of first-order character. In stoichiometric Nb$_3$Cl$_8$, the transition exhibits both macroscopic $\alpha$–$\beta$ phase coexistence and thermal hysteresis. In the substitutionally disordered compound Nb$_3$Cl$_{8-x}$0Br, hysteresis persists while macroscopic coexistence becomes unresolvable and the transition broadens substantially. The authors interpret this as disorder-induced fragmentation of the transition into locally favored domains, consistent with the Imry–Wortis destabilization of phase coexistence (2608.19352) and the Aizenman–Wehr rounding of thermodynamic discontinuities in low dimensions.

Structural background and phonon assignments

Nb$_{8-x}$1X$_{8-x}$2 consists of weakly coupled layers of Nb trimers on a breathing kagome lattice. The high-temperature $_{8-x}$3-phase has AB stacking between van der Waals layers ($_{8-x}$4, two-layer unit cell); the low-temperature $_{8-x}$5-phase develops alternating AB and AA$_{8-x}$6 stacking (six-layer unit cell), accompanied by an almost complete loss of magnetization consistent with a singlet ground state. The transition temperature rises from roughly 90 K in Nb$_{8-x}$7Cl$_{8-x}$8 to about 382 K in Nb$_{8-x}$9Br$_x$0. The low-temperature symmetry of Nb$_x$1Cl$_x$2 remains contested in the literature, with $_x$3, $_x$4, and $_x$5 all proposed; this paper contributes polarimetric evidence favoring $_x$6.

TDTS measurements of the transmission coefficient $_x$7 identify three doubly degenerate odd-parity $_x$8 phonons. Their frequencies at 5 K and 300 K are:

Compound	$_x$9 (5 K)	$x = 0, 1, 8$0 (300 K)	$x = 0, 1, 8$1 (5 K)	$x = 0, 1, 8$2 (300 K)	$x = 0, 1, 8$3 (5 K)	$x = 0, 1, 8$4 (300 K)
Nb$x = 0, 1, 8$5Cl$x = 0, 1, 8$6	3.22	3.11	—	—	3.92	3.90
Nb$x = 0, 1, 8$7Cl$x = 0, 1, 8$8Br	3.15	3.03	2.72	2.65	3.84	3.82
Nb$x = 0, 1, 8$9Br$E_u$0	2.75	2.73	2.17	2.13	3.56	3.60

The $E_u$1 mode softens across the transition and therefore serves as the primary order-parameter-sensitive probe; $E_u$2 is activated by Br substitution, its spectral weight growing and frequency softening monotonically from Nb$E_u$3Cl$E_u$4Br to Nb$E_u$5Br$E_u$6, consistent with the larger halogen mass and modified force constants. $E_u$7 is essentially insensitive to the transition. A magnetic origin for these modes is excluded by their lack of field dependence.

Clean first-order transition in Nb$E_u$8Cl$E_u$9

In Nb$_3$0Cl$_3$1, warming spectra show a second absorption peak emerging on the low-frequency side of $_3$2 near 110 K, which gains spectral weight while the original mode fades above approximately 130 K; cooling reverses this evolution between 90 K and 70 K. Because only one peak survives deep within each phase, the doublet cannot be attributed to symmetry-lowering-induced mode splitting. Its interpretation is instead direct spectroscopic evidence of macroscopic coexistence of $_3$3- and $_3$4-phase domains over a finite temperature interval, together with an abrupt $_3$5 frequency shift of about 0.1 THz and clear warming/cooling hysteresis. This constitutes the clean reference against which the disordered compound is compared.

Disorder-rounded transition in Nb$_3$6Cl$_3$7Br

Nb$_3$8Cl$_3$9Br presents a qualitatively different phenomenology. Both $_8$0 and $_8$1 evolve smoothly through the transition, with total shifts of approximately 0.12 THz and 0.07 THz respectively, centered near 135 K on warming and 110 K on cooling. No resolvable coexistence of two phonon branches appears, a result confirmed on a thinner sample with finer temperature steps. Yet the warming and cooling curves remain clearly separated: thermal hysteresis survives even though the spectroscopic discontinuity is strongly rounded. Additionally, the $_8$2 linewidth in Nb$_8$3Cl$_8$4Br is approximately three times that of Nb$_8$5Br$_8$6 and five times that of Nb$_8$7Cl$_8$8, providing independent spectroscopic evidence of enhanced quenched disorder from halogen substitution.

The paper's key claim follows directly: hysteresis and phase coexistence, often treated interchangeably as signatures of first-order transitions, arise from distinct physics and can be separated by disorder. Coexistence reflects near-degeneracy of bulk free energies balanced against interfacial cost, whereas hysteresis reflects kinetic irreversibility. In the disordered compound, spatial inhomogeneity causes different regions to transform at different temperatures, producing history dependence, while the same disorder fragments the system into many small locally favored domains rather than two bulk phases separated by stable interfaces. The authors distinguish the relevant barriers explicitly: the nucleation barrier governing clean-sample coexistence shrinks with domain size, whereas the depinning barrier sustaining hysteresis is set by the local disorder potential and is largely size-independent. This explains how one observable can be suppressed while the other persists or is enhanced.

Relation to theory

For second-order transitions, the Harris criterion ($_8$9) governs disorder relevance, but it does not apply to first-order transitions, where correlation lengths remain finite. Imry and Wortis argued that competition between bulk free-energy gain and interfacial energy cost destabilizes macroscopic coexistence in disordered systems, particularly in low dimensions. Aizenman and Wehr proved rigorously that arbitrarily weak quenched disorder rounds the discontinuities of first-order transitions for $\alpha$0. The observations in Nb$\alpha$1Cl$\alpha$2Br—hysteresis without resolvable coexistence—are qualitatively consistent with both scenarios, and also echo nonequilibrium results from the random-field Ising model (RFIM), where above a critical disorder level discontinuities smear while hysteresis and memory-dependent switching survive.

The authors draw analogies to two RFIM-related experiments: capillary condensation of $\alpha$3He in silica aerogel, where condensation changes from abrupt avalanche-like filling to smooth curves while adsorption–desorption hysteresis remains, and Co/CoO bilayers, where tuned structural disorder drives hysteretic loops from sharp reversal to smooth loops with scaling near an apparent critical disorder. They are careful to state that these serve only as conceptual analogies; no precise mapping of the RFIM onto Nb$\alpha$4Cl$\alpha$5Br$\alpha$6 is demonstrated.

Polarimetry and ground-state characterization

THz polarimetry under a 6.5 T field on Nb$\alpha$7Cl$\alpha$8Br shows negligible cross-polarized transmission $\alpha$9 at all temperatures, consistent with preserved threefold rotational and inversion symmetries across the transition and supporting the $\beta$0 assignment over lower-symmetry alternatives such as $\beta$1 or $\beta$2. Transformation to the circular basis shows nearly identical $\beta$3 and $\beta$4 spectra, indicating absence of circular dichroism and confirming that Br substitution preserves the nonmagnetic singlet ground state despite substantially modifying the transition character. The data further indicate that the infrared-active $\beta$5 phonons carry no observable net angular momentum, in contrast to the chiral Raman-active $\beta$6 modes reported elsewhere in this family.

Limitations and open questions

Several caveats bear directly on the interpretation. The bulk crystals studied are quasi-2D rather than strictly 2D, so the Aizenman–Wehr rounding theorem, which applies rigorously at $\beta$7, motivates but does not strictly govern the observed behavior; the consistency is qualitative. Only a single intermediate composition ($\beta$8) was measured, so the disorder dependence of the rounding—for example, whether a critical disorder level exists as in the RFIM—remains untested. The persistence of hysteresis in the disordered compound is expected to be timescale-dependent and to vanish in the ideal equilibrium limit, but no time-dependent measurements were performed. Direct imaging of the postulated locally favored domains, and exfoliation toward the few-layer or monolayer limit to realize a truly 2D system and observe the expected dimensional crossover, remain outstanding experimental challenges explicitly identified by the authors.

Conclusion

Using TDTS of Br-substitution-activated and transition-sensitive $\beta$9 phonons, this work demonstrates that substitutional disorder in the quasi-2D kagome magnet Nb$_3$0Cl$_3$1Br$_3$2 suppresses macroscopic phase coexistence while preserving thermal hysteresis, effectively decoupling two observables usually conflated as hallmarks of first-order transitions. The behavior aligns with the Imry–Wortis fragmentation picture and the Aizenman–Wehr rounding scenario, and establishes Nb$_3$3Cl$_3$4Br$_3$5 as a chemically tunable platform for quantitative studies of disorder-rounded first-order transitions approaching the 2D limit.