Configuration-dependent typical noise level for homogenized transformers

Develop a configuration-dependent characterization of the effective noise level in the homogenized transformer dynamics by replacing the worst-case variance proxy used to define the noise scale parameter with a trajectory-dependent quantity based on the time integral of the expected squared norm of the fluctuation field G(X(t),θ), and determine how this refinement alters the analysis in the simplex-configuration regime.

Background

In deriving the homogenized SDE limit, the authors bound stochastic fluctuations using a worst-case variance proxy σ to define the scale parameter α, ensuring global weak error controls. This uniform bound may be pessimistic in structured configurations, such as when the Gram matrix follows a simplex symmetry.

For simplex-like initial data, the authors suggest a more refined, configuration-dependent measurement of noise that tracks the actual trajectory via ∫_0T E||G(X(t),θ)||2 dt rather than a supremum over all configurations. They explicitly leave carrying out this refinement for future work.

References

In the present simplex setting one may instead hope to characterize a configuration-dependent typical noise level by removing the supremum and studying \int_{0}{T}\mathbb{E}|G(X(t),\bm\theta)|2 \, dt. We leave such a refinement for future work.

— Homogenized Transformers  (2604.01978 - Koubbi et al., 2 Apr 2026) in Subsection “Simplex data … low temperature” (Remark after Theorem on simplex dynamics)