Is seed-only RS3L pre-training sufficient for robustness

Determine whether training the RS3L backbone exclusively with re-seeded jet augmentations—i.e., keeping simulator settings fixed and varying only the random seed—suffices to achieve gains in robustness, without requiring additional augmentations such as final-state radiation scale variations or alternative parton shower models (e.g., Herwig7).

Background

The paper introduces RS3L, a re-simulation-based self-supervised learning approach that uses multiple augmentations derived from stochastic simulators to pre-train a jet-representation backbone. Augmentations include in-domain re-seeding (fixed settings, different random seeds) and out-of-domain variations (final-state radiation scale modifications and an alternate parton shower model, Herwig7).

Robustness is evaluated via Wasserstein distances between classifier outputs for nominal and augmented jets. The authors observe that networks trained with RS3L are generally more robust across simulator variations and note that even using only re-seeded augmentations in pre-training and fine-tuning yields “reasonably robust” networks.

This observation raises the unresolved question of whether robustness gains can be obtained solely through seed-only augmentations, without needing the broader set of systematic variations included in RS3L.

References

Whether this indicates that a gain in robustness is potentially attainable simply by training the backbone with re-seeded jets merits further investigation that we leave to future studies.

— Re-Simulation-based Self-Supervised Learning for Pre-Training Foundation Models  (2403.07066 - Harris et al., 2024) in Section 2.3.1, In-distribution classification and robustness

A notable open question is how robustly equivariant taggers behave under the systematic variations discussed in \Cref{sec:systematic-uq}. This has not yet been studied in any depth, though there is some evidence that they are very sensitive to Delphes versus Geant4-based detector simulation as will be discussed in \Cref{ch:zjets}.

— A High- and Variable-Dimensional Measurement of the $Z$+jets Differential Cross Section with the ATLAS Experiment and Artificial Intelligence  (2608.28449 - Greif, 28 Aug 2026) in Section 4.7.1, “Symmetry-Constrained Architectures”