Is seed-only RS3L pre-training sufficient for robustness

Determine whether training the RS3L backbone exclusively with re-seeded jet augmentations—i.e., keeping simulator settings fixed and varying only the random seed—suffices to achieve gains in robustness, without requiring additional augmentations such as final-state radiation scale variations or alternative parton shower models (e.g., Herwig7).

Background

The paper introduces RS3L, a re-simulation-based self-supervised learning approach that uses multiple augmentations derived from stochastic simulators to pre-train a jet-representation backbone. Augmentations include in-domain re-seeding (fixed settings, different random seeds) and out-of-domain variations (final-state radiation scale modifications and an alternate parton shower model, Herwig7).

Robustness is evaluated via Wasserstein distances between classifier outputs for nominal and augmented jets. The authors observe that networks trained with RS3L are generally more robust across simulator variations and note that even using only re-seeded augmentations in pre-training and fine-tuning yields “reasonably robust” networks.

This observation raises the unresolved question of whether robustness gains can be obtained solely through seed-only augmentations, without needing the broader set of systematic variations included in RS3L.