Test learning-rate sensitivity across IntWorld transfer experiments

Determine whether the learning-rate sensitivity observed in few-shot transfer to new PermWorld tasks also occurs when IntWorld models are fine-tuned on held-out integer tasks.

Background

The paper finds that the apparent advantage of PermWorld pretraining for few-shot transfer depends strongly on the fine-tuning learning rate: the advantage over random initialization is substantially larger at a learning rate of 10^-5 than at 3 × 10^-4.

Because this sensitivity may affect the interpretation of transfer gains, the authors identify cross-world replication as an unresolved empirical question. Specifically, they leave open whether analogous learning-rate sensitivity appears in IntWorld, where transfer is evaluated on held-out integer tasks such as predecessor, least common multiple, modular addition, and sorting.