Test whether inference-time depth findings persist after retraining

Determine whether the findings about the effects of depth, block position, and repeated learned weighting in Chronos-2 group attention remain valid when the model is trained from scratch with the modified attention structure.

Background

All interventions in the paper are performed at inference time on a released Chronos-2 checkpoint. The authors observe that forcing the first block’s attention weights to be uniform substantially improves in-context forecasting, while the effects of later blocks are comparatively small and the accumulated harm depends on repeated application across depth.

Because successive layers consume representations produced by preceding layers, retraining could cause the model to adapt to the modified structure. The paper therefore leaves unresolved whether its conclusions about depth, position, and repetition would transfer from inference-time ablations to a model trained with the altered architecture.