Information content of representations at the beginning of training

Determine whether the learned representation carries little information early in training, causing the representation-induced velocity offset to remain small and Representation Guidance to behave similarly to plain flow matching before the representation becomes informative.

Background

Representation Guidance is normally enabled after a warm-up period of plain flow matching, although the reported ablation finds little sensitivity to whether guidance begins at epoch 0, 10, 20, or 30.

The paper offers a conjectural explanation for this insensitivity: representations may contain little useful information at the beginning of optimization, making the guidance offset small and allowing training to remain close to ordinary flow matching. This proposed explanation is not established by the reported experiments.

References

We conjecture that the representation carries little information early in training, so $\Delta\mathbf{v}_{\text{rep}}$ is small and RG stays close to plain flow matching.

— Learning Representations to Guide Pixel Diffusion Transformers  (2610.12978 - Pan et al., 9 Oct 2026) in Appendix, Section “Additional Results,” paragraph “Warm-up length”