Test whether pretrained initialization removes the depth-dependent performance gap

Determine whether applying REVE foundation-model initialization to the depth-12 EEG encoder would eliminate the performance gap between the depth-12 and depth-22 randomly initialized encoders in within-task and cross-task video decoding.

Background

The study compares a depth-12 randomly initialized REVE-like encoder with a depth-22 randomly initialized encoder and finds that the shallower model performs better within task but worse on cross-task transfer at the largest cohort sizes. However, the comparison does not include a depth-12 encoder initialized from the pretrained REVE foundation model.

Consequently, the reported depth effect is potentially confounded with the interaction between architecture depth and initialization. Testing pretrained initialization at depth 12 would establish whether the observed depth-dependent gap persists when both architectures benefit from foundation-model initialization.

References

And with no pretrained depth-12 arm, whether that initialisation would erase the gap is untested.

— Scaling subjects in cross-modal alignment: video decoding with EEG foundation model  (2610.09287 - Truong et al., 7 Oct 2026) in Section 3.2, “Depth matters under random initialisation, and the sign flips”