Unified-template synergy between prompts and sequence length in ColBERT pre-training

Establish whether ColBERT treats the combination of explicit pre-training prompts ("search_query:" and "search_document:") and increased sequence length as a unified template that yields synergistic performance gains, and isolate the individual and combined mechanisms through controlled ablations across prompt variations and incremental length scales.

Background

The authors report additive gains from both adding prompts and increasing query/document lengths, with larger combined improvements when the unsupervised base is a ColBERT model pre-trained with prompts. They hypothesize a synergy whereby structural alignment (prompts) and representational capacity (longer sequences) jointly drive improvements.

They present this as a preliminary conjecture and call for broader experiments varying prompt forms and lengths to disentangle the mechanisms and confirm whether ColBERT uniquely treats this configuration as a unified template compared to dense models.