Generalization Beyond Five-Way Five-Shot Episodes

Determine whether the RepICL reusable in-context prediction procedure generalizes to episode shapes with different numbers of classes (ways) or labeled examples per class (shots).

Background

RepICL and RepShiftBench are evaluated exclusively on 5-way 5-shot classification episodes. The paper therefore does not establish whether the learned prediction procedure remains effective when the number of classes or the amount of labeled support data changes. Extending evaluation and method design to other episode shapes is identified as a future research direction.