When TabPFNv2-type architectures outperform TabICLv2 (and vice versa)

Characterize the dataset conditions and task regimes under which TabPFNv2-type architectures are better or worse than the TabICLv2 architecture, providing a systematic comparison that identifies performance advantages for each design.

Background

TabPFNv2-type cell-based architectures and the TabICLv2 design represent two different approaches to tabular in-context learning with distinct computational trade-offs and inductive biases.

The authors observed good empirical performance of TabPFNv2-type architectures but explicitly state they lack a clear conclusion about when those architectures should be preferred over TabICLv2. A principled characterization would guide architecture selection for practitioners and inform future model development.