Explain scaling differences among tabular foundation models

Determine how model capacity, pretraining, and inference configurations account for the differing data-scaling behavior of tabular foundation models, including Kumo, TabPFN, and TabICL.

Background

The paper evaluates several tabular foundation models by increasing the number of in-context training examples. Unlike parametric models, these foundation models exhibit substantially different scaling trends: some benefit consistently from larger contexts, whereas others flatten or reverse at larger context sizes. The authors identify model capacity, pretraining, and inference configurations as possible sources of this variation but do not establish their respective roles.