Causes of DeBERTa performance differences across investment universes

Ascertain whether the underperformance of the encoder-only model DeBERTa in the large investment universe is attributable to model size or to other factors, and explain the causes of DeBERTa’s varying performance across different small investment universes when fine-tuned to predict forward stock returns from financial newsflow.

Background

Empirically, decoder-only models often yield stronger portfolios in larger universes, while results are mixed in smaller universes. DeBERTa underperforms in the large universe and shows variable performance in small universes.

The authors explicitly state uncertainty about whether DeBERTa’s large-universe underperformance stems from model scale or other factors, and seek explanations for its varying results in small universes.