Optimal stacking ratios and block orderings for HART

Determine whether alternative ratios or orderings of Attention and RWKV blocks in the HART hybrid time-series foundation model yield further gains over the evaluated 1:1 alternating configuration.

Background

HART is evaluated with a fixed 1:1 stacking configuration in which each module group contains one Transformer attention block followed by one RWKV block. This design provides alternating global historical-token retrieval and recurrent temporal modeling.

The paper does not determine whether changing the relative number of attention and RWKV blocks, or changing their ordering, would improve forecasting performance. The authors explicitly leave this architectural-design question open.