Impact of recurrent depth on transformer performance

Determine how recurrent depth affects the performance of transformer-based language models by rigorously characterizing the relationship between the number of recurrent iterations and model performance across tasks and conditions.

Background

The work situates depth-recurrence within broader research on depth in transformers and latent reasoning. Although many studies examine depth effects in transformers, a clear understanding of how recurrent depth specifically influences performance is not established.

Clarifying the performance implications of recurrence depth would inform architectures and training strategies for models that exploit test-time compute by looping internal layers.