Relationship between effective depth and pruning tolerance

Determine whether model-level effective depth, particularly normalized effective depth, reliably predicts the number or fraction of layers that a decoder-only language model can remove using BI-ranked pruning while keeping perplexity within a specified tolerance.

Background

The paper distinguishes effective depth as a global accumulated-state diagnostic from BI as a local layer-pruning score. An exploratory analysis across eight decoder-only models reports a negative correlation between normalized effective depth and BI-ranked pruning tolerance in the full panel, but the primary depth-normalized fraction does not survive a leave-one-out analysis that removes OLMo-2-1B. The authors therefore regard the relationship as suggestive rather than conclusive.

The unresolved question is whether the observed association reflects a robust predictive relationship or measurement artifacts caused by the coarse pruning grid and variation in model depth. A finer pruning grid and broader model evaluation are needed to establish predictive validity.