Mechanism of Wanda pruning’s demographic error compounding

Identify whether Wanda’s magnitude-times-activation pruning criterion preferentially removes weights serving low-frequency demographic input distributions, thereby causing the observed demographic fairness-gap compounding in Whisper models.

Background

The paper finds that 50% unstructured Wanda pruning substantially increases demographic disparities in several settings, including a more-than-doubling of the Black/African American-versus-Asian temporal-taxation differential for Whisper-large-v3 on Fair-Speech. The effect increases with model capacity, but the experiments do not establish why Wanda pruning produces this pattern.

A long-tail explanation is proposed: pruning may remove weights associated with low-frequency input distributions that correspond to groups already poorly served at full precision. The paper notes that deletion-rate decompositions are consistent with, but do not decisively establish, this explanation, and suggests weight-level attribution to training-data subpopulations or controlled variation of long-tail representation in calibration data as decisive tests.

References

Our experiments establish the empirical asymmetry but cannot adjudicate the mechanism: the deletion-rate decompositions in Appendix~\ref{app:full-results} are consistent with a long-tail account but are not decisive. A decisive test would need weight-level attribution to training-data subpopulations, or a controlled experiment varying long-tail representation in the calibration set.

— Temporal Taxation Compounds Under Post-Training Compression of Whisper Models  (2609.28739 - Ginjala et al., 23 Sep 2026) in Section 6, “Concluding Remarks and Discussion”