Scale ConSDL to substantially larger foundation models

Determine whether the gains in cross-seed consistency achieved by Consistent Sparse Dictionary Learning (ConSDL) extend to foundation models with billions of parameters and substantially larger dictionaries.

Background

ConSDL is evaluated on relatively small vision and language foundation models because comparing multiple sparse autoencoder methods across dictionary widths, sparsity levels, and random seeds is computationally demanding. Although ConSDL improves dictionary and coefficient consistency in these experiments, the paper does not establish that the same improvements persist at the scale of billion-parameter models or much larger dictionaries. Establishing this would test whether ConSDL is practically useful for large-scale foundation-model interpretability.