Determine the appropriate estimand for position-varying latent effects

Determine whether ablation-based causal evaluation of a sparse-autoencoder latent should report a distribution over firing tokens, an activation-weighted expectation, or a context-conditioned estimand rather than a single scalar, and identify which alternative is most stable across dictionaries at equal computational cost.

Background

The paper shows that a latent’s measured causal effect can vary substantially across the tokens at which it fires, with measurement position accounting for a large share of the observed variance. Because the conventional top-activating-token protocol allows the dictionary to select the evaluation position, a scalar causal score may conflate the latent’s effect with the particular token chosen.

The authors explicitly leave unresolved what quantity should replace or supplement that scalar. They propose three empirically testable possibilities—an across-token distribution, an activation-weighted expectation, and a context-conditioned estimand—and state that the alternatives should be compared for cross-dictionary stability using the paper’s crossed design.