Extent of superposition across architectures, brain regions, and tasks

Ascertain the extent to which neural representations operate in superposition—encoding more latent features than neurons via linear compression—across diverse artificial neural network architectures, across biological brain regions, and across task regimes, using empirical measurements.

Background

The paper’s core analysis assumes that neural systems encode features in superposition, which systematically deflates standard representational alignment metrics when computed on raw activations. While there is growing evidence of polysemanticity and superposition-like behavior in artificial and biological systems, the authors note that the generality of superposition across architectures, brain regions, and different task contexts is not established.

Clarifying the prevalence and degree of superposition is crucial to determine when feature-based alignment methods are necessary and to understand whether observed misalignment stems from coding differences or true representational differences.