Representation of Feature Clusters Under Superposition

Characterize how neural networks represent clusters of related features when features are encoded in superposition, including how clustering structure is reflected in activation space and how such structure influences computation and interference.

Background

Superposition implies that multiple features can share overlapping neural resources; understanding how related features cluster within this regime is essential for disentangling representations and for building dictionaries (e.g., via sparse autoencoders) that reflect true underlying structure.

Clarifying cluster representation would aid in interpreting learned abstractions, mitigating interference, and improving methods for feature extraction and manipulation.