Operationalizing Computation in Superposition

Develop a rigorous operational framework for computation under superposition in neural networks, where features are encoded as overlapping linear combinations of neurons, in order to formally specify how such superposed feature representations implement and support computation.

Background

Superposition is hypothesized to explain polysemantic neurons by allowing neural networks to represent more features than neurons through encoding features as overlapping directions in activation space. This compression complicates mechanistic analysis because computation must act on entangled feature directions rather than isolated monosemantic neurons.

A formal operationalization would clarify how computations are executed on superposed features, enabling principled analysis, interventions, and verifiable explanations of model behavior in settings where superposition is prevalent.