Characterize the functionality of MLP modules in the plural-reference circuit

Characterize the functionality of the multilayer perceptron (MLP) modules within the circuit responsible for plural-reference pronoun prediction.

Background

The analysis focuses primarily on attention heads and does not thoroughly investigate MLPs. Although interventions on the late-layer MLP module strongly affect the probability of a plural pronoun, the role and computational function of that module within the plural-reference circuit remain unresolved.

References

We also do not thoroughly investigate the function of MLPs. Although we show that intervening on the MLP module at the late layer strongly affects $P_{pl}$, we know little about its functionality in the circuit.

— A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities  (2609.03687 - Danh et al., 3 Sep 2026) in Limitations section