Representation of English filler-gap constructions

Determine the expected representation of filler-gap constructions in English, including whether and to what extent distinct construction types (e.g., embedded and matrix wh-questions, restrictive relatives, clefts, pseudoclefts, and topicalization) share a common underlying representation.

Background

Filler-gap constructions in English occur in multiple types (such as wh-questions, relative clauses, clefts, pseudoclefts, and topicalization). Linguistic theory debates the degree to which these constructions share underlying representations or rely on distinct mechanisms.

The paper uses LLMs and the proposed perturbation method to examine representational transfer among these constructions, framing the broader linguistic question of their representation as open and investigating it through causal generalization patterns in models.

References

Thus, unlike coarse-grained word senses that enjoy relatively strong consensus, the expected representation of FG constructions in English is an open question, one which perturbation allows us to study through the vehicle of LMs.

— Perturbation: A simple and efficient adversarial tracer for representation learning in language models  (2603.23821 - Rozner et al., 25 Mar 2026) in Section 5: Syntactic Representations