Linear Separability of Language Representations

Determine whether each language occupies a distinct, linearly separable region of a multilingual Transformer’s activation space at every layer, such that a string generates text in language \(\mathcal{L}_i\) if and only if its representation at that layer lies in the corresponding region.

Background

The paper models language-specific activation regions as subsets of the latent representation space and uses lightweight linear probes to identify those regions. The proposed optimal-control intervention relies on these regions being meaningful and sufficiently well characterized: steering an activation into the region associated with a target language is intended to induce generation in that language.

The unresolved hypothesis is important because the theoretical guarantees of the proposed LiSeCo-based intervention concern membership in a classifier-defined region, whereas successful language control additionally requires that the region correspond causally to language generation. Establishing the hypothesized equivalence between activation-region membership and generated language would therefore validate a central assumption of the control framework.

References

In the Linear Separability Hypothesis, each language \mathcal{L}_i is hypothesized to occupy a distinct region \mathcal{R}_ti of activation space for each layer t. Formally, a string s\in\Sigma* generates text in language \mathcal{L}_i if and only if its corresponding representations x_t\in\mathcal{R}_ti, where \mathcal{R}_ti is a linearly-separable region in embedding space.

— Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control  (2609.16967 - Michaelov et al., 15 Sep 2026) in Section 5.1, paragraph titled “Linear Representation Hypothesis”

The Semantic Hub Hypothesis provides crucial insight into how multilingual control should be designed. It posits that semantically equivalent inputs s{\mathcal{L}_s} and s{\mathcal{L}_t} from source and target languages have representations such that:

— Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control  (2609.16967 - Michaelov et al., 15 Sep 2026) in Section 5.2, paragraph titled “Semantic Hub Hypothesis”