Ambiguity-Encoding Neurons in Neural Models

• Ambiguity-Encoding Neurons (AENs) are defined as sparse hidden dimensions in models whose activations reliably predict and influence ambiguity detection.
• In large language models, targeted perturbation of a few AENs causes significant drops in ambiguity classification performance, demonstrating their causal role.
• Research on AENs bridges direct neuron-level evidence and broader distributed coding theories, highlighting practical implications for ambiguity resolution and context-sensitive processing.

Searching arXiv for the specified papers to ground the article in the current preprint record. arXiv search query: (Tang et al., 2019) OR (Islah et al., 2023) OR (Marco et al., 2024) OR (Zhang et al., 17 Sep 2025) OR (Worden, 9 Dec 2025) OR (Bauer et al., 29 Dec 2025) Ambiguity-Encoding Neurons (AENs) denote neurons, hidden dimensions, or closely related representational units whose activity carries information about ambiguity, underspecification, or competition among candidate interpretations. In the literature surveyed here, the term is introduced operationally for LLMs: an AEN is a neuron, or tiny set of neurons, whose activation is highly predictive of whether a question is ambiguous under a linear probe and whose perturbation strongly degrades ambiguity classification (Zhang et al., 17 Sep 2025). Closely related work extends the concept beyond this narrow operational definition. Some papers provide direct sparse neuron-level evidence, whereas others provide representation-level, population-level, dendritic, or normative accounts that are AEN-adjacent rather than definitive. Taken together, this literature frames ambiguity not as a single phenomenon but as a family of coding problems involving lexical sense selection, question underspecification, perceptual competition, contextual disambiguation, and ambiguous-but-efficient population codes (Tang et al., 2019, Islah et al., 2023, Marco et al., 2024, Worden, 9 Dec 2025, Bauer et al., 29 Dec 2025).

1. Definition and conceptual boundaries

In its strictest current usage, an AEN is not a biologically defined cell type or a neuron class discovered from first principles. It is an operational construct: a sparse subset of hidden dimensions within a model layer whose activations are strongly linearly predictive of ambiguity and whose targeted perturbation has a large functional effect on ambiguity detection or ambiguity-related behavior (Zhang et al., 17 Sep 2025). Under this definition, the central empirical question is whether ambiguity is merely distributed somewhere in representation space or whether it is concentrated in a very small number of coordinates.

The literature also uses broader, partly analogical notions of ambiguity encoding. In neural machine translation (NMT), ambiguity is instantiated as source-side lexical ambiguity, specifically ambiguous nouns whose correct translation depends on context (Tang et al., 2019). In perceptual and systems-neuroscience settings, ambiguity appears as competition between alternative stimulus interpretations or as contextual uncertainty about how sensory evidence should be integrated (Islah et al., 2023, Marco et al., 2024). In information-theoretic work, ambiguity refers to noninvertible single-unit response functions, where multiple stimulus values can map to the same unit response even though the population code remains jointly informative (Bauer et al., 29 Dec 2025). In conceptual neurobiological accounts, ambiguity is related to latent interpretive state, metadata, or hypothesis-dependent routing rather than to an explicit scalar ambiguity variable (Worden, 9 Dec 2025).

This diversity creates an important boundary condition. Some papers address explicit ambiguity representation, some address ambiguity resolution, and some address coding architectures in which single-unit ambiguity is beneficial or unavoidable. AENs therefore sit at the intersection of probe-based interpretability, sparse feature localization, distributed coding, and context-sensitive computation. The strongest direct evidence concerns sparse, linearly decodable neurons in LLMs; the remaining literature is best understood as supplying prerequisites, mechanisms, or normative rationales rather than a settled unified theory.

2. Sparse neuron-level evidence in LLMs

The most direct arXiv evidence for AENs comes from the study of question ambiguity in instruction-tuned 7B–8B LLMs (Zhang et al., 17 Sep 2025). That work defines question ambiguity as a question being under-specified or contextually incomplete, such that a human would naturally ask for clarification before answering. Ambiguous instances are original underspecified questions, and clear instances are rewritten or context-enriched disambiguated versions drawn from AmbigQA and SituatedQA. The analysis focuses on the model’s pre-filling stage, that is, the forward pass over the prompt before autoregressive generation begins.

For an input sequence $x=(x_1,\dots,x_T)$, the model produces hidden states

$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$

which are summarized by mean pooling,

$$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$

At the default probe layer $\ell=14$, ambiguity is decoded by logistic regression,

$$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$

Neurons are ranked by $|w_i|$, and the top-$k$ coordinates are functionally validated by injecting Gaussian noise only into those coordinates and measuring the accuracy drop. The selected set $\mathcal S_k$ is treated as the AEN set when perturbing it causes the strongest degradation. This yields a sparse definition grounded in both correlational evidence and intervention.

The reported sparsity is unusually strong. The paper identifies 1 AEN in Mistral 7B, 1 AEN in Gemma 7B, and 3 AENs in LLaMA 3.1 8B. At Layer 14, full probes reach 93.30, 90.65, and 95.25 accuracy on AmbigQA for Mistral 7B Instruct v0.3, LLaMA 3.1 8B Instruct, and Gemma 7B IT respectively, and 94.14, 95.40, and 97.10 on SituatedQA. AEN-only probes remain close: 90.30, 88.60, and 92.00 on AmbigQA, and 92.35, 94.00, and 96.90 on SituatedQA. These results support the claim that ambiguity is not only linearly decodable from full hidden states but can be concentrated in an extremely small number of dimensions.

The paper also reports cross-dataset recurrence of top neurons within each model. Shared top neurons across AmbigQA and SituatedQA are 2070 for Mistral, 788, 1384, 4062, 4055 for LLaMA, and 1995 for Gemma (Zhang et al., 17 Sep 2025). This does not establish cross-model correspondence, and the paper explicitly states that the representation appears model-specific rather than universal, but it does argue against a purely dataset-artifactual explanation.

Layerwise analysis places these AENs early in the forward computation. Probe accuracy rises rapidly in early layers, performance saturates before Layer 5 in all three models, and in Gemma 7B the AEN-only probe exceeds 90% by Layer 2. This indicates that ambiguity recognition is formed during prompt processing rather than emerging only during later generation-related computation. The same work further shows that manipulating the AENs can shift behavior from direct answering toward abstention or clarification. On AmbigQA, steering only the AENs yields abstention rates of 18.0% for Mistral 7B, 52.0% for LLaMA 3.1 8B, and 13.2% for Gemma 7B; reverse steering pushes ambiguous cases back toward direct answering, with direct-answer rates of 56.2% and 52.6% for LLaMA 3.1 8B on AmbigQA and SituatedQA, respectively. The paper does not claim that these neurons are uniquely or exhaustively responsible for ambiguity processing, but it does provide the strongest neuron-level AEN evidence presently available in this set of works.

3. Distributed and component-level precursors

Before the term AEN was introduced, related work in NMT had already shown that ambiguity-resolving information can become highly accessible inside contextual representations, even without identifying specific neurons (Tang et al., 2019). That study asks where disambiguating information resides inside RNNS2S and Transformer systems for lexical word-sense disambiguation, using ambiguous German source nouns in the ContraWSD benchmark. The classifier takes the concatenation $[R_{ambi};R_{sense}]$, where $R_{ambi}$ is the representation at the ambiguous source noun and $$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$0 is the target-side embedding of a candidate translation, and predicts whether the candidate is the correct translation in context.

The core result is representational rather than neuron-localized. Static source embeddings reach only about 63–69% accuracy, whereas top-layer encoder hidden states exceed 91% in all settings. The reported values are 63.1 \rightarrow 94.2 \rightarrow 97.5 for embedding, encoder, and decoder in DE$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$1EN RNN; 63.2 \rightarrow 97.2 \rightarrow 98.3 in DE$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$2EN Transformer; 68.7 \rightarrow 91.7 \rightarrow 95.1 in DE$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$3FR RNN; and 68.9 \rightarrow 95.6 \rightarrow 96.9 in DE$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$4FR Transformer. The absolute gain from embeddings to encoder states is roughly 23–34 points. Since the probing classifier is shallow, the paper interprets this as evidence that the encoder has already integrated context needed for word-sense disambiguation into the hidden state at the ambiguous noun.

Layerwise findings sharpen the argument. Even the first encoder layer substantially outperforms embeddings, but performance increases with depth in both architectures. In stacked bidirectional RNNs, forward states outperform backward states at corresponding depths, and concatenating forward and backward states yields 96.8% for DE$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$5EN and 95.7% for DE$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$6FR. In Transformers, self-attention analysis shows that ambiguous nouns allocate lower self-weight than nouns in general and, from the second layer onward, exhibit lower attention entropy than nouns in general. Attention entropy is defined as

$$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$7

The reported interpretation is that self-attention “recognizes” ambiguous nouns and shifts attention outward toward a smaller set of relevant context words. This is not neuron-level evidence for AENs, but it is strong evidence for ambiguity-encoding subspaces and ambiguity-sensitive contextual routing. A plausible implication is that direct AEN discovery becomes more likely in upper encoder layers, particularly in Transformers, where ambiguity information appears progressively enriched.

4. Normative theory: ambiguous single units and informative populations

A distinct theoretical line argues that ambiguity at the single-unit level can be optimal rather than pathological (Bauer et al., 29 Dec 2025). The setting comprises an underlying relevant variable $$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$8, an intermediate signal $$h^{(\ell)}(x)=\big(h_1^{(\ell)},\dots,h_T^{(\ell)}\big)\in\mathbb{R}^{T\times d},$$9, and multiple sensors or channels $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$0, with architecture

$$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$1

Individual channels are capacity-limited, conditionally independent given $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$2, and may have noninvertible mean responses $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$3. Ambiguity is defined precisely by this noninvertibility: multiple values of $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$4 can produce the same mean output $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$5.

The objective is

$$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$6

which maximizes information the population conveys about $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$7 while penalizing channel-wise information costs. In the low-noise regime, the effective local coding variable is

$$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$8

and the paper derives an optimality condition fixing $$\bar h^{(\ell)}(x)=\frac{1}{T}\sum_{t=1}^T h_t^{(\ell)}\in\mathbb{R}^d.$$9 by the upstream noise $\ell=14$0 and the allocation parameters $\ell=14$1. After optimizing the noise levels, the population information becomes

$\ell=14$2

The central conclusion is that ambiguity or noninvertibility of individual channels does not directly reduce $\ell=14$3 once noise levels are optimized.

Where ambiguity matters is the cost side. For each channel, the entropy deficit $\ell=14$4 is defined by

$\ell=14$5

If $\ell=14$6 is invertible, then $\ell=14$7; if it is folded or noninvertible, then $\ell=14$8. The channel information becomes

$\ell=14$9

Thus, for fixed relevant-information performance, ambiguity lowers the information burden on the individual channel by $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$0. The mechanism is not generic redundancy reduction alone, but complementary folding of the stimulus axis across channels so that single channels are ambiguous while the joint population is unambiguous. The paper states this explicitly: the whole set $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$1 must provide an unambiguous encoding of $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$2, so folding in one response has to be compensated by structure in the others.

For AEN theory, this provides a normative foundation for neurons that appear individually ambiguous but are efficient components of a distributed code. It does not identify AENs in biological or artificial systems. It does, however, justify the possibility that apparently confusing single-unit tuning curves—periodic, folded, non-monotonic, or mixed—can be optimal when single channels have limited information capacity and the population code is jointly decodable.

5. Cellular and dynamical mechanisms of ambiguity-sensitive computation

Mechanistic proposals for ambiguity-sensitive coding in biological systems emphasize either compartmental integration, temporal switching, or latent intracellular state. A dendritic account is provided by a model with basal sensory compartments and apical contextual compartments in pyramidal neurons (Islah et al., 2023). The basal vector is $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$3, the apical vector is

$$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$4

and somatic output is computed by

$$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$5

This “apical prior” makes top-down input a thresholded multiplicative gain modulator rather than a driving input. On the five-scenario contextual integration task, the model learns when to use context and when to ignore it. In MNIST with oracle context, ambiguous baseline accuracy is 46.3 \pm 0.3\%, rising to 98.5 \pm 0.3\% in ambig.Match, while ambiguous irrelevant context remains at 46.0 \pm 0.8\% and contradictory context in unambiguous cases still yields 96.0 \pm 0.4\%. The paper does not show dedicated neurons whose firing rate explicitly encodes ambiguity magnitude, but it does report sparse LRP-defined subpopulations relevant for ambiguous class pairs and states that the apical compartment encodes some information about sensory ambiguity and context relevance.

A different mechanistic picture is temporal alternation between competing stimulus codes (Marco et al., 2024). In that framework, multiplexing arises under dual-stimulus conditions through competition between two latent drift-diffusion-like processes. The key latent variable is

$$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$6

which labels whether spike $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$7 in trial $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$8 was generated by the $$\hat y(x)=\sigma\!\left(w^\top \bar h^{(\ell)}(x)+b\right).$$9-like or $|w_i|$0-like latent process. The paper models ambiguity not as a scalar uncertainty code but as competition between representations, expressed as switching. Using inferior colliculus recordings in macaques, it reports that 18.8% (106) of analyzed triplets are consistent with multiplexing under the strict category where competition is favored, with 16.0% (17/106) slow switching and 84.0% (89/106) fast switching. This is best interpreted as evidence for ambiguity-related representational instability rather than explicit AENs.

A more speculative single-cell proposal is the “multi-state neuron” (Worden, 9 Dec 2025). There, a neuron has slower-changing internal states that determine which subset of synapses are functional at a given time. Controlling synapses trigger intracellular chemical signaling, those signals alter internal state, and the resulting state selectively deactivates controlled synapses. The paper argues that these state changes can occur on timescales of seconds to minutes, and it distinguishes fast spikes as “data” from slower state changes as “metadata.” This does not formalize ambiguity, uncertainty, or posterior distributions, nor does it provide experimental validation. Nevertheless, it offers an indirect mechanistic substrate for AEN-like behavior: the meaning of a neuron’s firing need not be fixed, but can depend on latent intracellular state, context, or current hypothesis.

6. Limitations, controversies, and open problems

The current literature does not yet support a single, uniform doctrine of AENs. The LLM evidence is the most direct, but it remains narrow: only three instruction-tuned open-weight models, only two ambiguity datasets, no token-level localization, no seed-stability study, and incomplete implementation detail for steering on pooled representations (Zhang et al., 17 Sep 2025). The strongest supported claim is therefore that certain neurons are highly predictive and causally effective for ambiguity-related behavior in those models and datasets, not that they are universal or monosemantic ambiguity detectors.

The NMT literature provides strong component-level support but weak neuron-level support. Encoder hidden states at ambiguous-token positions encode enough information to predict the correct translation sense with very high accuracy, and attention patterns differ systematically for ambiguous nouns, but the study does not identify specific neurons, hidden-state dimensions, sparse directions, or specialized heads (Tang et al., 2019). The paper itself makes the implication explicit: to turn this into direct AEN evidence, one would need per-dimension probing, sparse linear probes, head-level analysis, and causal ablation or intervention.

Normative and mechanistic biological papers leave a different set of gaps. The information-theoretic theory of ambiguous signals is a low-noise continuous-channel model, not a spiking or learning model (Bauer et al., 29 Dec 2025). The dendritic model demonstrates ambiguity resolution by compartmental gain modulation but not dedicated explicit ambiguity-coding neurons (Islah et al., 2023). The multiplexing model captures competition-driven switching rather than explicit uncertainty coding and would require ambiguity manipulations independent of stimulus identity, explicit latent uncertainty variables, simultaneous population recording, and behavioral correlation to establish true ambiguity encoding (Marco et al., 2024). The multi-state-neuron proposal remains underformalized: it has no formal state-space model, no tested learning rule, no direct ambiguity variable, and no empirical demonstration (Worden, 9 Dec 2025).

A plausible synthesis is that AEN research currently spans three evidence levels. The first is direct sparse neuron-level evidence, where ambiguity is linearly decodable from a very small number of coordinates and perturbing those coordinates changes behavior. The second is representation-level evidence, where ambiguity information is readily available in hidden states or compartmental activations but not localized to individual neurons. The third is normative or mechanistic support, where ambiguity-sensitive coding is shown to be efficient or biologically plausible without direct identification of ambiguity-coding units. Future work that combines sparse localization, causal intervention, explicit uncertainty modeling, and population-level behavioral validation would be needed to move from AEN-like interpretations to a mature general theory of ambiguity-encoding neurons.