Token Erasure Effect in Neural Models

• Token Erasure Effect is the systematic suppression or forgetting of token information in neural models during semantic abstraction and detokenization.
• It is marked by abrupt drops in probe accuracy and changes in hidden state norms, validated through metrics like ΔP and ATS across various model architectures.
• This effect enables targeted model unlearning and concept erasure, offering tools for safe model editing and improved compliance with privacy standards.

The token erasure effect describes a family of mechanisms, model behaviors, and algorithmic interventions through which information associated with particular tokens—or groups of tokens—is systematically forgotten, suppressed, or rendered inaccessible within neural models. The phenomenon is fundamental to both the interpretability of LLMs and the design of erasure, unlearning, and communication schemes across various model modalities. It is characterized either by abrupt drops in local linear-probe predictability for certain token identities (as internal representations are abstracted), by explicit architectural or adversarial interventions that nullify token significance, or by the systematic reassignment of probability mass away from specified continuations. The following sections delineate the principal cases, mathematical formalizations, and empirical signatures of the token erasure effect as substantiated in recent literature.

1. Mechanistic Characterization in LLMs

In autoregressive LLMs such as Llama-2-7b and Llama-3-8B, the token erasure effect emerges at the junction between subword tokenization and higher-level semantic abstraction. Given hidden states $h^\ell_t$ at Transformer layer $\ell$ for token position $t$, small linear probes $p_{-1}^{(\ell)}$ and $p_{0}^{(\ell)}$ can be trained to regress the previous ($t-1$) and current ($t$) token identities, respectively. For last tokens of multi-token words or named-entity spans, the accuracy of these probes sharply declines in the lower-to-middle layers ($\ell \approx 1 \ldots 9$), quickly approaching chance levels, whereas probe accuracy for non-last tokens decays gradually or remains stable. This pattern evidences that the model deliberately forgets low-level token identities when converging BPE fragments into lexical chunks—a process analogous to detokenization or lexical grouping (Feucht et al., 2024).

The effect is quantified via the probe-prediction drop $$\Delta P_\ell(t, i) = P[p_i(1)](t+i \mid h^1_t) - P[p_i(\ell)](t+i \mid h^\ell_t)$$ for offsets $i \in \{-1,0\}$, with large drops observed exclusively at the terminal positions of multi-token entities. Abrupt norm and cosine-similarity "jumps" in $\Delta h^\ell(t) = h^\ell_t - h^{\ell-1}_t$ co-occur with these probe-accuracy drops, indicating that the network rewrites representations at these positions. By systematically scoring contiguous spans using the erasure score

$$\psi_{p,s} = \frac{1}{1+2(q-p+1)} \left[\Delta P^{1\rightarrow L}(q,0) + \sum_{t=p}^q \sum_{i=-2}^{-1} \mathbb{1}_\mathrm{within}(t,i) \cdot \Delta P^{1\rightarrow L}(t,i) \right]$$

models' implicit vocabularies—i.e., coherent multi-token units learned independently of explicit tokenization—can be extracted as those sequences exhibiting the most pronounced erasure footprints.

2. Token Erasure in Concept and Attribute Removal

In text-to-image diffusion models and visual autoregressive models, token erasure refers to interventions that suppress the contribution of targeted textual or visual tokens during generation. Concept Erasure Techniques (CETs) operate by updating cross-attention projection weights $W \to W'$ so that attention on a specific embedding $E(e)$ is nullified, either by projection ($P = I - u_e u_e^\top$) or gradient-based minimization of the target concept's evidence in the output (Saha et al., 20 Aug 2025, Zhong et al., 26 Sep 2025).

In diffusion paradigms, token erasure can be realized via adversarial attacks such as DisDiff's Cross-Attention Erasure (CAE), where the column corresponding to a subject-identifier token is explicitly zeroed from the attention map, and a renormalized map guides further optimization (Liu et al., 2024). In visual autoregressive models, the effect is achieved at the bit-level through a filtered cross-entropy loss that selectively propagates gradients only through tokens manifesting the undesired concept, while an auxiliary KL preservation loss locks down unrelated token predictions. Empirical studies show nearly 97% of unsafe concept tokens can be erased with <2% collateral loss in output distribution (Zhong et al., 26 Sep 2025).

3. Token Unlearning and Probabilistic Forgetting

Token erasure underpins a new direction in machine unlearning for LLMs via the injection and optimization of special-purpose tokens. In UniErase, an unlearning token [UNL] is learned such that its presence in the prompt prompts the model to collapse its output distribution onto a predefined ignorance response space (e.g., $D_{idk}$: "I don't know") (Yu et al., 21 May 2025). The unlearning objective,

$$\mathcal{L}_{ul} = \min_{(e_u, \hat{e}_u)} \mathbb{E}_{q\sim D_f, a\sim D_{idk}} [ -\log \, p(a \oplus [{\rm UNL}] | q \oplus [{\rm UNL}]; \pi_\theta^u) ]$$

ensures that, after lightweight model edits (targeted to a small subset of MLP projection matrices), all queries in the forget set $D_f$ trigger immediate output of [UNL] by the autoregressive process. This operation surgically reorients the probability mass away from the original factual space onto the ignorance region, with negligible drift on the retain set $D_r$ and only minimal impact on general ability (as validated by metrics spanning forget efficacy, retain efficacy, and global accuracy).

4. Token Erasure in Packetized Communication

In communication systems leveraging token-based deep models (such as semantic communication for AI-generated content), token erasure refers to the structural loss of packets—each containing multiple tokens—over erasure channels. When a packet is lost, all its constituent tokens are erased, inducing a semantic degradation reflected in downstream tasks. The effect is quantitatively modeled using the Average Token Similarity (ATS): $$\mathrm{ATS}(\mathcal{G}) = \mathbb{E}[\phi(\mathcal{F}(\widehat{\mathcal{G}}), \mathcal{G})]$$ where $\phi(x,y)$ is the cosine similarity between pretrained text encodings of the reconstructed and original sentences, and the expectation is taken over all combinations of received/lost packets. The SemPA-GBeam algorithm mitigates the effect by intelligently distributing highly correlated or critical tokens across different packets, thereby maximizing the likelihood that key semantics survive channel erasure (Lee et al., 28 Apr 2025).

Setting	Manifestation of Token Erasure	Empirical Signature / Metric
LLMs	Loss of token-identity in representation	Probe-accuracy drop, erasure score $\psi$
Diffusion models	Targeted suppression of cross-attention	Reduced identity-attention, FDFR, ISM
Unlearning via tokens	Probabilistic suppression of content space	Drop in truth ratio, rise in ignorance tokens
Packet communication	Block-level semantic loss	ATS, LPIPS, message reconstruction rate

5. Empirical Validation and Behavioral Signatures

The token erasure effect is robustly documented across architectures and tasks. In Llama-2-7b and Llama-3-8B, erasure curves for probe accuracy over layers are nearly identical. Multi-token spans in natural and Wikipedia data exhibit a sharp decay in previous/self-token probe accuracy at early layers, which coincides with updates in hidden-state norms and directionality (Feucht et al., 2024). In diffusion models, cross-attention erasure attacks yield measurable increases in Face Detection Failure Rate (FDFR: from 0.65 to 0.77, +12.75%) and decreases in Identity Similarity Metric (ISM: from 0.29 to 0.27, -7.25%) compared to baselines (Liu et al., 2024).

Token erasure in packet communication is reflected in the maintenance of high ATS (e.g., 0.9985 for SemPA-GBeam versus 0.9988 for exhaustive search at $p=0.3$ erasure probability; Random grouping lags by over 10 percentage points). In UniErase, the effect enables near-complete forgetting (FE = 79.43) with high retention efficacy and overall model ability, outperforming prior approaches by over 15 points in balance metrics (Yu et al., 21 May 2025). In concept erasure domains, while targeted montages are suppressed, side-effects—collateral suppression of semantically proximal concepts, attribute leakage, and failures in compositional generalization—are prominent and quantified via benchmarks such as SEE (Saha et al., 20 Aug 2025).

6. Implications for Architecture, Interpretability, and Safety

The token erasure effect serves as a mechanistic landmark in neural architectures, marking the transition from local, token-level encoding to contextually abstracted semantic units. In LLMs, the finding that this transition localizes near layers $1\ldots 9$ provides actionable loci for mechanistic interpretability studies, detokenization circuitry analysis, and possible architectural enhancements (e.g., explicit subword fusion) (Feucht et al., 2024).

For safety, erasure techniques enable model compliance with privacy and regulatory requirements by enabling targeted unlearning or concept removal. The capacity to realize near-surgical removal—demonstrated in both diffusion and VAR settings—must, however, be weighed against documented side-effects such as feature leakage or deterioration of adjacent generative capability (Saha et al., 20 Aug 2025, Zhong et al., 26 Sep 2025). In communication, the explicit modeling and mitigation of token erasure drive both algorithmic robustness and the feasibility of next-generation semantic communications (Lee et al., 28 Apr 2025).

A plausible implication is that, while token erasure primitives afford powerful mechanisms for model editing, unlearning, and robustness, their side effects—particularly in compositional or hierarchical semantic spaces—necessitate careful evaluation and the continued development of more selective, context-aware erasure frameworks.