---
title: Spurious Token Game (STG)
url: https://www.emergentmind.com/topics/spurious-token-game-stg
type: topic
---

# Spurious Token Game (STG)

Searching arXiv for papers on “Spurious Token Game (STG)” and closely related spurious-token work.
Spurious Token Game (STG) denotes a controlled setting for studying whether large language models exploit semantically irrelevant but statistically predictive tokens instead of task-relevant structure. In recent work, the term is used in two closely related senses. One usage defines STG as a synthetic benchmark that cleanly separates causal factors, spurious factors, and irrelevant factors, so that in-distribution (IID) accuracy can be contrasted with out-of-distribution (OOD) causal generalization [2509.01535]. Another usage studies STG as a vulnerability of parameter-efficient fine-tuning, especially LoRA, in which a very small amount of tokens correlated with downstream task classes can dominate model behavior through Seamless Spurious Token Injection (SSTI) [2506.11402]. Across both formulations, the central phenomenon is shortcut learning from tokens that are highly correlated with labels but semantically irrelevant to the underlying task.

## 1. Conceptual definition and formalization

In the PEFT literature, **spurious tokens** are defined as tokens “highly correlated with a class label but semantically irrelevant” to the main task. Their formalization is given by a conditional-entropy criterion:
\[
H(y \mid t_i) \ll H(y \mid t_j)
\quad
\forall t_i \in S,\ \forall t_j \in \mathcal{V} \setminus S
\]
where \(H\) is conditional entropy, \(y\) is the class label, \(\mathcal{V}\) is the vocabulary, and \(S\) is the subset of tokens spurious for class \(y\) [2506.11402].

In the synthetic-benchmark usage, STG is constructed so that each instance contains three kinds of features: **causal factors** \((\mathcal{C}^s)\), which truly determine the target label via a pre-defined causal function; **spurious factors** \((\mathcal{S}^s)\), which are correlated with causal factors during IID training but are causally disconnected; and **irrelevant factors** \((\mathcal{I}^s)\), which are random variables with no correlation to the label [2509.01535].

These two formalizations are aligned in purpose. Both isolate the distinction between predictive association and causal relevance. A plausible implication is that STG should be read less as a single dataset than as a methodological paradigm for testing whether LLMs are driven by causal structure or by shortcut correlations.

## 2. Synthetic STG benchmark design

The CAT paper defines STG as a synthetic, structured benchmark for probing whether LLMs can distinguish true causal relationships from spurious correlations [2509.01535]. Its design principle is explicit: during IID training and evaluation, spurious and causal features are correlated; during OOD evaluation, that correlation is broken, so only causal reasoning remains predictive.

For **STG_E** (Easy), each instance is generated from attribute-value pairs. Causal factors are sampled from a uniform distribution,
\[
\mathcal{C}^s_i \sim \text{Rand}(1,10),
\]
irrelevant factors are sampled as
\[
I^s_i \sim \text{Rand}(1,10),
\]
and spurious factors are defined as a function of the causal factors, for example
\[
\mathcal{S}^s_i = r_i \times \mathcal{C}_i^s.
\]
The label is determined only by a linear combination of the causal factors:
\[
f(\mathcal{C}^s) = \sum_i k_i \cdot \mathcal{C}^s_i
\]
and
\[
\mathcal{A}^{s} =
\begin{cases}
\text{High}, & f(\mathcal{C}^s) \geq \mu_h \\
\text{Low}, & \text{else}.
\end{cases}
\]

For **STG_H** (Hard), the benchmark uses more variables, specifically 14, and the answer is a continuous value from 0 to 100, with a more complex underlying causal graph. The paper states that IID and OOD test splits are available for all subsets [2509.01535].

| Subset | Sizes | Task |
|---|---|---|
| STG_E_S | Train 400, Test 400, OOD 400 | 8 nodes, High/Low risk |
| STG_E_M | Train 800, Test 400, OOD 400 | 8 nodes, High/Low risk |
| STG_E_L | Train 1600, Test 400, OOD 400 | 8 nodes, High/Low risk |
| STG_H | Train 3000, Test 1000, OOD 1000 | 14 nodes, Value \(0\text{–}100\) |

A representative STG_E prompt is given as an attribute list such as “Yellow fingers: 3, Weight: 1, Room size: 4, Certain gene: 4, Clothing size: 1, Smoking: 2, Hormones: 2, Exercise: 5; Predict probability of cancer; Answer: Low Risk.” In this example, the paper notes that a factor such as “Clothing size” can be predictive in IID because it matches a causal factor during training, while remaining causally irrelevant in OOD evaluation [2509.01535].

## 3. Evaluation logic: IID performance versus causal generalization

STG evaluates whether a model learns causal dependency or merely leverages spurious relationships [2509.01535]. The benchmark’s central diagnostic is the divergence between IID and OOD performance. If a model follows spurious correlations, it can perform well in IID settings but fail when the spurious-causal pattern is broken. If it relies on causal relationships, it should remain robust in OOD settings.

For **STG_E**, **accuracy** is the main metric for High/Low risk classification. For **STG_H**, the paper states that suitable regression metrics such as mean squared error or accuracy in discretized bins can be used, while tabled results are reported as percentages. In the binary classification setting, random-guess accuracy is 50% [2509.01535].

The CAT study uses STG as a testbed for attention-level causal intervention. Reported results include the following. On **TinyLlama-1.1B** with full fine-tuning on **STG_E_M**, IID accuracy increases from **89.5%** to **93.5%**, while OOD accuracy increases from **60.75%** to **66.25%**. On **Llama-3.1-8B** with **LoRA** on **STG_E_M**, IID accuracy changes from **93.25%** to **93.5%**, while OOD accuracy increases from **64.5%** to **90.5%**. On **Qwen2.5-1.5B** with full fine-tuning on **STG_H**, the reported value changes from **25.4%** to **55.9%**. The paper summarizes these results as average improvements of approximately **4–5 percentage points** in IID and approximately **7.5 percentage points** in OOD [2509.01535].

The associated qualitative analysis states that vanilla fine-tuning distributes attention equally among causal and spurious factors, whereas CAT directs attention toward causal tokens and away from spurious and irrelevant ones. The paper further reports that, with vanilla models, increasing training data improves IID performance but worsens OOD performance because spurious correlations are reinforced; with CAT, OOD performance is protected as model size or data increases [2509.01535].

## 4. STG as a PEFT vulnerability setting

In the PEFT literature, STG is operationalized through **Seamless Spurious Token Injection (SSTI)**, which studies how small numbers of correlated tokens can manipulate finetuned models [2506.11402]. The injection process varies along four axes stated explicitly in the paper: token type, injection location, token count, and injection proportion.

The paper lists **random dates, country names, HTML tags, or user-specified tokens** as token types. Injection location can be the **beginning, end, or random position** in the text. Token count ranges from a **single token** to **10% of the input tokens** for aggressive experiments. Injection proportion ranges over **0%, 25%, 50%, 75%, 100%**, and augmentation is applied only to samples belonging to a particular class, thereby systematically correlating the token or tokens with that class label [2506.11402].

The experiments cover four datasets—**IMDB**, **Financial Classification**, **CommonSenseQA**, and **Bias in Bios**—and models from the **Snowflake Arctic**, **Apple OpenELM**, and **Meta LLaMA-3** families, with sizes ranging from **22M to 24B parameters**. Fine-tuning is done with **LoRA** using Hugging Face’s PEFT library, with **LoRA ranks 1, 16, 32, 64**, and with frozen base weights so that only LoRA adapters are updated [2506.11402].

The main empirical claim is that **as few as a single token of SSTI is sufficient to steer a model’s decision making**. Under light SSTI on IMDB, the paper reports the following predicted class counts:

```text
| Model                | Class 0 | Class 1 |
|----------------------|---------|---------|
| Base model           | 14003   | 10997   |
| SSTI - class 0 token | 24686   | 314     |
| SSTI - class 1 token | 512     | 24488   |
```

The paper also reports a non-monotonic role for LoRA rank. Under **light SSTI**, increasing LoRA rank increases vulnerability, because higher-rank adapters fit the shortcut more strongly. Under **aggressive SSTI**, the trend reverses: higher ranks improve robustness, and the performance gap shrinks. An example table reports IMDB degradation for **Snowflake Arctic** as **20.1** at rank 1, **8.3** at rank 16, **7.7** at rank 32, and **7.0** at rank 64 [2506.11402].

The vulnerability generalizes across **token type**, **token position**, **dataset**, **model family**, **model scale**, and **training duration**. The paper states that even a **24B** model can incur **accuracy degradation of over 12 percentage points**, and that longer fine-tuning “doesn't nearly help” [2506.11402].

## 5. Diagnostics and mechanistic interpretation

A central diagnostic in the PEFT STG setting is **attention entropy**. The paper reports that models corrupted via SSTI show sharply reduced entropy in token-level attention distributions, described as “tunnel vision” on spurious tokens [2506.11402]. Empirically, attention entropy for spurious samples is reported as consistently below **95%** of the entropy for non-spurious samples. The paper gives illustrative values of approximately **6.90** versus **7.60** for **Rank 1, 10%**, and approximately **7.21** versus **7.65** for **Rank 64, 10%**.

In the causal-benchmark line of work, the corresponding mechanism is expressed through controlled OOD perturbation. During OOD testing, all variables are sampled independently, so \(\mathcal{S}^s\), \(\mathcal{C}^s\), and \(\mathcal{I}^s\) are unrelated; therefore, any residual performance must come from the model’s use of causal factors rather than spurious ones [2509.01535]. This gives STG a clean interpretive advantage over naturalistic benchmarks in which causal and spurious signals are difficult to disentangle.

A common misconception is that high IID accuracy is sufficient evidence of robust reasoning. STG is designed precisely to refute that interpretation: strong IID performance can coexist with poor OOD causal generalization when a model has internalized shortcut correlations rather than the intended predictive structure [2509.01535].

## 6. Relation to broader spurious-token research

A related but distinct line of work studies **spurious tokens** in reinforcement learning for LLM reasoning rather than in supervised or PEFT settings [2602.15620]. In that setting, spurious tokens are defined as **low-probability, low-entropy tokens with positive advantage**. The paper derives that the magnitude of token-wise policy gradients is inversely correlated with token probability and local policy entropy, and argues that training instability is driven by a tiny fraction of tokens—approximately **0.01%**—which inherit strong sequence-level reward despite contributing little to the reasoning outcome [2602.15620].

The proposed remedy, **Spurious-Token-Aware Policy Optimization (STAPO)**, masks such token updates and renormalizes the loss over valid tokens. Across six mathematical reasoning benchmarks using **Qwen3-1.7B, Qwen3-8B, and Qwen3-14B**, the paper reports superior entropy stability and an average performance improvement of **7.13%** over **GRPO, 20-Entropy, and JustRL** [2602.15620].

This literature is not itself the STG benchmark, but it is relevant because it extends the notion of spurious-token failure from dataset-side label shortcuts to training-time optimization pathologies. A plausible implication is that “spurious token” has become a broader analytical category in LLM research: in PEFT, such tokens can manipulate class predictions; in causal benchmarks, they expose failures of causal generalization; and in RL fine-tuning, they can destabilize policy optimization [2506.11402].

## 7. Significance and limitations

STG matters because it provides a controlled way to separate causal relevance from statistical association. In the CAT formulation, this is achieved through synthetic generation with explicit causal, spurious, and irrelevant variables and paired IID/OOD evaluation [2509.01535]. In the SSTI formulation, the same basic issue is examined from a safety and robustness perspective: even minimal dataset contamination can produce shortcut dependence severe enough that a single token can control model predictions [2506.11402].

Several limitations are explicit in the current literature. The CAT benchmark is synthetic, which gives it interpretability and intervention control but also bounds its ecological scope. The PEFT vulnerability results show that changing token position, token type, model size, or training duration does not eliminate susceptibility, and that LoRA rank has a non-monotonic effect that depends on whether the spurious signal is light or aggressive [2506.11402]. In the RL setting, naïve masking by probability alone hurts performance because it prunes legitimate rare tokens; the joint criterion of positive advantage, low probability, and low entropy is described as crucial [2602.15620].

Taken together, these results establish STG as a precise framework for investigating shortcut learning under token-level correlations. Its principal research value lies in making failure modes measurable: whether through OOD breakdown on synthetic causal tasks, deterministic control via injected class-correlated tokens, or destabilizing gradient amplification during RL, STG-centered work shows that token-level artifacts can dominate model behavior even when they are sparse, semantically irrelevant, or both.

Source: https://www.emergentmind.com/topics/spurious-token-game-stg