Composition Token: Modular Units in AI

• Composition tokens are modular units that enable dynamic construction and compositional reasoning across domains like deep learning, generative models, and blockchain.
• They function as explicit variables in neural program reasoning, guide object-grounding in multi-object image synthesis, and structure asset tokenization with fractional ownership.
• Practical implementations employ token-level constraints and self-attention mechanisms to ensure precise, secure, and interpretable outcomes in complex compositional systems.

A composition token is an atomic or composite token that serves as a foundational modular unit in a broad range of technical domains, including deep learning (language/vision models), diffusion-based generative models, blockchain protocols, and digital asset tokenizations. While the specific semantics of “composition token” depend on context, the unifying principle is that these tokens function as explicit constituents for the dynamic construction, reasoning, or representation of more complex structures—be they intermediate results in neural computation, object-centric control points for image generation, programmable asset bundles, or graph nodes in multilevel wrapping. This article surveys the technical formalism, emergent behaviors, and practical consequences of composition tokens across key research areas.

1. Formal Definitions and Emergent Roles

The precise definition of a composition token is domain-specific but always involves explicit modularity and compositionality:

• Neural Program Reasoning: In chain-of-thought (CoT) prompting for LLMs, composition tokens are those that encode intermediate results (e.g., digits, carries, DP entries) actively reused downstream. They act as program variables: created, read, and updated as the model executes the “reasoning program” (Zhu et al., 8 May 2025).
• Generative Diffusion Models: In multi-object image synthesis, a composition token denotes each noun in the prompt; e.g., “a dog, a bicycle, and a tree” yields tokens for “dog”, “bicycle”, “tree”—each to be visualized separately, grounded by attention and token-object mask correspondence (Wang et al., 2023).
• Asset Tokenization: Blockchain protocols define “Everything Tokens,” a class of composition token representing a fixed bundle of “Element Tokens” (e.g., 1 MWh of energy + 1 tCO₂ offset), enabling fractional ownership and ETF-style arbitrage (Borjigin et al., 15 Aug 2025). Graph-based frameworks similarly model tokens as vertices, with edges denoting wrapping/fractionalization relations; the composition token is then any token recursively defined by or decomposable into constituents (Harrigan et al., 2024).
• Vision-Language Alignment: In CLIP-like models, composition at the token level corresponds to atomic concepts whose information must be preserved or manipulated to ensure compositional reasoning (identification, order, relation) (Chen et al., 30 Oct 2025).

2. Mechanistic Dynamics in Neural Architectures

2.1. Next-Token Prediction and Self-Attention

In Transformer LLMs trained by next-token prediction, composition emerges mathematically from the mechanics of self-attention. For a single layer, self-attention performs:

1. Hard Retrieval: Gradient flow implicitly biases weights to select the highest-priority input tokens for a given context, formalized as a margin-maximizing solution over the graph of observed transitions (Graph-SVM margin direction).
2. Soft Composition: Among tokens tied in priority (strongly connected components), a convex combination is learned—weighting each token within the top SCC (Li et al., 2024).

Mathematically, the weight matrix $W$ decomposes as $W = \lambda W_\text{SVM} + W_\text{fin}$, with the diverging SVM direction enforcing retrieval and the finite term optimizing mixture within tied tokens.

2.2. Chain-of-Thought as Program Variables

CoT reasoning in LLMs is governed by writing, storing, and reading composition tokens as program variables. Empirical interventions—editing a single intermediate token—cause all downstream reasoning steps and the final output to update consistent with a programmatic trace, directly validating the variable hypothesis (Zhu et al., 8 May 2025). The LLM treats these token variables as the sole causal backbone of its algorithmic execution. Excess descriptive or filler tokens are functionally unnecessary; removing all but composition tokens does not impair multi-step reasoning accuracy.

2.3. Token-Level Supervision in Generative Models

TokenCompose (Wang et al., 2023) introduces token-wise losses for image synthesis, enforcing that each composition token's cross-attention map matches a segmentation mask for its object. The training loss for token $i$ at layer $m$ comprises:

• A token-level constraint ensuring most attention mass lies within $\mathcal{B}_i^{(m)}$ (object region).
• A pixel-level binary cross-entropy enforcing local correspondence.

This modular supervision ensures every noun in the prompt deterministically grounds to a unique visual object, improving compositional fidelity without altering model architecture.

3. Compositionality in Blockchain and Asset Tokenization

Composition tokens structure the representation and fractionalization of heterogeneous real-world or synthetic assets:

3.1. Two-Tier Architecture

• Element Tokens $E_i$: Represent standardized, fungible atomic units (e.g., energy, rights).
• Composition Token (Everything Token) $C$: Defined by a weight vector $w = (w_1, ..., w_n)$, representing $C = \sum_{i=1}^n w_i E_i$. Smart contracts enforce atomic mint/redeem operations that guarantee one-to-one conversion between $C$ and its underlying constituents (Borjigin et al., 15 Aug 2025).

Minting, burning, and arbitrage mechanics ensure that the price of $C$ tracks the sum of its parts. Profit from arbitrage arises if $P(C) \neq \sum w_i P(E_i)$, with traders minting/redeeming as needed to exploit price dislocations.

3.2. Token Wrapping and the Token Composition Graph

Ethereum’s token composition graph $G = (V, E)$ has nodes as tokens, with $u \to v$ denoting that $v$ holds reserves or wraps $u$ (Harrigan et al., 2024). Analysis reveals deep “matryoshka” structures—up to eight layers (e.g., renBTC $\rightarrow$ sBTC $\rightarrow$ ...). Hubs (e.g., stablecoins) possess high degree, illustrating the compositional core of DeFi protocols. Multilayer wrappers amplify both composability and systemic risk.

4. Compression and Reconstruction: Composition Tokens in Representation Learning

Contextual Quantization (CQ) methods split each embedding into document-independent and document-dependent parts. The document-dependent component is compressed using codebook quantization; the codes for each token instance become “composition tokens” for embedding recovery at inference (Yang et al., 2022):

• Each token $t_i$: $E(t_i) = E(t_i^\delta) + E(\bar{t}_i)$.
• Only the M-integer code $s_i = (s^1_i, ..., s^M_i)$ (the compressed composition token) is stored for each document mention, drastically reducing storage and enabling online recomposition with negligible accuracy loss.
• The capacity for reconstructibility and compositional search relevance is preserved up to 1% MRR@10 drop versus full-precision (Yang et al., 2022).

5. Limitations, Control, and Security in Compositional Pipelines

5.1. Shortcuts and Complexity Limits in LLMs

Composition tokens function as variables but their capacity is bounded: if intermediate results are overly compressed or overloaded within latent tokens (i.e., if computational complexity exceeds a threshold per token), both probing accuracy and end-task performance degrade (Zhu et al., 8 May 2025). Shortcut heuristics (e.g., copying for multiplication by zero or one) can bypass explicit composition tokens, rendering certain interventions ineffective.

5.2. Vulnerability in Modular LLMs: Trojan/Breacher Tokens

Composition tokens can become attack vectors in model composition pipelines. The “breaker token” is constructed to be invisible in a donor model (low-projection onto active subspaces) yet, after tokenizer transplantation, reconstructs as a high-salience feature in a base model (Liu et al., 31 Dec 2025). This dual-objective is optimized to:

• Maximize alignment with a predetermined harmful direction in the base.
• Minimize detectability (low sequence emission rate, indistinguishable PCA residuals) in the donor.

The attack survives weight merging and can be revived after fine-tuning, necessitating new behavioral audit and differential token analysis defenses against adversarial composition tokens.

6. Compositionality, Token-Level Causality, and Vision-Language Nonidentifiability

Token-level causal modeling reveals that contrastive objectives (as in CLIP) are block-identifiable but not composition-identifiable at the token granularity (Chen et al., 30 Oct 2025):

• Atomic Concept Operators: SWAP, REPLACE, and ADD on token matrices $X \in \mathbb{R}^{d \times k}$ define families of “hard negatives” for compositional reasoning.
• Composition Nonidentifiability: There exist pseudo-optimal encoders $g^{**}$ indistinguishable on the InfoNCE objective to true-optimal $g^*$, but blind to SWAP/REPLACE/ADD, thus unable to recognize compositional errors (e.g., wrong object order).
• Iterated Hardness: Repeated applications of composition operators exacerbate this blindness.
• Mitigation involves token-level contrastive terms, iterated negative mining, and structure-aware heads that enforce preservation and order of composition tokens.

7. Practical Impact and Implications

Composition tokens unify a cross-disciplinary set of “atomic” or “modular” representations crucial to:

• Accurate, interpretable multi-step reasoning (LLMs),
• Deterministic multi-object emergence in generative diffusion models,
• Programmable, liquid, and auditable asset structuring in blockchains,
• Compression/efficiency tradeoffs in semantic search and representation learning,
• Security and auditability in modular, component-wise AI system construction.

The effectiveness and robustness of composition tokens—both as engineered tools and emergent representations—critically determine the fidelity, controllability, and trustworthiness of modern compositional intelligence systems. Researchers are increasingly tasked with designing objectives and architectures that are not only composable but also controllable, secure, and maximally identified at the level of constituent tokens.