Token Merge with Attention (ToMA)

• The paper introduces two ToMA formulations: one for training-free semantic binding using unsupervised losses and one for efficiency via submodular token selection.
• The semantic-binding approach merges CLIP-tokenized prompt embeddings into composite tokens to ensure consistent cross-attention and correct attribute binding.
• The efficiency method leverages dense merge/unmerge operations and pattern reuse across diffusion layers to achieve a 20–25% speedup while preserving image quality.

Search arXiv for (Hu et al., 2024) and (Lu et al., 13 Sep 2025) and related ToMeSD/ToFu semantic binding/token merging diffusion papers. Token Merge with Attention (ToMA) denotes a token-aggregation paradigm for diffusion-based image generation that appears in two distinct but related formulations in the recent literature. In one formulation, ToMA is a reinterpretation of Token Merging (ToMe) for training-free semantic binding in text-to-image synthesis: subject, attribute, and sub-object tokens are merged into composite text tokens so that they share a single cross-attention map, with auxiliary unsupervised losses refining those composites during early denoising (Hu et al., 2024). In a second formulation, ToMA is an efficiency method for diffusion transformers: token reduction is recast as submodular selection plus attention-like merge/unmerge linear maps, with GPU-aligned matrix operations and pattern reuse across layers and timesteps to reduce generation latency while preserving image quality (Lu et al., 13 Sep 2025). The shared premise is that carefully structured token aggregation can alter the behavior of diffusion models without retraining, but the optimization target differs fundamentally between semantic correctness and computational efficiency.

1. Terminological scope and conceptual split

The term ToMA is not used for a single canonical algorithm across the supplied literature. One usage arises from a reinterpretation of ToMe as a “Token Merge with Attention” framework for semantic binding in text-to-image synthesis, while the later paper titled “ToMA: Token Merge with Attention for Image Generation with Diffusion Models” uses the same acronym for a latency-oriented token reduction method (Hu et al., 2024, Lu et al., 13 Sep 2025). This distinction is substantive rather than merely terminological.

Usage	Primary objective	Core mechanism
ToMe reinterpreted as ToMA	Semantic binding in T2I	Merge subject+attribute text tokens into composite tokens; apply end-token substitution and two unsupervised losses
ToMA (2025)	Efficiency in diffusion models	Select diverse tokens by submodular optimization; merge/unmerge via dense matrix operations; reuse patterns across layers and timesteps

In the semantic-binding setting, token merging acts on CLIP-tokenized text embeddings and modifies cross-attention conditioning. In the efficiency setting, token merging acts on transformer token features inside the diffusion model and reduces effective attention workload. A plausible implication is that both methods exploit redundancy in tokenized representations, but they do so for different failure modes: semantic misbinding in one case and quadratic attention complexity in the other.

2. Semantic-binding ToMA: prompt structure and composite token formation

In the semantic-binding formulation, the prompt after CLIP tokenization is written as

$$P = [\mathrm{SOT}, w_1, w_2, \dots, w_L, \mathrm{EOT}, \dots, \mathrm{EOT}],$$

with embeddings

$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$

The prompt is assumed to describe $K$ main entities $n^1, n^2, \dots, n^K$, each associated with a set of attribute or sub-object tokens

$a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$

For each entity, the index set $S_k \subset \{1,\dots,M-1\}$ denotes the tokens corresponding to the $k$-th noun and its attributes. The supplied example is “a cat wearing sunglasses and a dog with a hat,” with $n^1=\text{“cat”}$, $a^1=\{\text{“wearing”},\text{“sunglasses”}\}$, $n^2=\text{“dog”}$, and $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$0 (Hu et al., 2024).

To bind each subject $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$1 with its attributes $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$2, ToMA introduces a composite token $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$3 by merging the embeddings in $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$4:

$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$5

In the simplest case,

$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$6

All original tokens in $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$7 are then removed and replaced by the composite embedding sequence

$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$8

The explicit motivation is semantic binding, defined as associating a given object with its attribute, termed attribute binding, or linking it to related sub-objects, termed object binding. The method is training-free and avoids full-model fine-tuning or externally specified layouts. Because the merged token stands in for an entire subject-attribute group, the method is designed to prevent the separate tokens from competing for inconsistent spatial assignments during generation.

3. Cross-attention integration and end-token substitution

In a standard diffusion U-Net at timestep $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$9, the flattened feature queries $K$0 attend to text keys

$K$1

through

$K$2

where $K$3. After merging, the textual key matrix becomes

$K$4

Since

$K$5

the new attention scores satisfy

$K$6

After the softmax, the composite attention map $K$7 is therefore a weighted mixture of the original maps $K$8 for $K$9 (Hu et al., 2024).

The stated consequence is that the newly introduced token $n^1, n^2, \dots, n^K$0 “inherits” and consolidates the spatial focus of all constituent tokens, guaranteeing that the bound subject, attributes, and sub-objects now share a single cross-attention map. Within the paper’s framing, this is the mechanism by which semantic binding is enforced without retraining.

A complementary mechanism is End-Token Substitution (ETS). The motivation is that CLIP’s $n^1, n^2, \dots, n^K$1 tokens often accumulate global semantics and can re-introduce stray attributes. To remove this spurious coupling, the method replaces the original end-token embeddings with embeddings drawn from a “plain” prompt containing only the subjects $n^1, n^2, \dots, n^K$2:

$n^1, n^2, \dots, n^K$3

The final textual condition is then

$n^1, n^2, \dots, n^K$4

This isolates the subject set from residual prompt-wide semantics carried by repeated end tokens.

4. Training-free refinement, algorithmic workflow, and reported performance

During the first $n^1, n^2, \dots, n^K$5 timesteps of denoising, given as “e.g. 20% of $n^1, n^2, \dots, n^K$6 total,” the semantic-binding ToMA updates only the composite token embeddings via gradients on two unsupervised losses. The entropy regularization term encourages each composite token’s attention map to concentrate on a small region:

$n^1, n^2, \dots, n^K$7

The semantic binding loss enforces that each $n^1, n^2, \dots, n^K$8 leads to the same denoising vector as the full prompt $n^1, n^2, \dots, n^K$9:

$a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$0

The combined objective is

$a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$1

A small number of gradient steps is taken on $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$2 before diffusion continues. No network weights are changed; only composite tokens are refined (Hu et al., 2024).

The pseudocode given in the summary is sequential. It tokenizes the prompt, parses it into $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$3 subject-plus-attribute groups $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$4, initializes each composite token by uniform averaging over its group, substitutes the original tokens with $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$5, replaces post-terminal $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$6 embeddings with $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$7 from a subjects-only prompt, forms the updated text matrix $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$8, initializes $a^k = \{t_{k,1}, \dots, t_{k,r_k}\}.$9, and then denoises from $S_k \subset \{1,\dots,M-1\}$0 to $S_k \subset \{1,\dots,M-1\}$1. For $S_k \subset \{1,\dots,M-1\}$2, denoising proceeds normally; otherwise the method extracts $S_k \subset \{1,\dots,M-1\}$3 and $S_k \subset \{1,\dots,M-1\}$4, builds $S_k \subset \{1,\dots,M-1\}$5, computes $S_k \subset \{1,\dots,M-1\}$6 and $S_k \subset \{1,\dots,M-1\}$7, updates $S_k \subset \{1,\dots,M-1\}$8, re-assembles $S_k \subset \{1,\dots,M-1\}$9, and denoises with the updated condition.

The reported evaluation uses T2I-CompBench, specified as “300 prompts each for color, texture, shape attributes, measured by BLIP-VQA”; a GPT-4o object binding benchmark with “50 hard prompts of the form ‘an X with itemA and a Y with itemB,’ scored by GPT-4o”; and human-preference via ImageReward plus DetScore. The metrics listed are BLIP-VQA, GPT-4o binding accuracy, and human-preference. For base SDXL, the reported BLIP-VQA values are Baseline SDXL $k$0, Best prior (CoMat) $k$1, and ToMA (Ours) $k$2. The GPT-4o score is reported as $k$3 for ToMA versus $k$4 for SynGen, and human preference/ImageReward logits are reported to rise by $k$5 overall. Qualitative examples are described as showing perfect attribute placement, such as “cat wears glasses ○, dog wears hat ○,” with no cross-object leakage, including in multi-object, multi-attribute scenes. The summary concludes that the method achieves state-of-the-art semantic binding on both compositional benchmarks and open-ended GPT-4o object-binding tests.

5. Efficiency-oriented ToMA: submodular token selection and attention-like merge/unmerge

The 2025 ToMA paper addresses a different problem: scalability limits in diffusion models caused by transformers’ quadratic attention complexity. Its starting point is a set of token features $k$6 over a ground set

$k$7

The goal is to select a smaller subset

$k$8

or equivalently impose a budget $k$9 for reduction ratio $n^1=\text{“cat”}$0. Selection is cast as maximizing a facility-location style submodular objective

$n^1=\text{“cat”}$1

with cosine similarity reported as giving the best empirical diversity, though an RBF or inner-product can also be used (Lu et al., 13 Sep 2025).

The paper adopts the classical greedy algorithm for monotone submodular maximization under a cardinality constraint: initialize $n^1=\text{“cat”}$2, then repeatedly choose

$n^1=\text{“cat”}$3

where

$n^1=\text{“cat”}$4

The summary states the standard guarantee from Nemhauser et al. (1978): a $n^1=\text{“cat”}$5 approximation to the optimal $n^1=\text{“cat”}$6. Marginal-gain updates are accelerated by caching $n^1=\text{“cat”}$7 for every $n^1=\text{“cat”}$8, reducing each update to $n^1=\text{“cat”}$9 time rather than recomputing from scratch.

After selection, each original token is assigned to its closest center,

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$0

which induces clusters

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$1

Merge is implemented by a dense matrix $a^1=\{\text{“wearing”},\text{“sunglasses”}\}$2 with entries

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$3

For stacked token features $a^1=\{\text{“wearing”},\text{“sunglasses”}\}$4, the merged representation is

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$5

Standard multihead attention is then applied on $a^1=\{\text{“wearing”},\text{“sunglasses”}\}$6; for one head,

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$7

followed by

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$8

Unmerge is implemented via

$a^1=\{\text{“wearing”},\text{“sunglasses”}\}$9

with

$n^2=\text{“dog”}$0

Operationally, each original token copies the merged result of its cluster center.

The GPU-efficiency claim rests on representation rather than on token reduction alone. Merge and unmerge are both single dense GEMMs, $n^2=\text{“dog”}$1 and $n^2=\text{“dog”}$2, which the summary states can be fused into highly optimized kernels by cuBLAS and cuBLASLt. The paper explicitly contrasts this with ToMeSD and ToFu, which rely on sorting and scatter/gather-like operations.

6. Pattern reuse, complexity, empirical outcomes, and interpretive cautions

The efficiency-oriented ToMA further exploits latent locality and sequential redundancy. Within each transformer block, the assignment $n^2=\text{“dog”}$3 is said to change little from layer to layer, so $n^2=\text{“dog”}$4 and $n^2=\text{“dog”}$5 can be reused across heads or entire blocks and recomputed only when the marginal gain of adding a new center exceeds a threshold $n^2=\text{“dog”}$6. Across diffusion timesteps, the last timestep’s clustering is used as the initial seed for the next, providing a warm start and often avoiding any change. The summary describes this “pattern reuse” as making the expensive greedy selection, with cost roughly $n^2=\text{“dog”}$7 per update, occur “very infrequently—typically once every 4–8 layers” (Lu et al., 13 Sep 2025).

The complexity accounting is explicit. Baseline full attention per layer is given as approximately $n^2=\text{“dog”}$8 FLOPs. For ToMA, attention on merged tokens costs $n^2=\text{“dog”}$9, while the two GEMMs contribute approximately $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$00 FLOPs, yielding

$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$01

For the typical value $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$02, the summary computes

$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$03

described as about a $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$04 baseline FLOPs. The paper’s central argument is that theoretical FLOPs alone are not decisive: the merged attention is on $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$05 rather than $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$06, the GEMMs are hardware-friendly, and submodular oracle calls are rare. The reported end-to-end GPU measurements with FlashAttention are that ToFu achieves only $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$07–$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$08 latency reduction, whereas ToMA “consistently achieves 20–25% end-to-end speedup vs full FlashAttention.”

The headline empirical results are more specific. On Stable Diffusion XL (SDXL) 1.0, ToMA reduces single-image generation latency by $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$09; on Flux, it yields a $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$10 speedup. Across $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$11 ImageNet prompts, the reported DINO-CLIP similarity shift satisfies $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$12 relative to the full-token baseline, characterized as visually indistinguishable. Ablations over reduction ratio $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$13 report that quality is essentially flat up to $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$14, then degrades gracefully. Reclustering every layer versus every $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$15 layers versus every $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$16 layers is reported to show that $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$17–$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$18 gives $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$19–$C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$20 speedup with no measurable quality loss. Visual examples are said to show that even at $C = [c_0^{\mathrm{SOT}}, c_1, \dots, c_{M-1}].$21 token reduction, fine textures such as hair strands and foliage remain crisp, with only imperceptible blur in uniform backgrounds.

A recurrent source of confusion is to treat the two ToMA formulations as interchangeable. That would be inaccurate. The semantic-binding variant merges text tokens to enforce subject-attribute coherence and uses entropy plus semantic binding losses during early denoising. The efficiency variant merges internal transformer tokens to reduce latency and uses submodular selection, dense merge/unmerge maps, and reuse heuristics. This suggests a broader design space around token aggregation in diffusion models, but the supplied papers do not claim that the semantic-binding objective and the efficiency objective are the same problem, nor that one method subsumes the other.