---
title: Token-Domain Multiple Access (ToDMA)
url: https://www.emergentmind.com/topics/token-domain-multiple-access-todma
type: topic
---

# Token-Domain Multiple Access (ToDMA)

Token-Domain Multiple Access (ToDMA) is a large-model-driven grant-free multiple access framework for semantic communications in which a massive set of devices encode, transmit, and recover information in the token domain. ToDMA leverages pre-trained tokenization and modulation codebooks, compressed sensing for token detection, channel state information (CSI) clustering for user separation, and context-aware masked token prediction using multimodal large language models (MLLMs) to resolve token collisions. This pipeline achieves high transmission efficiency and low end-to-end latency, with semantic-level quality superior to both orthogonal and context-unaware non-orthogonal communication schemes, across text and image modalities [2505.10946][2502.06118].

## 1. Architecture and Signal Processing Pipeline

ToDMA defines a unified transmitter and receiver pipeline implemented as follows:

- **Tokenization (Source Coding):** Each device encodes its source data (such as an image patch or text sequence) \(s\) into a sequence of tokens via a shared learned tokenizer,  
  \[
  T: s \mapsto (t_1, \dots, t_N)
  \]
  where each \(t_n \in \{1, \dots, Q\}\) indexes a common token codebook of size \(Q\). Tokens are equivalently represented as one-hot vectors \(\mathbf{b}_n \in \{0,1\}^Q\) [2505.10946][2502.06118].

- **Modulation (Channel Coding):** Token indices are mapped by a shared complex-valued modulation codebook \(\mathbf{U} \in \mathbb{C}^{L \times Q}\),  
  \[
  m: t_n \mapsto \mathbf{x}_n = \mathbf{U}\mathbf{b}_n \in \mathbb{C}^L
  \]
  so each device transmits over \(N\) time slots the codeword matrix \(\mathbf{X} = [\mathbf{x}_1, \ldots, \mathbf{x}_N] \in \mathbb{C}^{L \times N}\). The orthonormality \(U^HU = I_Q\) is enforced for efficient projection and separation [2502.06118].

- **Channel Model:** In a grant-free uplink, \(K \ll K_T\) active devices simultaneously transmit to an \(M\)-antenna base station (BS). The received signal at slot \(n\) is  
  \[
  \mathbf{Y}_n = \sum_{k=1}^K \mathbf{x}_{k,n} \mathbf{h}_k^T + \mathbf{Z}_n = \mathbf{U}\left( \sum_{k=1}^K \mathbf{b}_{k,n} \mathbf{h}_k^T \right) + \mathbf{Z}_n
  \]
  where \(\mathbf{h}_k \in \mathbb{C}^M\) is the (slot-invariant) channel of device \(k\) and \(\mathbf{Z}_n\) is i.i.d. additive white Gaussian noise [2505.10946][2502.06118].

## 2. Receiver Design: Token Detection and Channel Clustering

The base station implements a multi-step recovery procedure:

- **Compressed Sensing Token Detection:** For each time slot, the receiver projects \(\mathbf{Y}_n\) onto columns of \(\mathbf{U}\), forming  
  \[
  \hat{\mathbf{H}}_n = \mathbf{U}^H \mathbf{Y}_n = \mathbf{H}_n + \mathbf{U}^H \mathbf{Z}_n
  \]
  where \(\mathbf{H}_n \in \mathbb{C}^{Q \times M}\) is row-sparse, encoding which tokens are active. Approximate Message Passing (AMP) is employed to detect the active token set \(\hat{\mathcal{P}}_n\) and estimate their CSI vectors \(\{\hat{\mathbf{h}}_{\phi, n}\}\) [2505.10946].

- **Channel Clustering for Source Assignment:** Each device's channel remains constant over \(N\) slots. For robust token-to-user assignment, the receiver clusters the estimated CSI vectors \(\widehat{\mathcal{F}} = \{\hat{\mathbf{h}}_{\phi, n}\}\) into \(K\) groups using K-means++,  
  \[
  \min_{\{\mathcal{C}_k\}} \sum_{k=1}^K \sum_{\mathbf{h} \in \mathcal{C}_k} \|\mathbf{h} - \mathbf{c}_k\|_2^2
  \]
  Each detected token is then assigned to the closest cluster center (user), yielding a set of partially reconstructed token sequences per device [2505.10946].

## 3. Semantic Orthogonality and Collision Mitigation

- **Semantic Orthogonality:** Token sequences from different devices occupy distant regions in a learned embedding space, measured via low average cosine similarity or large distributional divergence:
  \[
  \text{sim}(B_i,B_j) = \frac{1}{N}\sum_{n=1}^N \frac{\langle E_{t_{i,n}}, E_{t_{j,n}} \rangle}{\|E_{t_{i,n}}\| \cdot \|E_{t_{j,n}}\|}
  \]
  This separation enables disentangling of overlapping transmissions even when tokens collide [2502.06118].

- **Collision Handling via Masked Prediction:** When multiple devices choose the same token in the same slot, clustering cannot resolve assignments, and corresponding tokens are masked (\([{\rm MASK}]\)). Each device's partial token sequence
  is completed using a pre-trained multimodal language model (MLLM), e.g., BERT (text) or MaskGIT (image), by predicting candidate tokens in context, restricted to the detected collision set \(\widetilde{\mathcal{P}}_n\):
  \[
  \hat{t}_m = \arg\max_{q \in \widetilde{\mathcal{P}}_n} P(t_m = q \mid \text{context})
  \]
  This targeted search reduces masked token prediction complexity from \(O(Q)\) to \(O(|\widetilde{\mathcal{P}}_n|)\) [2505.10946][2502.06118].

## 4. Bayesian Interpretation and Receiver Inference

Maximum a posteriori (MAP) inference for the joint set of all tokens transmitted by all devices is formally intractable:
\[
\{\hat{t}_{k,n}\} = \arg\max_{\{t_{k,n}\}} P(\{t_{k,n}\} | \{\mathbf{Y}_n\}_{n=1}^N, \mathbf{U}, \{\mathbf{h}_k\})
\]
ToDMA operationalizes an approximate MAP solution by chaining:

- Matched-filter token detection and energy thresholding;
- Nearest-CSI channel assignment (clustering);
- Contextual masked token re-prediction, as a conditional MAP estimate restricted to detected collision candidates, with the transformer acting as a probabilistic scorer [2502.06118].

## 5. Practical Implementation and Complexity

A single shared tokenizer and modulation codebook ensure coherent symbol mapping and efficient grant-free random access across a large device set. Transformer-based tokenizers provide context-sensitive token quantization, increasing semantic robustness [2502.06118][2505.10946].

Per-slot receiver complexity is determined by:

| Operation                 | Complexity                   | Dominant Factors               |
|---------------------------|-----------------------------|--------------------------------|
| Token Detection           | \(O(NLM + QM)\)             | Matrix projections             |
| Assignment                | \(O(K|\mathcal{P}_n|M)\)    | Channel clustering; \(|\mathcal{P}_n|\approx K\) |
| Masked Prediction         | \(O(KNd^2)\) (per sample)   | Transformer embedding dimension \(d\); reduced by candidate restriction |

The candidate set restriction for masked prediction permits the use of large token vocabularies (\(Q\approx 10^3-10^4\)) and long sequences efficiently [2502.06118].

Recommended system design principles include:

- \(M \geq Q\) BS antennas for under-determined least squares separation;
- Codebook orthonormality and transformer depth (8–12 layers) for effective masking;
- Thresholds (\(T_h \approx 2\sigma^2\)) calibrated for detection/assignment accuracy [2502.06118].

## 6. Performance Evaluation

Simulation studies in image (ImageNet-100, VQ-GAN, MaskGIT) and text (QUOTES500K, BERT) transmission scenarios confirm the following performance characteristics [2505.10946][2502.06118]:

- **Token Error Rate (TER):** ToDMA maintains TER near those of error-free orthogonal QAM (Orth-Com) for increasing active device count \(K\), while context-unaware non-orthogonal schemes degrade rapidly with collisions.
- **Semantic Quality:** ToDMA's PSNR in images remains within 1–2 dB of Orth-Com up to \(K = 80\) (Orth-Com at PSNR ≈ 32 dB for \(K=20\)); text BERTScore ≈ 0.92 at \(K=20\) (vs 0.78 for non-contextual baseline). Visual reconstructions are nearly indistinguishable from the orthogonal scheme [2505.10946].
- **Latency:** ToDMA achieves \(\sim4\times\) lower end-to-end latency relative to Orth-Com at typical BER targets, due to non-orthogonal, grant-free access and minimal pre-transmission coordination.
- **Collision Recovery:** MLLM-based masked token prediction successfully resolves >90% of token collisions, as shown in token-map heatmaps and output quality [2505.10946][2502.06118].

## 7. Insights, Limitations, and Design Implications

Key properties of ToDMA, supported by empirical results and theoretical design, include:

- Exploitation of semantic orthogonality enables disentanglement of overlapping codewords at the receiver, leveraging the low inter-sequence similarity in embedding space.
- Joint source-channel coding is realized in the token domain, bypassing the inefficiency of separate bit-wise coding.
- Grant-free operation with global codebook/tokenizer coordination dramatically reduces signaling overhead and scales to large device populations.
- Leveraging masked tokens and MLLMs for context-driven completion enables robust resolution of collisions with tractable inference complexity.
- *A plausible implication is* that future ToDMA extensions could adaptively tune codebook size, sequence length, and transformer depth to modality and application requirements.

ToDMA currently assumes shared tokenization/modulation infrastructure and well-calibrated model/antenna configurations; deviations might affect separation or completion accuracy. The necessity of large, pre-trained context models (e.g., MaskGIT, BERT) implicates scalability constraints in resource-limited settings. These considerations delineate active research directions within semantic communications leveraging token-domain access mechanisms [2505.10946][2502.06118].

Source: https://www.emergentmind.com/topics/token-domain-multiple-access-todma