Text-Guided Token Communication System

• Text-Guided Token Communication System is a semantic communication framework that uses textual cues to guide token selection, coding, and recovery for robust multimodal transmission.
• It employs pretrained transformers, discrete codebooks, and adaptive token masking to optimize resource allocation and mitigate errors based on semantic importance.
• Applications span wireless image transmission and multimodal interactive coding, demonstrating improved semantic fidelity and resilience under challenging channel conditions.

Searching arXiv for the core paper and closely related token-communication work to ground the article in current literature. A Text-Guided Token Communication System is a semantic communication architecture in which the basic communication unit is a token rather than a bit, symbol, or pixel, and textual information—captions, class labels, prompts, questions, or task instructions—acts as side information, primary payload, or semantic prior for token selection, coding, transmission, and reconstruction. In the broader TokCom paradigm, the transmitter tokenizes source modalities into discrete indices from a shared codebook, while the receiver uses pretrained transformers, generative foundation models, or multimodal LLMs to recover missing or corrupted tokens under textual guidance (Qiao et al., 17 Feb 2025). The result is a communication model aligned with the internal representation of contemporary transformer systems, with direct implications for semantic compression, context-aware error mitigation, and multimodal reasoning.

1. Conceptual foundations

Token communication departs from conventional digital communication by treating tokens as the primitive information units. In TokCom, a tokenizer maps a source modality $X$ to a discrete sequence $T=(t_1,\dots,t_N)$, with $t_i \in \{1,\dots,Q\}$, and a de-tokenizer maps the recovered sequence $\hat T$ back to $\hat X$. The shared knowledge base is the token codebook together with the pretrained generative or foundation model that operates on tokens. Conventional digital systems optimize bit error rate or frame error rate; classic semantic communication and DeepJSCC often use continuous latent features and end-to-end task-specific models; TokCom instead uses discrete tokens with semantic meaning, explicit digital codebooks, and pretrained transformers or GFMs/MLLMs (Qiao et al., 17 Feb 2025).

Within this framework, text guidance has three canonical roles. First, text may serve as side information or context: a short class label or caption is sent alongside another modality and conditions token recovery. Second, text may be the primary payload, as in language-token wireless transmission. Third, text may function as a semantic prior, where the transmitter sends very few or no image or video tokens and the receiver generates content from the textual intent plus minimal token hints. This triad—text as context, payload, and prior—organizes much of the recent literature on text-guided TokCom (Qiao et al., 17 Feb 2025).

A common misconception is to equate text guidance with caption-based postprocessing. In the cited systems, text is often structurally integrated into the token pipeline itself: it affects which tokens are transmitted, how they are protected, how missing tokens are inferred, and, in some architectures, how the entire multimodal task is specified. Another misconception is to treat semantic communication as merely a softer rate–distortion objective. TokCom is explicitly model-centric: it reuses pretrained token interfaces and transformer prediction mechanisms, rather than retraining an analog latent mapper for every task and channel (Qiao et al., 17 Feb 2025).

2. Text-guided system model

The core text-guided multimodal interface can be written as a joint token model over text and non-text sequences. Let $T^{\text{text}}=(u_1,\dots,u_M)$ be text tokens and $T^{\text{img}}=(v_1,\dots,v_N)$ be image tokens. A unified transformer processes concatenated multimodal tokens as

$$F_\theta(T^{\text{text}}, T^{\text{img}}) \rightarrow \{p(u_i|\cdot), p(v_j|\cdot)\}.$$

In a text-guided image system, the transmitter may drop image tokens that are highly predictable given the text, using the conditional uncertainty

$$H(v_j \mid T^{\text{text}})=\mathbb{E}\big[-\log p(v_j \mid T^{\text{text}})\big],$$

and the receiver reconstructs masked image tokens by

$$\hat v_j=\arg\max_v p(v \mid T^{\text{text}}, T^{\text{img}}_{\text{observed}}, \text{channel info}).$$

This formulation is explicit in TokCom’s mapping of text-guided token communication to cross-modal conditional prediction (Qiao et al., 17 Feb 2025).

The same principle appears in text-only wireless token transmission. There, the source is a token sequence $T=(t_1,\dots,t_N)$0, each token is mapped to bits and then to QAM symbols, and a pretrained masked LLM provides contextual priors. The receiver combines per-token channel likelihoods with MLM-based priors through the iterative rule

$T=(t_1,\dots,t_N)$1

This converts token detection into a Bayesian fusion of physical observations and language structure, and it also enables transmitter-side masking of highly predictable tokens to reduce transmission rate (Shin et al., 25 Jan 2026).

At the architectural level, text-guided systems are naturally layered. TokCom identifies an application or semantic layer containing tasks and pretrained GFMs/MLLMs, a semantic source coding layer for tokenization and token selection, a semantic channel coding layer for context-aware resource allocation, and a network/MAC layer for packetization, context-aware ARQ, routing, and semantic multiple access. This layered view is one reason TokCom emphasizes discrete token units and explicit codebooks rather than implicit analog latents (Qiao et al., 17 Feb 2025).

3. Representative architectures

A strong cross-modal instantiation is the TaiChi/TokenCom framework, which embeds text guidance directly into a multimodal vision-language communication stack. TaiChi uses a dual-visual tokenizer architecture, a Bilateral Attention Network for fusing high- and low-resolution visual tokens, a KAN-based modality projector with learnable activation functions to align visual features with the text semantic space, and an LLM backbone such as Gemma-2B or Qwen2.5-14B. Text acts as an instruction or query control signal, and the multimodal token sequence is processed by an LLM encoder before joint VLM-channel coding (Jiang et al., 28 Feb 2026).

A more communication-centric image pipeline appears in text-guided wireless image transmission. In that design, a pretrained TA-TiTok encoder maps a $T=(t_1,\dots,t_N)$2 image to 128 discrete tokens from an 8,192-entry codebook, each token index occupies 13 bits, and tokens are grouped into packages of 8 tokens. Each package undergoes CRC attachment with $T=(t_1,\dots,t_N)$3, 5G NR polar coding with block length $T=(t_1,\dots,t_N)$4, interleaving, rate matching, and 4-QAM transmission over an AWGN channel. At the receiver, CRC failures determine which token positions are masked, and a text-guided masked generative model, MaskGen, reconstructs those tokens using captions generated by Molmo-7B and encoded by CLIP (Liu et al., 8 Jul 2025).

Text-only systems expose a complementary architectural line. In one formulation, the transmitter computes contextual uncertainty for each language token with BERT, greedily masks the most predictable positions, and transmits only unmasked tokens over a Rayleigh block-fading channel. The receiver initializes with maximum-likelihood detection on observed tokens and $T=(t_1,\dots,t_N)$5 on omitted positions, then iteratively refines the entire sequence using the shared MLM prior. A related framework jointly designs masking and detection and reports up to $T=(t_1,\dots,t_N)$6 and $T=(t_1,\dots,t_N)$7 performance gains on the Europarl corpus and WikiText-103 datasets, respectively (Shin et al., 4 May 2026).

Another architectural pattern replaces raw image transmission with caption-first semantic generation. In sequential semantic generative communication, the transmitter converts an image to a caption using BLIP, splits the sentence into word tokens, and transmits those words sequentially according to either lowest-LPIPS transmission, most attentive transmission, or least attentive transmission. The receiver uses Stable Diffusion v2 to generate images from partial prompts, and communication stops once LPIPS falls below $T=(t_1,\dots,t_N)$8. This is a text-guided token system in the strict sense that textual tokens are both the transmitted codewords and the semantic control interface for image reconstruction (Nam et al., 2023).

4. Coding, masking, and adaptive protection

Resource allocation in text-guided token communication is driven by conditional predictability. In TokCom, highly predictable tokens may be dropped or given less protection, while unpredictable tokens receive stronger coding, power, or modulation robustness. This extends naturally to semantic channel coding: tokens with higher semantic importance or higher conditional uncertainty given text are retained or protected more heavily, whereas predictable tokens are left to generative recovery (Qiao et al., 17 Feb 2025).

The text-only masking literature operationalizes this principle with entropy. For a masking set $T=(t_1,\dots,t_N)$9, the transmitter computes token-wise entropy from an MLM and greedily masks the token with the smallest entropy at each step. This creates a shared source–channel strategy: the transmitter omits tokens that the receiver can infer, and the receiver fuses channel likelihoods with contextual priors. Reported gains include up to approximately $t_i \in \{1,\dots,Q\}$0 in semantic similarity on Europarl and up to approximately $t_i \in \{1,\dots,Q\}$1 on WikiText-103 for iterative detection alone, as well as gains over random masking up to approximately $t_i \in \{1,\dots,Q\}$2 and $t_i \in \{1,\dots,Q\}$3, respectively (Shin et al., 25 Jan 2026).

Another robustness strategy is transmitter-side token re-encoding without length expansion. TokCode introduces a token encoding framework for robust semantic recovery that incurs no additional transmission overhead and supports plug-and-play deployment. It uses a T5-based token encoder with LoRA adaptation and a sentence-semantic-guided foundation model adaptation algorithm so that, after random packet erasures, the sentence embedding of the received encoded prompt remains close to that of the original prompt. In prompt-based generative image transmission, TokCode mitigates semantic distortion and can approach the performance upper-bound even under harsh channels where $t_i \in \{1,\dots,Q\}$4 to $t_i \in \{1,\dots,Q\}$5 of tokens are randomly lost (Hu et al., 14 Apr 2026).

Unequal protection also appears in multimodal and video settings. Video TokenCom integrates user-intended textual descriptions with discrete video tokenization and unequal error protection to enhance semantic fidelity under restrictive bandwidth constraints. Tokens highly related to the user intent are encoded using full codebook precision, whereas non-intended tokens are represented through reduced codebook precision differential encoding, and source and channel coding are adapted to varying resources and link conditions (Men et al., 2 Mar 2026). This suggests a general principle: text guidance need not merely reconstruct after errors; it can determine the precision and protection level assigned before transmission.

5. Applications and empirical evidence

The best documented image TokCom case study uses ImageNet100 with $t_i \in \{1,\dots,Q\}$6 images, $t_i \in \{1,\dots,Q\}$7 image tokens, codebook size $t_i \in \{1,\dots,Q\}$8, 16 tokens per packet, rate-$t_i \in \{1,\dots,Q\}$9 convolutional code with CRC, 16-QAM, and flat Rayleigh fading. A 7-bit class label is transmitted over a robust control channel and used as cross-modality information for MaskGIT. At SNR $\hat T$0 dB, packet error rate is approximately $\hat T$1, and TokCom with cross-modality information achieves $\hat T$2 TCE improvement over conventional ARQ with comparable CLIP semantic similarity. At SNR $\hat T$3 dB, where PER is approximately $\hat T$4, TokCom with cross-modality information maintains CLIP score $\hat T$5 with only $\hat T$6 degradation versus ideal transmission; TokCom without cross-modality information is noticeably worse (Qiao et al., 17 Feb 2025).

Text-guided wireless image transmission extends this line with standard 5G NR polar coding and a stronger pretrained token stack. At bandwidth ratio $\hat T$7, the system outperforms ADJSCC in LPIPS and CLIP similarity for SNRs above $\hat T$8 dB while mitigating the cliff effect at lower SNRs. Longer captions, approximately 70 tokens rather than approximately 30 tokens, improve PSNR, LPIPS, and CLIP scores at low SNR when MaskGen is used, while the benefit saturates at higher SNRs. Cross-dataset evaluation on Flickr shows that the token system continues to outperform ADJSCC in LPIPS for SNR $\hat T$9 dB and in CLIP similarity across all SNRs, illustrating the retraining-free generalization claim made by the paper (Liu et al., 8 Jul 2025).

Text-guided token systems also scale to multimodal interactive coding. UniMIC uses GPT-style text tokens, MagViT-v2 image tokens, and scenario-specific Transformer entropy models—generic, masked, and text-conditioned—to minimize inter-token redundancy. It reports substantial bitrate savings and robustness even at ultra-low bitrates $\hat X$0 bpp on text-to-image generation, text-guided inpainting, outpainting, and visual question answering. In T2I, for example, UniMIC reports $\hat X$1 bpp with lossless token reconstruction on the cloud-to-edge leg, while conventional and learned pixel codecs show either higher bitrate or degraded semantic alignment (Mao et al., 26 Sep 2025).

Multi-user variants show that text guidance is not restricted to point-to-point links. ToDMA lets many devices share a token codebook and a modulation codebook, detects active tokens and CSI via compressed sensing, clusters token-associated CSI across time slots, and uses BERT or MaskGIT to fill masked positions caused by collisions. The result is significantly lower latency than orthogonal communication and better distortion or perceptual quality than context-unaware non-orthogonal methods for both text and image tasks (Qiao et al., 16 May 2025). This establishes text guidance as a mechanism for collision mitigation as well as reconstruction.

6. Limitations, misconceptions, and open research directions

A recurring limitation is computational cost. TokCom relies on large GFMs/MLLMs, MaskGen, VLMs, or large MLMs, so deployment often requires collaborative device-edge-cloud partitioning. The literature explicitly points to on-device tokenizers and shallow models, with heavier reconstruction offloaded to the edge or cloud when energy and latency budgets permit (Qiao et al., 17 Feb 2025). TaiChi’s smaller Gemma-2B configuration is still a large model, and text-guided image transmission with TA-TiTok and MaskGen incurs significantly higher inference FLOPs than ADJSCC even though it avoids retraining (Liu et al., 8 Jul 2025).

Another limitation is semantic misalignment. Generative reconstruction may be plausible yet wrong. TokCom identifies hallucination, ambiguity in text guidance, and misalignment with user intent as central concerns, especially when too many tokens are missing. TokCode likewise shows that at low packet loss, learned redundancy may slightly reduce fine-grained detail, while at high loss rates it improves semantic recovery. These results suggest that semantic quality is not a monotone proxy for factual correctness; text-guided systems must distinguish between plausible completion and intended completion (Qiao et al., 17 Feb 2025).

Security and privacy remain open. Tokens carry high-level semantics, so token streams may expose sensitive intent or content. The cited TokCom literature explicitly notes adversarial token editing, prompt injection, hallucinations, and the need for cryptography suited to token streams as well as safety and consistency constraints in generative models (Qiao et al., 17 Feb 2025). A related but distinct line of work argues that even token communication may remain ambiguous and redundant relative to latent communication, and studies direct exchange of embeddings, hidden states, or KV-caches in LLM-based multi-agent systems (Liu, 4 Jun 2026). This does not supersede text-guided token communication, but it does define an important boundary of the topic: tokens remain the interpretable and digitally compatible interface, whereas latent communication pursues a different efficiency–interpretability trade-off.

Future work is consistently described in terms of better tokenizers, adaptive token budgets, more realistic channels, cross-modal alignment, semantic-aware PHY/MAC, and broader multimodal integration. The open direction running through the literature is clear: use text not merely to describe content after transmission, but to shape what is tokenized, what is sent, what is protected, and what is inferred. In that sense, the Text-Guided Token Communication System is not a single architecture but an emerging class of systems that align wireless communication with the token-level computation of pretrained multimodal models (Qiao et al., 17 Feb 2025).