ATS-ToDMA: Adaptive Token Selection and Token-Domain Multiple Access for Cross-Modal Semantic Communications

Abstract: Adaptive token processing has emerged as a promising approach for improving the efficiency of semantic communication systems. However, existing semantic communication frameworks largely overlook token-level multiple access and the impact of semantic interference among simultaneously transmitted semantic tokens. In this paper, we propose Adaptive Token Selection and Token-Domain Multiple Access (ATS-ToDMA), a novel cross-modal semantic communication framework that jointly performs semantic token selection, interference-aware scheduling, and semantic-aware power allocation. The proposed framework introduces a Semantic Signal-to-Interference-plus-Noise Ratio (SSINR) metric that captures the combined effects of channel impairments and semantic interference arising from token similarity. A transformer-based scheduler is developed to allocate selected semantic tokens across token-domain transmission slots while mitigating both intra-modal and cross-modal semantic interference. To characterize the behavior of the proposed system, analytical bounds on semantic interference and feasible token occupancy are derived, together with a closed-form approximation for semantic-aware power allocation. Simulation results demonstrate significant gains in semantic throughput and semantic decoding accuracy while reducing aggregate semantic interference and transmit power compared with OMA, Semantic NOMA, Random-TS, and Greedy ATS benchmarks.

• The paper introduces ATS-ToDMA, a framework that optimizes semantic token selection, interference-aware scheduling, and power allocation for cross-modal communications.
• It employs modality-specific encoders and a transformer-based scheduler to enhance throughput, decoding accuracy, and energy efficiency under SSINR constraints.
• Simulation results show significant improvements, including +31.4% semantic throughput and -28.9% aggregate interference, offering practical insights for next-generation semantic networks.

ATS-ToDMA: Adaptive Token Selection and Token-Domain Multiple Access for Cross-Modal Semantic Communications

Introduction

The paper presents ATS-ToDMA, a cross-modal semantic communication (SemCom) framework that fundamentally rethinks multi-user, multi-modal wireless communications by prioritizing the transmission and scheduling of semantic tokens. Existing SemCom systems primarily operate at the signal or feature level, neglecting the semantic interference that arises from similarity among simultaneously transmitted tokens, particularly in resource-constrained, multi-user scenarios. The proposed ATS-ToDMA framework addresses the joint optimization of semantic token selection, interference-aware scheduling, and power allocation, enabling enhanced resource utilization, scalability, and adaptability for next-generation networks.

System Model and Framework

ATS-ToDMA is designed for multi-user, cross-modal SemCom with support for textual, visual, and audio streams. The main pipeline includes modality-specific encoders, adaptive token selection (ATS), transformer-based scheduling, and a channel-aware, joint multi-modal decoder.

The semantic encoder maps raw signals, such as image patches or text fragments, into normalized semantic tokens. The ATS module applies importance thresholding, retaining only the semantically valuable tokens, thereby reducing redundancy and minimizing unnecessary interference.

Figure 1: The overall SemCom architecture integrates adaptive token selection at the transmitter to filter task-relevant semantic tokens, reducing wireless resource consumption.

For scalable and interference-aware multi-user access, a transformer-based scheduler exploits global token relationships and importance scores via self-attention. This scheduler assigns tokens to token-domain (ToDMA) slots, incorporating both intra- and cross-modal semantic interactions. Strict slot occupancy and interference constraints are enforced using a hybrid of differentiable (soft) and post-processing (hard) techniques, ensuring deployment-phase constraint satisfaction.

Figure 2: The ATS-ToDMA architecture supports multi-user, cross-modal semantic transmission by exploiting a transformer-based scheduler for token-domain slot assignment and cross-modal fusion at the receiver.

Semantic Interference Modeling and SSINR Formulation

A central contribution is the explicit modeling of semantic interference, which differs fundamentally from electromagnetic interference:

• Semantic Similarity ($\xi_{ij}$): Quantified by cosine similarity between token embeddings; high similarity induces decoding ambiguity.
• Semantic Interference Term ($I_{ij}$): Depends on similarity, power, and a modality-sensitive distortion coefficient ($\alpha_{ij}$), capturing both intra- and cross-modal interactions.
• Semantic Signal-to-Interference-plus-Noise Ratio (SSINR): This metric generalizes SINR to the semantic domain by aggregating physical noise and semantic ambiguity in a unified framework.

Tokens' allocation across slots is optimized under SSINR constraints, dictating feasible slot occupancy and per-token power.

Theoretical Bounds and Optimization

ATS-ToDMA provides tight analytical results substantiating the interplay between token selection, scheduling, and interference:

• The quadratic growth of aggregate semantic interference with ToDMA slot occupancy is formally bounded.
• The framework derives maximal feasible token occupancy ensuring target SSINR, revealing channel-aware, data-driven slot allocation policies.
• Closed-form, low-complexity power allocation formulas are introduced, combining channel state information and token-level semantic significance.

Simulation Results

Simulation studies validate and interpret the analytical findings:

• Semantic Throughput: ATS-ToDMA achieves higher throughput than OMA, semantic NOMA, and greedy or random baselines, with larger gains under dense user regimes.
• Decoding Accuracy: Robustness to noise and interference is improved, especially at lower SNR, due to intelligent token filtering and scheduling.
• Semantic Similarity Threshold: Intermediate thresholds, selected by ATS, maximize throughput by balancing information preservation and interference suppression.

  Figure 3: Semantic throughput versus semantic similarity threshold shows optimal tradeoff at intermediate threshold values.

• Theoretical upper bounds on semantic interference and token occupancy closely match empirical results, establishing reliable design guidelines.
• The closed-form power allocation achieves near-optimality with minimal computational overhead, substantially reducing average power relative to uniform allocation.

Comparative Analysis and Design Implications

ATS-ToDMA's joint modeling of adaptive token selection, semantic-label multiple access, and power optimization establishes several critical insights:

• Blindly increasing token occupancy rapidly escalates semantic interference, undermining decoding reliability.
• Channel-aware mechanisms enable more aggressive (yet safe) slot utilizations under favorable SNR/CSI.
• Semantically isolated tokens—i.e., those with low inter-token similarity—consistently require less power, achieving higher energy efficiency.
• Cross-modal transmission incurs less semantic ambiguity per transmitted token due to lower effective distortion coefficients.

When benchmarked, the ATS-ToDMA framework outperforms state-of-the-art methods across all major Key Performance Indicators (KPIs): semantic throughput (+31.4%), semantic decoding accuracy (+8.5%), aggregate interference (–28.9%), SSINR (+42.6%), and average transmit power (–21.0%).

Practical and Theoretical Implications

The rigorous, scheduling- and interference-aware design proposed in ATS-ToDMA provides a systematic solution to the practical limitations of existing semantic communication systems. Treating semantic tokens as schedulable resources establishes a direct analogy to traditional interference management yet leverages the discriminative capability of deep semantic embeddings and attention-based models. Theoretical advances in semantic SINR modeling and closed-form power control offer new analytical tools for future semantic networks.

Practically, ATS-ToDMA's scalability and adaptivity are central for AI-native, task-oriented networks such as edge intelligence, immersive communication, and autonomous systems. Moreover, the framework is modular and amenable to further extension with real foundation-model embeddings, large-scale datasets, or dynamic conditions (e.g., mobility, heterogeneous semantic traffic).

Conclusion

The ATS-ToDMA framework introduces a holistic, theoretically substantiated approach to scalable, interference- and reliability-aware semantic communication for multi-user, cross-modal wireless systems. By unifying adaptive token selection, transformer-based scheduling, and semantic SINR-aware power allocation, the approach enables substantial performance and efficiency gains. Theoretical results are corroborated by simulations, and practical design insights are demonstrated. Future research directions include integration with foundation models, large-scale deployment studies, and adaptation to dynamic wireless environments.