MU-SemDaCom: Multi-User Semantic Data Communication

• MU-SemDaCom is a communication framework where multiple users exchange semantic features and data streams to optimize tasks such as image reconstruction, translation, and VQA.
• It jointly designs semantic encoding, channel coding, and resource allocation across diverse settings including downlink broadcast, uplink fusion, and degraded channels.
• The approach employs advanced methods like transfer learning, knowledge distillation, and diffusion models to enhance reliability, reduce interference, and adapt to heterogeneous device capabilities.

Searching arXiv for the cited MU-SemDaCom papers to ground the article in current records. arxiv_search({"query":"(Nguyen et al., 2024) Optimizing Multi-User Semantic Communication via Transfer Learning and Knowledge Distillation", "max_results": 5}) arxiv_search({"query":"multi-user semantic communication transfer learning knowledge distillation", "max_results": 10}) Multi-User Semantic and Data Communication (MU-SemDaCom) denotes a class of multi-user communication frameworks in which transmitters and receivers exchange semantic features, task-relevant representations, or mixed semantic/data streams rather than relying solely on conventional bit-centric transmission. In the cited literature, MU-SemDaCom appears in downlink broadcast, uplink fusion, OFDMA, MU-MISO, degraded broadcast channel, relay-aided, and reasoning-aware settings, with tasks including image reconstruction, visual question answering (VQA), image retrieval, machine translation, segmentation, and concurrent task execution plus data reconstruction (Nguyen et al., 2024, Xie et al., 2021, Yuan et al., 29 Sep 2025). Its defining premise is that communication, inference, and resource allocation are jointly designed around semantic objectives such as PSNR, SSIM, LPIPS, FID, answer accuracy, classification accuracy, mIoU, BLEU, or effective semantic information, rather than being treated as independent layers.

1. Conceptual scope and problem setting

MU-SemDaCom generalizes semantic communication from a single link to a networked setting with multiple users, heterogeneous devices, and often heterogeneous objectives. Representative formulations include a single base station broadcasting an image to $K$ users with differing computing capacities (Nguyen et al., 2024), a multi-user uplink semantic communication system with $K$ single-antenna users and one $M$-antenna fusion receiver (Xie et al., 2021), a downlink MU-MISO system serving both data users and semantic users over the same frequency band (Yuan et al., 29 Sep 2025), and decentralized multi-user systems in which receivers may invoke reasoning modules when communication is interrupted (Singh et al., 2024).

The meaning of “semantic” varies by task. In reconstruction-oriented systems, semantics is the latent representation that supports image recovery under channel impairment, as in Swin-based JSCC broadcast and degraded broadcast semantic fusion (Nguyen et al., 2024, Wu et al., 2024). In task-oriented systems, semantics is the representation sufficient for downstream inference, as in MU-DeepSC for VQA, DeepSC-IR for image retrieval, and DeepSC-MT for machine translation (Xie et al., 2021, Xie et al., 2021). In mixed semantic/data settings, semantic users pursue a task such as classification while data users reconstruct raw source data, and both coexist under a common resource-allocation framework (Yuan et al., 29 Sep 2025).

A recurrent misconception is that semantic communication is synonymous with perceptual reconstruction. The literature does not support that reduction. Some frameworks optimize only task execution via cross-entropy or answer accuracy (Xie et al., 2021, Xie et al., 2021); others optimize reconstruction fidelity (Nguyen et al., 2024, Wu et al., 2024); others explicitly combine task execution and reconstruction through a convex-combination objective based on mutual information (Tillmann et al., 22 Oct 2025). Another misconception is that multi-user semantic communication must remain orthogonal at the physical layer. The published designs include NOMA-based superposition (Li et al., 2023), non-orthogonal codewords (Wang et al., 10 Dec 2025), semantic fusion over degraded broadcast channels (Wu et al., 2024), and shuffle-based orthogonalization intended to make interference appear Gaussian-like after de-shuffling (Zhang et al., 28 Jul 2025).

2. Canonical system models and architectural patterns

A common MU-SemDaCom pattern is joint source-channel coding at the transmitter and learned semantic decoding at the receiver. In the downlink Swin-based framework, the base station uses a semantic encoder $E^S_\psi$ and channel encoder $E^C_\beta$ to map an image $I$ into a transmitted vector $X_I$, each user receives $Y_{I,k}=X_I+N_k$, and user $k$ reconstructs $\hat I_k=D^S_{\sigma_k}(D^C_{\gamma_k}(Y_{I,k}))$ (Nguyen et al., 2024). In the multimodal VQA setting, different users transmit image and text features, a multi-antenna receiver performs detection and channel decoding, and a 12-cell MAC network fuses the recovered modalities into an answer distribution (Xie et al., 2021).

Several works depart from the basic point-to-point JSCC abstraction by embedding multi-user structure directly into the semantic layer. DMCE considers multiple users such as road cameras transmitting semantic features of multi-source data to a receiver, which equalizes the channel, recovers features, and fuses them into a segmentation map (Zeng et al., 2024). In degraded broadcast channels, the transmitter extracts semantic features for two users separately and fuses them into one broadcast representation, with the fusion weights adapting to semantic similarity and the CSI pair $K$0 (Wu et al., 2024). Federated latent-space alignment introduces a semantic pre-equalizer $K$1 at the access point and local semantic equalizers $K$2 at users to bridge different latent spaces under a power constraint (Poce et al., 19 Feb 2026).

The architectural diversity is substantial.

Framework	Communication setting	Core mechanism
Swin-based heterogeneous broadcast (Nguyen et al., 2024)	Downlink BS to $K$3 users	Shared BS JSCC; user-specific decoders
MU-DeepSC / DeepSC-VQA (Xie et al., 2021)	Multi-user multimodal uplink	Joint image/text encoding and MAC fusion
DMCE (Zeng et al., 2024)	Multi-user MIMO semantic fusion	Diffusion-enhanced CSI estimation
DBC semantic fusion (Wu et al., 2024)	Two-user degraded broadcast	Semantic feature fusion with CSI embedding
Federated latent alignment (Poce et al., 19 Feb 2026)	Downlink MIMO broadcast	AP pre-equalizer and local semantic equalizers

This architectural spread suggests that MU-SemDaCom is better understood as a design space than as a single protocol stack. The commonality lies in end-to-end optimization around semantic objectives, not in a fixed transmitter or multiple-access structure.

3. Semantic representations, neural backbones, and distortion measures

The semantic representation in MU-SemDaCom is typically a learned latent vector or tensor produced before physical-layer transmission. In the Swin-based downlink system, the base station semantic encoder is a four-stage Swin Transformer with stage block counts $K$4, while high-computing and low-computing users employ different Swin-decoder depths, namely $K$5 and $K$6 (Nguyen et al., 2024). A related dynamic multi-user design also uses a modified Swin Transformer at the base station, but conditions the semantic encoder on user computing capacity and network status via target embedding and dynamic CBR selection (Nguyen et al., 2023).

Other systems use modality-specific backbones. MU-DeepSC adopts the first 30 residual blocks of ResNet-101 for image semantic encoding, one Bi-LSTM for text semantic encoding, four CNN layers for image channel encoding and decoding, five dense layers for text channel decoding, and a Memory-Attention-Composition network for multimodal reasoning (Xie et al., 2021). The unified Transformer-based DeepSC family uses Transformer encoders for both text and image semantics, then small fully connected JSC encoders and decoders for transmission (Xie et al., 2021). DMCE uses a pretrained ResNet-50 semantic encoder, a JSCC encoder for feature transmission, and an attention U-Net diffusion model to enhance CSI (Zeng et al., 2024). Generative MU-SemDaCom for OFDMA replaces exhaustive transmission with a diffusion-based decoder that regenerates missing null-space components from partial range-space observations (Grassucci et al., 2024).

The choice of objective reveals the intended semantics. Reconstruction-centric works minimize image-domain distortion such as

$K$7

or use MSE, PSNR, MS-SSIM, SSIM, LPIPS, and FID (Nguyen et al., 2024, Grassucci et al., 2024). Task-oriented systems optimize VQA cross-entropy, translation cross-entropy, metric-learning losses for retrieval, answer accuracy, BLEU, or classification accuracy (Xie et al., 2021, Xie et al., 2021). Segmentation-oriented MU-SemDaCom evaluates mIoU over 9 classes (Zeng et al., 2024). Reasoning-aware formulations define semantic information, semantic reliability, and effective semantic information as the product of reliability and semantic information (Singh et al., 2024). Mixed semantic/data systems fit logistic models for end-to-end distortions and minimize weighted-sum distortion across heterogeneous users (Yuan et al., 29 Sep 2025).

This variety underscores that “semantic fidelity” is not a single metric. It is task-dependent, and the semantic representation is defined operationally by the decoder and loss.

4. Training regimens and optimization methodologies

Training in MU-SemDaCom is often staged rather than fully monolithic. The heterogeneous-user Swin framework first pretrains a high-computing user as teacher, freezes all base-station and teacher-decoder parameters, and then trains a low-computing user as student with partial transfer learning and knowledge distillation using

$K$8

(Nguyen et al., 2024). A related dynamic system trains end-to-end under varying SNR and CBR, using a hybrid loss

$K$9

to improve perceptual quality (Nguyen et al., 2023).

Task-oriented models also use multi-stage schedules. DeepSC-IR first trains the semantic encoder with a retrieval loss and then trains the JSC codec with MSE; DeepSC-MT and DeepSC-VQA add an end-to-end fine-tuning phase after semantic and JSC pretraining (Xie et al., 2021). MU-DeepSC instead trains the full multimodal chain with VQA cross-entropy, with channel corruption injected in the forward pass over AWGN, Rayleigh, or Rician channels (Xie et al., 2021). DMCE employs a three-stage schedule: semantic and JSCC modules under an ideal channel, then diffusion-model training on CSI, then JSCC-decoder and semantic-decoder fine-tuning with the diffusion-enhanced channel in the loop (Zeng et al., 2024).

Optimization becomes more elaborate when semantic communication is coupled to resource allocation or heterogeneous latent spaces. Federated latent alignment solves a block-convex problem over a shared pre-equalizer and local equalizers by a federated ADMM scheme with closed-form user updates and AP-side aggregation (Poce et al., 19 Feb 2026). MU-ASCC relaxes a mixed-integer nonconvex problem and alternates between adaptive rate allocation by subgradient descent and joint power/beamforming by uplink–downlink duality with SCA (Yuan et al., 29 Sep 2025). JSRE recasts joint scheduling, RIS phase control, and semantic compression as an MDP, then uses truncated DRL, a semantic similarity estimator, and semantic model caching to reduce retraining overhead (Wang et al., 22 Mar 2026).

The literature therefore does not converge on a single optimization paradigm. End-to-end SGD, staged teacher–student transfer, federated linear-algebraic updates, AO, SCA, PPO-style DRL, and game-theoretic equilibria all appear, depending on whether the dominant difficulty is representation learning, interference management, resource allocation, or semantic alignment.

5. Multiple access, interference management, and semantic resource allocation

A central problem in MU-SemDaCom is how to separate, prioritize, or deliberately fuse users’ semantic streams. NOMASC addresses non-orthogonal transmission by combining asymmetric feature quantization, learned symbol mapping, and neural multi-user detection in a two-user downlink NOMA setting, avoiding iterative SIC and approaching OMA performance across image–image and image–text pairings (Li et al., 2023). NOC4SC instead assigns each user a user-specific codeword and applies an NSM block that injects both codeword and SNR conditioning to carve out approximately orthogonal feature subspaces without bandwidth spreading (Wang et al., 10 Dec 2025).

Other works reduce transmitted information by exploiting commonality or generative completion. Group-wise semantic splitting multiple access clusters users by semantic similarity, extracts group-level common features and user-specific private features, multicasts the common features, unicasts the private features, and regularizes the common embeddings with a repulsion loss (Koh et al., 26 Nov 2025). The generative OFDMA framework transmits only the range component associated with each user’s strongest subcarriers and asks a diffusion decoder to regenerate the null-space component, reframing multi-user OFDMA as an inverse problem (Grassucci et al., 2024). Degraded broadcast semantic fusion similarly avoids traditional time or power partitioning by learning a semantic fusion rule that balances the two users’ PSNRs (Wu et al., 2024).

Resource allocation is likewise semanticized. A semantic relay framework maximizes a multiuser weighted sum-rate by jointly designing relay power and bandwidth allocation, under an information-causality constraint in which the relay cannot forward more than it receives semantically from the base station (Hu et al., 2023). Hypergame-based decentralized control lets transmitters choose bit allocations and receivers choose local reasoning, cloud reasoning, drop, or accept-decoded actions while accounting for misperceptions about others’ strategies (Thomas et al., 2024). Reasoning-aware MU-SemDaCom explicitly trades communication against computing: receivers may reason missing semantics locally when links are unavailable, subject to CPU and bit budgets (Singh et al., 2024).

Interference itself can be treated semantically rather than solely physically. Enhanced OMDMA uses a single universal DeepJSCC model, assigns each user a unique shuffling pattern, and relies on shuffle-based orthogonalization so that non-target users’ structured semantic interference appears Gaussian-like after inverse mapping and de-shuffling; diffusion denoising then mitigates the resulting effective noise (Zhang et al., 28 Jul 2025). This suggests a broader shift in MU-SemDaCom: interference management is increasingly performed in latent space, by codewords, permutations, diffusion priors, or semantic grouping, rather than only by classical orthogonalization.

6. Empirical behavior, trade-offs, and open issues

The empirical record is heterogeneous but consistent in one respect: multi-user semantic designs improve performance when semantics, channel effects, and user heterogeneity are learned jointly. In the teacher–student Swin broadcast system, at 3 dB SNR the low-computing decoder reaches about 31 dB PSNR when trained alone, about 33 dB with iterative training, about 36 dB with the freeze-encoder procedure, about 38 dB with transfer learning, and about 37 dB with knowledge distillation; across SNRs, transfer learning and distillation maintain at least 3 dB gains over the iterative baseline (Nguyen et al., 2024). In MU-DeepSC for VQA, answer accuracy remains above 80% down to about 0 dB on AWGN, whereas separate source–channel coding requires SNR above 10 dB to match; the symbol count per image is 12,544 versus 41,718, and the reported complexity is about $M$0 versus $M$1 multiply–accumulates (Xie et al., 2021).

The same pattern appears in robustness-oriented designs. DMCE improves mIoU by more than 25% at low SNR regimes and reduces CSI NMSE by about 14 dB on average across SNRs (Zeng et al., 2024). Reasoning-aware MU-SemDaCom reports at least a $M$2 enhancement in throughput and substantially higher reliability than classical systems that do not incorporate reasoning (Singh et al., 2024). MU-ASCC, in a two-user MU-MISO setting with one data user and one semantic user, reduces weighted distortion by 39.9% versus ZF-BPG and 29.6% versus DJSCC at 8 dBW and $M$3 (Yuan et al., 29 Sep 2025). Group-wise semantic splitting reports up to 3.26% performance improvement over conventional schemes (Koh et al., 26 Nov 2025).

The trade-offs are equally clear. Perfect or accurate CSI is assumed in several systems, and practical channel-estimation errors remain a stated limitation for generative OFDMA and other resource-allocation schemes (Grassucci et al., 2024). Receiver complexity and latency remain significant for diffusion-based decoders, including explicit mention of $M$4 diffusion steps in DMCE and generative OFDMA (Zeng et al., 2024, Grassucci et al., 2024). Scaling beyond small user counts remains challenging for NOMA-style or codeword-based systems because interference and approximate orthogonality deteriorate as the number of users rises (Li et al., 2023, Wang et al., 10 Dec 2025). Latent-space mismatch across AI-native devices is sufficiently severe that a separate line of work introduces semantic pre-equalization and federated alignment to restore downstream accuracy (Poce et al., 19 Feb 2026).

A final misconception is that semantic communication eliminates the need for conventional resource allocation. The evidence points in the opposite direction: semantic relevance introduces new control variables rather than removing old ones. Compression ratio, semantic pilot overhead, bit budgets, CPU budgets, RIS phase shifts, power allocation, bandwidth allocation, clustering, common/private semantic splitting, and user scheduling all remain explicit optimization variables in current MU-SemDaCom research (Poce et al., 19 Feb 2026, Hu et al., 2023, Wang et al., 22 Mar 2026). The field therefore sits at the intersection of JSCC, multi-user detection, generative modeling, distributed optimization, and task-aware network control, rather than replacing those areas with a single universal abstraction.