---
title: 'MU-SemDaCom: Multi-User Semantic Data Communication'
url: https://www.emergentmind.com/topics/multi-user-semantic-and-data-communication-mu-semdacom
type: topic
---

# MU-SemDaCom: Multi-User Semantic Data Communication

Searching arXiv for the cited MU-SemDaCom papers to ground the article in current records.
arxiv_search({"query":"2406.03773 Optimizing Multi-User Semantic Communication via Transfer Learning and Knowledge Distillation", "max_results": 5})
arxiv_search({"query":"multi-user semantic communication transfer learning knowledge distillation", "max_results": 10})
Multi-User Semantic and Data Communication (MU-SemDaCom) denotes a class of multi-user communication frameworks in which transmitters and receivers exchange semantic features, task-relevant representations, or mixed semantic/data streams rather than relying solely on conventional bit-centric transmission. In the cited literature, MU-SemDaCom appears in downlink broadcast, uplink fusion, OFDMA, MU-MISO, degraded broadcast channel, relay-aided, and reasoning-aware settings, with tasks including image reconstruction, visual question answering (VQA), image retrieval, machine translation, segmentation, and concurrent task execution plus data reconstruction [2406.03773], [2112.10255], [2509.24247]. Its defining premise is that communication, inference, and resource allocation are jointly designed around semantic objectives such as PSNR, SSIM, LPIPS, FID, answer accuracy, classification accuracy, mIoU, BLEU, or effective semantic information, rather than being treated as independent layers.

## 1. Conceptual scope and problem setting

MU-SemDaCom generalizes semantic communication from a single link to a networked setting with multiple users, heterogeneous devices, and often heterogeneous objectives. Representative formulations include a single base station broadcasting an image to $K$ users with differing computing capacities [2406.03773], a multi-user uplink semantic communication system with $K$ single-antenna users and one $M$-antenna fusion receiver [2112.10255], a downlink MU-MISO system serving both data users and semantic users over the same frequency band [2509.24247], and decentralized multi-user systems in which receivers may invoke reasoning modules when communication is interrupted [2406.15199].

The meaning of “semantic” varies by task. In reconstruction-oriented systems, semantics is the latent representation that supports image recovery under channel impairment, as in Swin-based JSCC broadcast and degraded broadcast semantic fusion [2406.03773], [2406.10556]. In task-oriented systems, semantics is the representation sufficient for downstream inference, as in MU-DeepSC for VQA, DeepSC-IR for image retrieval, and DeepSC-MT for machine translation [2108.07357], [2112.10255]. In mixed semantic/data settings, semantic users pursue a task such as classification while data users reconstruct raw source data, and both coexist under a common resource-allocation framework [2509.24247].

A recurrent misconception is that semantic communication is synonymous with perceptual reconstruction. The literature does not support that reduction. Some frameworks optimize only task execution via cross-entropy or answer accuracy [2108.07357], [2112.10255]; others optimize reconstruction fidelity [2406.03773], [2406.10556]; others explicitly combine task execution and reconstruction through a convex-combination objective based on mutual information [2510.20067]. Another misconception is that multi-user semantic communication must remain orthogonal at the physical layer. The published designs include NOMA-based superposition [2303.06597], non-orthogonal codewords [2512.09356], semantic fusion over degraded broadcast channels [2406.10556], and shuffle-based orthogonalization intended to make interference appear Gaussian-like after de-shuffling [2507.20477].

## 2. Canonical system models and architectural patterns

A common MU-SemDaCom pattern is joint source-channel coding at the transmitter and learned semantic decoding at the receiver. In the downlink Swin-based framework, the base station uses a semantic encoder $E^S_\psi$ and channel encoder $E^C_\beta$ to map an image $I$ into a transmitted vector $X_I$, each user receives $Y_{I,k}=X_I+N_k$, and user $k$ reconstructs $\hat I_k=D^S_{\sigma_k}(D^C_{\gamma_k}(Y_{I,k}))$ [2406.03773]. In the multimodal VQA setting, different users transmit image and text features, a multi-antenna receiver performs detection and channel decoding, and a 12-cell MAC network fuses the recovered modalities into an answer distribution [2108.07357].

Several works depart from the basic point-to-point JSCC abstraction by embedding multi-user structure directly into the semantic layer. DMCE considers multiple users such as road cameras transmitting semantic features of multi-source data to a receiver, which equalizes the channel, recovers features, and fuses them into a segmentation map [2401.16017]. In degraded broadcast channels, the transmitter extracts semantic features for two users separately and fuses them into one broadcast representation, with the fusion weights adapting to semantic similarity and the CSI pair $(\mathsf{CSI}_1,\mathsf{CSI}_2)$ [2406.10556]. Federated latent-space alignment introduces a semantic pre-equalizer $F$ at the access point and local semantic equalizers $G_l$ at users to bridge different latent spaces under a power constraint [2602.17271].

The architectural diversity is substantial.

| Framework | Communication setting | Core mechanism |
|---|---|---|
| Swin-based heterogeneous broadcast [2406.03773] | Downlink BS to $K$ users | Shared BS JSCC; user-specific decoders |
| MU-DeepSC / DeepSC-VQA [2108.07357] | Multi-user multimodal uplink | Joint image/text encoding and MAC fusion |
| DMCE [2401.16017] | Multi-user MIMO semantic fusion | Diffusion-enhanced CSI estimation |
| DBC semantic fusion [2406.10556] | Two-user degraded broadcast | Semantic feature fusion with CSI embedding |
| Federated latent alignment [2602.17271] | Downlink MIMO broadcast | AP pre-equalizer and local semantic equalizers |

This architectural spread suggests that MU-SemDaCom is better understood as a design space than as a single protocol stack. The commonality lies in end-to-end optimization around semantic objectives, not in a fixed transmitter or multiple-access structure.

## 3. Semantic representations, neural backbones, and distortion measures

The semantic representation in MU-SemDaCom is typically a learned latent vector or tensor produced before physical-layer transmission. In the Swin-based downlink system, the base station semantic encoder is a four-stage Swin Transformer with stage block counts $(2,2,6,2)$, while high-computing and low-computing users employ different Swin-decoder depths, namely $(2,6,2,2)$ and $(2,2,2,2)$ [2406.03773]. A related dynamic multi-user design also uses a modified Swin Transformer at the base station, but conditions the semantic encoder on user computing capacity and network status via target embedding and dynamic CBR selection [2307.03402].

Other systems use modality-specific backbones. MU-DeepSC adopts the first 30 residual blocks of ResNet-101 for image semantic encoding, one Bi-LSTM for text semantic encoding, four CNN layers for image channel encoding and decoding, five dense layers for text channel decoding, and a Memory-Attention-Composition network for multimodal reasoning [2108.07357]. The unified Transformer-based DeepSC family uses Transformer encoders for both text and image semantics, then small fully connected JSC encoders and decoders for transmission [2112.10255]. DMCE uses a pretrained ResNet-50 semantic encoder, a JSCC encoder for feature transmission, and an attention U-Net diffusion model to enhance CSI [2401.16017]. Generative MU-SemDaCom for OFDMA replaces exhaustive transmission with a diffusion-based decoder that regenerates missing null-space components from partial range-space observations [2405.09866].

The choice of objective reveals the intended semantics. Reconstruction-centric works minimize image-domain distortion such as
$$
\mathcal{L}_{\text{data}}(I,\hat I_k)=\|I-\hat I_k\|_2^2
$$
or use MSE, PSNR, MS-SSIM, SSIM, LPIPS, and FID [2406.03773], [2405.09866]. Task-oriented systems optimize VQA cross-entropy, translation cross-entropy, metric-learning losses for retrieval, answer accuracy, BLEU, or classification accuracy [2108.07357], [2112.10255]. Segmentation-oriented MU-SemDaCom evaluates mIoU over 9 classes [2401.16017]. Reasoning-aware formulations define semantic information, semantic reliability, and effective semantic information as the product of reliability and semantic information [2406.15199]. Mixed semantic/data systems fit logistic models for end-to-end distortions and minimize weighted-sum distortion across heterogeneous users [2509.24247].

This variety underscores that “semantic fidelity” is not a single metric. It is task-dependent, and the semantic representation is defined operationally by the decoder and loss.

## 4. Training regimens and optimization methodologies

Training in MU-SemDaCom is often staged rather than fully monolithic. The heterogeneous-user Swin framework first pretrains a high-computing user as teacher, freezes all base-station and teacher-decoder parameters, and then trains a low-computing user as student with partial transfer learning and knowledge distillation using
$$
\mathcal{L}_{\text{train}}=\mathcal{L}_{\text{data}}(I,\hat I_2)+\alpha \mathcal{L}_{\text{distill}},
\quad
\mathcal{L}_{\text{distill}}=\|\hat I_1-\hat I_2\|_2^2
$$
[2406.03773]. A related dynamic system trains end-to-end under varying SNR and CBR, using a hybrid loss
$$
\mathcal{L}^{\text{Hyb}}=\gamma \mathcal{L}^{\text{ms-ssim}}+(1-\gamma)\mathcal{L}^{l1}+\epsilon \mathcal{L}^{l2}
$$
to improve perceptual quality [2307.03402].

Task-oriented models also use multi-stage schedules. DeepSC-IR first trains the semantic encoder with a retrieval loss and then trains the JSC codec with MSE; DeepSC-MT and DeepSC-VQA add an end-to-end fine-tuning phase after semantic and JSC pretraining [2112.10255]. MU-DeepSC instead trains the full multimodal chain with VQA cross-entropy, with channel corruption injected in the forward pass over AWGN, Rayleigh, or Rician channels [2108.07357]. DMCE employs a three-stage schedule: semantic and JSCC modules under an ideal channel, then diffusion-model training on CSI, then JSCC-decoder and semantic-decoder fine-tuning with the diffusion-enhanced channel in the loop [2401.16017].

Optimization becomes more elaborate when semantic communication is coupled to resource allocation or heterogeneous latent spaces. Federated latent alignment solves a block-convex problem over a shared pre-equalizer and local equalizers by a federated ADMM scheme with closed-form user updates and AP-side aggregation [2602.17271]. MU-ASCC relaxes a mixed-integer nonconvex problem and alternates between adaptive rate allocation by subgradient descent and joint power/beamforming by uplink–downlink duality with SCA [2509.24247]. JSRE recasts joint scheduling, RIS phase control, and semantic compression as an MDP, then uses truncated DRL, a semantic similarity estimator, and semantic model caching to reduce retraining overhead [2603.21097].

The literature therefore does not converge on a single optimization paradigm. End-to-end SGD, staged teacher–student transfer, federated linear-algebraic updates, AO, SCA, PPO-style DRL, and game-theoretic equilibria all appear, depending on whether the dominant difficulty is representation learning, interference management, resource allocation, or semantic alignment.

## 5. Multiple access, interference management, and semantic resource allocation

A central problem in MU-SemDaCom is how to separate, prioritize, or deliberately fuse users’ semantic streams. NOMASC addresses non-orthogonal transmission by combining asymmetric feature quantization, learned symbol mapping, and neural multi-user detection in a two-user downlink NOMA setting, avoiding iterative SIC and approaching OMA performance across image–image and image–text pairings [2303.06597]. NOC4SC instead assigns each user a user-specific codeword and applies an NSM block that injects both codeword and SNR conditioning to carve out approximately orthogonal feature subspaces without bandwidth spreading [2512.09356].

Other works reduce transmitted information by exploiting commonality or generative completion. Group-wise semantic splitting multiple access clusters users by semantic similarity, extracts group-level common features and user-specific private features, multicasts the common features, unicasts the private features, and regularizes the common embeddings with a repulsion loss [2511.21411]. The generative OFDMA framework transmits only the range component associated with each user’s strongest subcarriers and asks a diffusion decoder to regenerate the null-space component, reframing multi-user OFDMA as an inverse problem [2405.09866]. Degraded broadcast semantic fusion similarly avoids traditional time or power partitioning by learning a semantic fusion rule that balances the two users’ PSNRs [2406.10556].

Resource allocation is likewise semanticized. A semantic relay framework maximizes a multiuser weighted sum-rate by jointly designing relay power and bandwidth allocation, under an information-causality constraint in which the relay cannot forward more than it receives semantically from the base station [2311.06854]. Hypergame-based decentralized control lets transmitters choose bit allocations and receivers choose local reasoning, cloud reasoning, drop, or accept-decoded actions while accounting for misperceptions about others’ strategies [2409.17985]. Reasoning-aware MU-SemDaCom explicitly trades communication against computing: receivers may reason missing semantics locally when links are unavailable, subject to CPU and bit budgets [2406.15199].

Interference itself can be treated semantically rather than solely physically. Enhanced OMDMA uses a single universal DeepJSCC model, assigns each user a unique shuffling pattern, and relies on shuffle-based orthogonalization so that non-target users’ structured semantic interference appears Gaussian-like after inverse mapping and de-shuffling; diffusion denoising then mitigates the resulting effective noise [2507.20477]. This suggests a broader shift in MU-SemDaCom: interference management is increasingly performed in latent space, by codewords, permutations, diffusion priors, or semantic grouping, rather than only by classical orthogonalization.

## 6. Empirical behavior, trade-offs, and open issues

The empirical record is heterogeneous but consistent in one respect: multi-user semantic designs improve performance when semantics, channel effects, and user heterogeneity are learned jointly. In the teacher–student Swin broadcast system, at 3 dB SNR the low-computing decoder reaches about 31 dB PSNR when trained alone, about 33 dB with iterative training, about 36 dB with the freeze-encoder procedure, about 38 dB with transfer learning, and about 37 dB with knowledge distillation; across SNRs, transfer learning and distillation maintain at least 3 dB gains over the iterative baseline [2406.03773]. In MU-DeepSC for VQA, answer accuracy remains above 80% down to about 0 dB on AWGN, whereas separate source–channel coding requires SNR above 10 dB to match; the symbol count per image is 12,544 versus 41,718, and the reported complexity is about $3\times 10^8$ versus $1\times 10^9$ multiply–accumulates [2108.07357].

The same pattern appears in robustness-oriented designs. DMCE improves mIoU by more than 25% at low SNR regimes and reduces CSI NMSE by about 14 dB on average across SNRs [2401.16017]. Reasoning-aware MU-SemDaCom reports at least a $16.6\%$ enhancement in throughput and substantially higher reliability than classical systems that do not incorporate reasoning [2406.15199]. MU-ASCC, in a two-user MU-MISO setting with one data user and one semantic user, reduces weighted distortion by 39.9% versus ZF-BPG and 29.6% versus DJSCC at 8 dBW and $L=256$ [2509.24247]. Group-wise semantic splitting reports up to 3.26% performance improvement over conventional schemes [2511.21411].

The trade-offs are equally clear. Perfect or accurate CSI is assumed in several systems, and practical channel-estimation errors remain a stated limitation for generative OFDMA and other resource-allocation schemes [2405.09866]. Receiver complexity and latency remain significant for diffusion-based decoders, including explicit mention of $T=1000$ diffusion steps in DMCE and generative OFDMA [2401.16017], [2405.09866]. Scaling beyond small user counts remains challenging for NOMA-style or codeword-based systems because interference and approximate orthogonality deteriorate as the number of users rises [2303.06597], [2512.09356]. Latent-space mismatch across AI-native devices is sufficiently severe that a separate line of work introduces semantic pre-equalization and federated alignment to restore downstream accuracy [2602.17271].

A final misconception is that semantic communication eliminates the need for conventional resource allocation. The evidence points in the opposite direction: semantic relevance introduces new control variables rather than removing old ones. Compression ratio, semantic pilot overhead, bit budgets, CPU budgets, RIS phase shifts, power allocation, bandwidth allocation, clustering, common/private semantic splitting, and user scheduling all remain explicit optimization variables in current MU-SemDaCom research [2602.17271], [2311.06854], [2603.21097]. The field therefore sits at the intersection of JSCC, multi-user detection, generative modeling, distributed optimization, and task-aware network control, rather than replacing those areas with a single universal abstraction.

Source: https://www.emergentmind.com/topics/multi-user-semantic-and-data-communication-mu-semdacom