AI/ML Joint Source-Channel Coding

• AI/ML-based Joint Source-Channel Coding is a deep learning approach that fuses source coding, channel coding, and modulation to optimize performance over non-ideal channels.
• Recent advances show improved SNR gains and graceful quality degradation, outperforming classical separation schemes in dynamic, finite blocklength scenarios.
• Modern architectures leverage convolutional and transformer models with quantization and attention mechanisms to achieve adaptable, secure, and resource-efficient communication systems.

AI/ML-Based Joint Source-Channel Coding

AI/ML-based joint source-channel coding (AI/ML-JSCC) refers to the class of communication system designs in which deep neural networks, or more broadly machine learning models, learn end-to-end mappings from source data (e.g., images, audio) to physical channel inputs and back, integrating the functions of source coding, channel coding, and modulation. This concept departs from the traditional separation principle that dominates practical digital communications, instead leveraging statistical learning to optimize fidelity and robustness for finite blocklengths, non-ideal channels, and practical constraints. Recent progress, notably under the DeepJSCC paradigm, has demonstrated compelling gains over separate source-channel coding (SSCC) baselines in wireless imaging, semantic communications, multi-user transmission, and beyond, including graceful quality degradation under adverse channel conditions and substantial implementation flexibility (Gündüz et al., 2024, Xu et al., 2022, Tung et al., 2021, Xie et al., 2024, Tung et al., 2022).

1. Fundamental Principles and Motivation

Classically, the source-channel separation theorem asserts that, asymptotically, compression (source coding) and error protection (channel coding) can be performed independently without loss in optimality, provided source and channel statistics are stationary and blocklengths infinite. The source code minimizes distortion at a given rate $R(D)$; the channel code achieves capacity $C$. For transmission rate $r$, reliable communication is guaranteed if $rR(D) \leq C$, for i.i.d. memoryless settings (Gündüz et al., 2024). However, in practical communication scenarios—finite blocklengths, stringent latency, non-ergodic and fading channels, multi-user or broadcast scenarios, and tight resource constraints—the separation theorem does not hold strictly and can result in suboptimal performance, "cliff effects," or inefficient use of bandwidth and power.

AI/ML-JSCC re-examines the JSCC problem by parameterizing the encoder and decoder mappings as deep neural networks, which are trained end-to-end over the entire communication channel (including noise, fading, and even synchronization artifacts) to minimize expected task-specific distortion under power and constellation constraints. This approach generalizes symbol-by-symbol uncoded and hybrid analog-digital mappings, exploits data and channel structure, directly optimizes for target metrics (PSNR, SSIM, semantic loss), and adapts to non-ideal conditions beyond the analytic reach of conventional codes (Gündüz et al., 2024, Xu et al., 2022, Tung et al., 2021).

2. Canonical Architectures and Quantization Mechanisms

AI/ML-JSCC encoders and decoders are most commonly implemented as convolutional NN autoencoders for images, but increasingly as transformer-based models for increased scalability and global semantic modeling (Yang et al., 2023). A typical transmitter mapping is

$$f_{\theta} : \mathbb{R}^{H\times W\times C} \rightarrow \mathbb{C}^k,\ \text{with}\quad s = f_{\theta}(X)$$

where $X$ is an image source, $s$ is a latent vector to be mapped to channel. The decoder is

$$g_{\phi} : \mathbb{C}^k \rightarrow \mathbb{R}^{H\times W\times C}$$

with $g_{\phi}(Y)$ reconstructing the source from the noisy observation $Y = s + N$.

Practical deployments require channel inputs to be restricted to a finite constellation (e.g., $Q$-QAM due to hardware modulator constraints). DeepJSCC-Q, for instance, enforces

$$Q = \{c_1, ..., c_M\} \subset \mathbb{C},\qquad \bar{s}_i = \arg\min_{c \in Q} |s_i - c|$$

incorporating a soft-to-hard quantization (with temperature annealing) to retain differentiability for SGD-based training. This quantization is handled by using a soft assignment during the backward pass for gradients:

$$\tilde{s}_i = \sum_{j=1}^M \alpha_{ij} c_j,\qquad \alpha_{ij} \propto \exp(-\sigma_q |s_i - c_j|^2)$$

Gradients propagate using this relaxation, while actual transmitted symbols remain strictly in $Q^k$ (Tung et al., 2021, Tung et al., 2022).

Modern architectures augment the basic autoencoder with:

• Residual and attention modules for spatial adaptivity and SNR-aware bit allocation
• Transformer-based backbones for high-resolution and universal semantic coding (Yang et al., 2023)
• Hypernetworks for on-the-fly adaptation to varying SNR/channel state, parameterizing network weights as functions of instantaneous channel conditions (Xie et al., 2024)
• Modular quantization and KL-divergence regularizers to promote full exploration of the channel alphabet and to meet power constraints (Tung et al., 2021, Tung et al., 2022)

3. Optimization Objectives and Training Protocols

The standard end-to-end learning objective is to minimize reconstruction distortion

$$\min_{\theta, \phi} \mathbb{E}_{X, N}[d(X, \hat{X})]$$

where $(\theta, \phi)$ parameterize the encoder and decoder, $d(\cdot,\cdot)$ is typically MSE, $1- \mathrm{SSIM}$, negative LPIPS, or a downstream task-specific loss (e.g. classification cross-entropy, BLEU score for text) (Xu et al., 2022, Gündüz et al., 2024). Training is performed with stochastic gradient descent (Adam or similar) over large, representative datasets (CIFAR-10, Kodak, ImageNet) and across randomized channel conditions spanning the anticipated SNR range.

For quantized channel constraints or discrete latents, the straight-through estimator is widely employed: the forward pass is hard projection; the backward pass uses soft/differentiable relaxation for efficient gradient propagation. Annealing schedules for quantization sharpness and KL-regularization are crucial for stable convergence and full symbol utilization (Tung et al., 2022, Tung et al., 2021). For time-varying/fading channels or mission-critical settings (e.g. LEO satellites), joint training is performed over Markov multi-state fading models and attention modules are used to ensure adaptation without per-state model overhead (Kondrateva et al., 2024, Kondrateva et al., 2024).

Emerging methods extend autoencoder-based schemes to multi-user non-orthogonal transmission via user-specific learnable projections (DeepJSCC-PNOMA), to secure communications via lattice-based encryption in the latent domain (Tung et al., 2022), and to per-instance minimal-code overfitting (Implicit-JSCC) for ultra-low decoding complexity (Wu et al., 24 Dec 2025).

4. Performance Benchmarks and Empirical Behavior

AI/ML-JSCC models are consistently benchmarked against the separation pipeline: modern source coding (BPG, VTM) + capacity-approaching channel code (LDPC, polar) + standard modulation (QPSK, 16/64-QAM). Key findings include:

• Substantial SNR gains over digital separation: e.g., DeepJSCC-Q exceeds BPG+LDPC+QPSK at low-to-moderate SNR by up to 3 dB PSNR, with performance converging to unconstrained DeepJSCC as constellation order increases ($M>4096$) (Tung et al., 2021, Tung et al., 2022).
• Graceful quality degradation: unlike separation schemes (which show a "cliff effect" at SNR threshold), DeepJSCC architectures degrade signal quality smoothly with SNR, supporting robust streaming/real-time operation (Xu et al., 2022, Kondrateva et al., 2024, Gündüz et al., 2024).
• Semantic/task-oriented superiorities: when optimizing for feature or classification accuracy, DeepJSCC models outperform both separate coding and task-agnostic ML baselines, due to joint shaping of semantic information and unequal error protection (Xu et al., 2022, Xie et al., 2024).
• Adaptivity and efficiency: single models trained with conditioning or modulation networks (attention, hypernetworks) can generalize across wide SNR/rate spans, closely matching performance of per-SNR specialized designs, with sub-20 KB overhead (Xie et al., 2024, Yang et al., 2023).

Exemplar results:

Scheme	PSNR@SNR=10dB, ρ=1/12 (CIFAR-10)
BPG+LDPC QPSK	29.8 dB
DeepJSCC-Q (M=64)	31.2 dB
DeepJSCC-Q (L-64, q-learned)	31.7 dB
SwinJSCC (Transformer)	29.3 dB (Kodak, 1/16, 7 dB)

Robustness to channel mismatch, fading, clipping, and even multi-hop architecture, has been empirically established (Kondrateva et al., 2024, Bian et al., 2023, Yang et al., 2021).

5. Extensions: Multi-User, Security, and Modality-Agnostic JSCC

DeepJSCC architectures generalize to multiple access, distributed, and broadcast scenarios by learning user-specific projections and leveraging non-orthogonal multiplexing (NOMA), achieving high user scalability with negligible parameter growth and outperforming separation-based and classic ML NOMA baselines (Yilmaz et al., 23 Mar 2025). In secure communication, DeepJSCC coupled with IND-CPA-secure public-key encryption in the latent space (LWE-based) achieves strong cryptographic semantic security guarantees and outperforms BPG+AES+LDPC under chosen-plaintext attack (Tung et al., 2022). Instance-specific overfitting (Implicit-JSCC) realizes low-complexity, storage-free, universally modality-agnostic coding, surpassing both classical and deep learning-based JSCC for high SNR and streaming edge scenarios (Wu et al., 24 Dec 2025).

6. Practical Considerations and Implementation Insights

AI/ML-JSCC frameworks can be deployed on contemporary hardware platforms (CubeSats, edge GPUs, 6G software-defined radios) with real-time latency (<20 ms per image/video frame) and manageable resource footprints. Memory- and compute-efficient adaptation strategies (hypernetworks, attention) facilitate universal deployment across wide bandwidths, SNRs, and devices (Xie et al., 2024, Kondrateva et al., 2024). Prototype platforms such as DeepStream demonstrate that ML-JSCC surpasses classical coded OFDM at all SNRs in over-the-air testing on USRP software radios (Chi et al., 7 Sep 2025).

Trade-offs include:

• Model size versus performance: transformer/attention-based models improve coding gain for high-resolution or multi-modal data but incur higher parameter counts.
• Quantization complexity for large constellations, though moderate order (M=16/64) already approaches separation upper bounds.
• Adaptation granularity: SNR-/channel-conditioned single models avoid per-condition retraining at minimal storage cost (Xie et al., 2024, Kondrateva et al., 2024).
• For distributed JSCC with side information (Wyner–Ziv), AI/ML-JSCC achieves practical, low-latency, gracefully degrading transmission unobtainable by digital binning-based approaches (Yilmaz et al., 2023).

7. Open Challenges and Outlook

Despite rapid advances, significant research challenges and opportunities remain:

• Universal JSCC: extension to continuous adaptation over sources, channels, and tasks in a single universal model
• Robustness and security: provable protection against traffic analysis, adversarial attacks, and dynamic adaptation to channel and adversary uncertainties
• Efficient multi-user, MIMO, broadcast and relay JSCC architectures for dense networks and federated learning/data fusion scenarios (Yilmaz et al., 23 Mar 2025, Bian et al., 2023)
• Hardware-aware design for extreme edge devices and satellite applications (Kondrateva et al., 2024, Kondrateva et al., 2024)
• Joint optimization of semantic metrics and system objectives (e.g., end-to-end inference, latency, energy)

AI/ML-JSCC, evolving from the DeepJSCC lineage and extended to cover practical constellation, channel, and hardware constraints, constitutes a convergent, end-to-end paradigm with demonstrated superiority in all-in-one communication tasks under real-world conditions. Ongoing work is focused on standard-compliant, scalable, robust, and adaptive JSCC systems for 6G and beyond (Gündüz et al., 2024, Xu et al., 2022, Tung et al., 2021, Tung et al., 2022, Xie et al., 2024).