DeepJSCC: Deep Learning Joint Source-Channel Coding

• DeepJSCC is a communication framework that jointly learns source and channel coding using neural networks, eliminating the need for an intermediate bitstream.
• It replaces traditional layered architectures with adaptive autoencoders capable of handling finite blocklength, latency, and bandwidth constraints for robust image transmission.
• Key features include SNR-adaptive modules, progressive transmission, and integration of differentiable channel models to achieve semantic fidelity over diverse wireless channels.

Searching arXiv for recent DeepJSCC papers to ground the article with up-to-date references. Deep Learning-based Joint Source-Channel Coding (DeepJSCC) denotes a family of end-to-end learned communication systems that map source signals directly to channel symbols and reconstruct them from noisy observations without using a bitstream as the mandatory intermediate representation. In the image-transmission setting most commonly studied in the literature, a neural encoder produces complex-valued channel symbols, a differentiable channel layer injects the wireless impairment model, and a neural decoder reconstructs the image under a distortion or semantic objective. Within semantic communications, this shifts design emphasis from reliable bit transport toward task-relevant or perceptually meaningful reconstruction under finite blocklength, latency, bandwidth, and power constraints. Recent work has expanded this paradigm from fixed point-to-point autoencoders toward adaptive, generative, secure, interoperable, and deployable systems over OFDM, MIMO, relay, distributed, and time-varying channels (Xu et al., 2022, Raha et al., 28 Jul 2025).

1. Conceptual foundation

Classical communication architectures follow Shannon’s separation theorem, decomposing the pipeline into source coding, channel coding, and modulation. The surveyed literature emphasizes that this architecture is optimal only in the asymptotic regime of infinite blocklength and unlimited complexity; in practical regimes it can incur latency, large bandwidth use, iterative decoding complexity, and the cliff effect, whereby performance collapses once channel quality drops below the correction capability of the code (Xu et al., 2022).

DeepJSCC replaces this layered design with an end-to-end mapping from a source signal $\boldsymbol{x}\in\mathbb{R}^n$ to channel symbols $\boldsymbol{z}\in\mathbb{C}^k$ and then to a reconstruction $\hat{\boldsymbol{x}}$. A standard definition introduces source bandwidth $n$, channel bandwidth $k$, and bandwidth ratio $\rho \triangleq k/n$. The essential departure from separation is that DeepJSCC does not transmit bits as the common intermediate representation; instead, it jointly learns source and channel representations from data and can be trained against application losses rather than only bit-recovery objectives (Xu et al., 2022).

This formulation is particularly aligned with semantic communications. The literature frames semantic communication as the delivery of task-relevant information rather than exact source reproduction, especially for applications such as industrial robotics, autonomous driving, drone networks, and VR/metaverse. In such settings, DeepJSCC is attractive because it operates well at finite blocklengths, can optimize semantic or perceptual criteria such as LPIPS and MS-SSIM, and degrades gracefully as channel quality worsens rather than failing catastrophically (Xu et al., 2022).

2. Canonical transceiver and optimization formulations

A canonical DeepJSCC transceiver is an autoencoder with a differentiable channel layer in the bottleneck. In one common formulation, the encoder and decoder satisfy

$$z = f_\phi(x), \qquad y = z + n,\qquad \hat{x}=g_\theta(y),$$

with a power constraint such as $\|f_\phi(x)\|_2^2 \le k$ or an equivalent normalized average-power condition. In more general semantic formulations, the encoder and decoder may additionally take channel SNR $\gamma$, compression ratio $R$, and architectural activation variables as conditions, and the channel may be written as $\boldsymbol{z}\in\mathbb{C}^k$0 with normalized-gain special case $\boldsymbol{z}\in\mathbb{C}^k$1 (Lee et al., 2023, Raha et al., 28 Jul 2025).

Training is typically end-to-end by minimizing reconstruction distortion. MSE remains the standard loss in many image-transmission papers, but the surveyed literature explicitly states that DeepJSCC can be trained with application-specific losses, including PSNR-related distortion, LPIPS, MS-SSIM, BLEU-based objectives, or composite losses. In MIMO settings with contextual symbol estimation, the total loss can combine image reconstruction and an auxiliary estimation term weighted by $\boldsymbol{z}\in\mathbb{C}^k$2, and the ICL-based estimation loss is reported to improve convergence (Xu et al., 2022, Hua et al., 1 Dec 2025).

A recurring architectural point is that the channel model is incorporated during training as a fixed but differentiable operator. This enables backpropagation through AWGN, fading, OFDM signal processing blocks, clipping nonlinearities, and equalization submodules. The result is not merely a learned decoder atop a conventional modem; it is a jointly optimized source–channel mapping whose latent representation is shaped by the target wireless impairment model itself (Yang et al., 2021, Yang et al., 2021).

3. Adaptive and dynamic DeepJSCC

A major development has been the move from fixed operating points to adaptive DeepJSCC. One line of work introduces SNR-adaptive DeepJSCC with feature learning modules, attention feature modules, a decision module, and a required task performance module. The attention feature module fuses intermediate features with CSI and generates an attention mask so that poor channels emphasize only the most robust features, while good channels allow more detailed transmission. The same framework also supports bandwidth adaptation by dividing the latent vector into multiple layers and randomly transmitting the first $\boldsymbol{z}\in\mathbb{C}^k$3 layers during training, thereby inducing a successive-refinement behavior (Xu et al., 2022).

Bandwidth-agile and progressive variants make this refinement explicit. DeepJSCC-$\boldsymbol{z}\in\mathbb{C}^k$4 studies successive refinement and multiple descriptions over $\boldsymbol{z}\in\mathbb{C}^k$5 layers using convolutional autoencoders. The literature reports three architectures—multiple decoder, single decoder, and residual transmission—and states that DeepJSCC-$\boldsymbol{z}\in\mathbb{C}^k$6 can learn progressive transmission with negligible loss relative to a single transmission, while also claiming the first practical multiple-description JSCC scheme developed and tested for practical information sources and channels. In two-layer settings, the second-layer reconstruction is reported to be about $\boldsymbol{z}\in\mathbb{C}^k$7–$\boldsymbol{z}\in\mathbb{C}^k$8 dB better in PSNR than the first layer, and natural images encoded with DeepJSCC over Gaussian channels are described as almost successively refinable (Kurka et al., 2020, Kurka et al., 2019).

Time-varying channels motivate a stronger form of adaptation. DRJSCC divides encoded symbols into multiple blocks and, when the channel changes, re-encodes the remaining untransmitted blocks using instantaneous CSI. The RC module performs weight correction, dispersing, reconstruction, aggregating, and residual-style integration. Reported experiments on CIFAR-10 show that DRJSCC matches mainstream models under stable conditions, outperforms fixed-SNR DeepJSCC by about $\boldsymbol{z}\in\mathbb{C}^k$9 dB even when training and test SNR match, and is markedly more robust when SNR changes once, twice, or every block (Pan et al., 2023).

Dynamic architecture reconfiguration extends adaptivity into the network itself. DD-JSCC generalizes the single fixed encoder–decoder into a single dynamically reconfigurable model whose layer structure adapts in real time to transmitter and receiver computational capabilities, power constraints, compression ratios, and channel conditions. Its hierarchical layer activation enforces nested architectures, reducing the number of possible configurations from exponential to linear, while sequential randomized layer selection acts as an implicit regularizer. Reported results show up to $\hat{\boldsymbol{x}}$0 dB PSNR improvement over fixed Deep-JSCC architectures and training-cost reduction by over $\hat{\boldsymbol{x}}$1, with one dynamic model replacing six separately trained fixed configurations (Raha et al., 28 Jul 2025).

Computational adaptivity has also been studied at the feature level. FAJSCC classifies features into important and less important groups, applies self-attention to the former and spatial attention to the latter, and uses an importance ratio $\hat{\boldsymbol{x}}$2 to control computational budget. The model allows independent adjustment of encoder and decoder complexity within one trained model. Reported experiments indicate higher image-transmission performance than recent state-of-the-art models with lower GFLOPs and memory than SwinJSCC, and identify the decoder’s error-correction function as requiring the largest computational complexity in FAJSCC (Choi et al., 7 Apr 2025).

4. Channel, topology, and network generalizations

DeepJSCC has been extended well beyond AWGN point-to-point links. OFDM-guided DeepJSCC incorporates differentiable OFDM modulation/demodulation, cyclic prefix handling, multipath fading, pilot processing, explicit channel estimation, MMSE equalization, and clipping into the end-to-end graph. The EXPLICIT decoder design combines classical signal processing with learnable residual refinement subnets and is reported to outperform both direct transmission and fully implicit decoding. Over multipath fading channels, the method yields a reported $\hat{\boldsymbol{x}}$3–$\hat{\boldsymbol{x}}$4 dB SNR gain for equivalent image quality compared with BPG+LDPC baselines, improves further with CSI feedback, and remains robust to clipping and channel-model mismatch (Yang et al., 2021, Yang et al., 2021).

In MIMO systems, DeepJSCC has been integrated with transformer-based in-context learning. The ICL denoiser is a decoder-only transformer with a GPT-2 backbone that receives pilot pairs and the query output as context and predicts transmitted symbols, while channel-related context is also injected into the DeepJSCC encoder and decoder. The framework addresses both open-loop and closed-loop MIMO, extends to transmitter and receiver IQ imbalance, and reports that the ICL denoiser significantly outperforms least-squares estimation. Under severe IQ imbalance in one reported case at $\hat{\boldsymbol{x}}$5 dB, the LS MSE is about $\hat{\boldsymbol{x}}$6, whereas the ICL denoiser MSE is about $\hat{\boldsymbol{x}}$7 (Hua et al., 1 Dec 2025).

Networked settings have produced several further generalizations. For cooperative relays, DeepJSCC-AF, DeepJSCC-DF, and DeepJSCC-PF have been proposed for half-duplex operation, with PF achieving similar performance to DF at lower computational complexity; later work extends PF to transformer-based half- and full-duplex relays with block-based transmission and adaptive transmission modules, and reports superior performance to BPG operating at the maximum achievable rate of conventional decode-and-forward and compress-and-forward protocols (Bian et al., 2022, Bian et al., 2024). For correlated image sources, distributed DeepJSCC uses lightweight edge encoders and a powerful center decoder with an SNR-aware cross-attention module to exploit overlap between stereo images sent over independent noisy channels, improving both links by using the noisy representation from the other link (Wang et al., 2022).

5. Generative and iterative receivers

A substantial branch of the literature addresses the perceptual limitations of distortion-oriented DeepJSCC. DiffJSCC augments a conventional DeepJSCC front-end with a Stable Diffusion-based receiver. The receiver first reconstructs an intermediate image, extracts spatial features via the Stable Diffusion VAE encoder, textual features via BLIPv2 and the CLIP text encoder, and combines these with SNR through a control module that fine-tunes the diffusion process while keeping the Stable Diffusion UNet fixed. On Kodak images of size $\hat{\boldsymbol{x}}$8, the paper highlights reconstruction from only $\hat{\boldsymbol{x}}$9 complex symbols, corresponding to $n$0 symbols per pixel at $n$1 dB SNR. Relative to plain DeepJSCC, DiffJSCC reports about a $n$2 dB drop in PSNR and about a $n$3 drop in MS-SSIM, but roughly a $n$4 reduction in LPIPS and a $n$5 reduction in FID (Yang et al., 2024).

SGD-JSCC similarly employs diffusion, but makes semantic side information explicit. It transmits latent JSCC features together with semantic features derived from text descriptions or edge maps, reconstructs the semantic side information at the receiver, and uses it to guide a diffusion transformer that denoises the noisy JSCC latent before final decoding. The paper emphasizes pilot-free SNR estimation from the received latent in slow fading and a training-free denoising strategy for fast fading. Reported results indicate that at $n$6 dB the method still produces meaningful reconstructions, and under fast fading the LPIPS degradation relative to AWGN or slow fading is less than $n$7 (Zhang et al., 2 Jan 2025).

Generative decoders need not rely on diffusion. G-UNet-JSCC replaces the conventional decoder with a U-Net-based generator trained by a weighted sum of MSE and SSIM, while cGAN-JSCC adds adversarial training against a patch-based discriminator using a two-stage procedure. The reported findings are resolution-dependent: for low-resolution images cGAN-JSCC achieves better reconstruction performance and greater robustness to channel variations than G-UNet-JSCC, whereas for high-resolution images G-UNet-JSCC outperforms cGAN-JSCC in both PSNR and SSIM and avoids adversarial artifacts such as the semi-transparent artifact reported for cGAN-JSCC (Ye et al., 26 Feb 2026).

Iterative refinement offers a non-generative alternative. ISEC reinterprets DeepJSCC decoding as a modified MAP inference problem in codeword space and uses a bias-free CNN denoiser to approximate the gradient of the log-prior density. Starting from the noisy received codeword, the receiver iteratively updates the latent estimate using a data-consistency term and a denoiser-based prior term. The method improves distortion and perceptual metrics over one-shot decoding and remains more reliable under SNR mismatch and even when a model trained on AWGN is evaluated under additive white Laplacian noise (Lee et al., 2023).

6. Security, interoperability, and real-world deployment

Standard DeepJSCC exposes a security weakness because the learned channel symbols remain correlated with the source. DeepJSCEC addresses this by inserting quantization, LWE-based public-key encryption, modulation, decryption, and soft quantization between the encoder and decoder while preserving end-to-end training through the differentiable components. The design targets security against chosen-plaintext attacks without assumptions on the eavesdropper’s channel condition. Reported CIFAR-10 results give Bob PSNR $n$8 dB and SSIM $n$9, while Eve achieves PSNR $k$0 dB and SSIM $k$1, close to a random-average image baseline of PSNR $k$2 dB and SSIM $k$3; in remote classification, Bob reaches $k$4 accuracy, Eve $k$5, and the random baseline $k$6 (Tung et al., 2022).

Interoperability introduces a different impairment: semantic mismatch between heterogeneous latent spaces. Semantic channel equalization studies multi-vendor deployments in which transmitter and receiver DeepJSCC models are not jointly trained. It introduces linear equalizers, lightweight neural aligners, and a zero-shot Parseval-frame equalizer to align the transmitter latent space to the receiver’s expected latent geometry. The paper reports that CNN-based aligners achieve near-ideal PSNR with very few pilots, while the Parseval-frame equalizer can reach around $k$7 dB PSNR at high SNR with only about $k$8 coefficients, providing a zero-shot deployment option (Pannacci et al., 6 Oct 2025).

Real-world prototyping has begun to test whether such systems remain advantageous outside simulation. DeepStream implements OFDM-compatible DeepJSCC for real-time image transmission and video streaming on USRP X310 software-defined radios. It introduces feature-to-symbol mapping, cross-subcarrier precoding, and progressive coding under latency constraints, and reports that 16-bit floating-point deployment performs nearly identically to 32-bit. At $k$9 dB SNR, the prototype achieves a PSNR of $\rho \triangleq k/n$0 dB for image transmission and an MS-SSIM of $\rho \triangleq k/n$1 dB for video streaming, whereas the standard separated scheme fails to recover meaningful information (Chi et al., 7 Sep 2025).

7. Open problems and directions

The literature explicitly identifies several unresolved problems. A 2022 survey lists security, universal architectures that span multiple source modalities and channel types, optical/visible-light/underwater/satellite channels, multi-user JSCC with interference management, OFDM peak-to-average power ratio, and efficient training of larger models as open challenges (Xu et al., 2022).

Subsequent work sharpens these concerns rather than closing them. Dynamic architectures and feature-importance control indicate that a single fixed model is often not the right abstraction for heterogeneous devices and operating conditions (Raha et al., 28 Jul 2025, Choi et al., 7 Apr 2025). Diffusion-based receivers achieve strong perceptual gains but introduce heavier receiver computation and rely on auxiliary semantic or generative priors whose failure modes are different from those of distortion-optimized autoencoders (Yang et al., 2024, Zhang et al., 2 Jan 2025). Interoperability work shows that semantic noise from latent mismatch is a first-class impairment in heterogeneous deployments, not merely an implementation inconvenience (Pannacci et al., 6 Oct 2025). Prototype results suggest that waveform compatibility, PAPR control, latency adaptation, and quantized inference are central deployment constraints rather than secondary engineering details (Chi et al., 7 Sep 2025).

Taken together, these results suggest that DeepJSCC has evolved from a fixed AWGN image autoencoder into a broader research program on learned communication systems. Its present scope includes adaptive architectures, contextual estimation, cooperative and distributed topologies, perceptual generation, iterative inference, cryptographic protection, latent-space alignment, and over-the-air implementation. A plausible implication is that future progress will depend less on demonstrating that end-to-end learning can outperform separated baselines in selected regimes and more on reconciling semantic fidelity, channel robustness, interoperability, security, and deployment complexity within unified system designs.