WITT: A Wireless Image Transmission Transformer for Semantic Communications (2211.00937v1)

Abstract: In this paper, we aim to redesign the vision Transformer (ViT) as a new backbone to realize semantic image transmission, termed wireless image transmission transformer (WITT). Previous works build upon convolutional neural networks (CNNs), which are inefficient in capturing global dependencies, resulting in degraded end-to-end transmission performance especially for high-resolution images. To tackle this, the proposed WITT employs Swin Transformers as a more capable backbone to extract long-range information. Different from ViTs in image classification tasks, WITT is highly optimized for image transmission while considering the effect of the wireless channel. Specifically, we propose a spatial modulation module to scale the latent representations according to channel state information, which enhances the ability of a single model to deal with various channel conditions. As a result, extensive experiments verify that our WITT attains better performance for different image resolutions, distortion metrics, and channel conditions. The code is available at https://github.com/KeYang8/WITT.

• The paper introduces WITT, a novel framework using Vision Transformers, specifically Swin Transformers and a Channel ModNet component, to optimize wireless semantic image transmission by adapting to channel conditions.
• Experimental results show WITT outperforms existing CNN-based methods in image fidelity (PSNR, MS-SSIM) across various channel conditions and bandwidths on standard datasets.
• WITT's success suggests Transformer architectures are highly promising for future adaptive and multi-modal semantic communication systems.

Wireless Image Transmission Transformer (WITT) for Semantic Communications

The paper proposes a novel framework, termed Wireless Image Transmission Transformer (WITT), for optimizing semantic image transmission by leveraging a Vision Transformer architecture. The primary contribution is adapting Transformer's Swin mechanism within semantic communication frameworks to surpass the inherent limitations of convolutional neural networks (CNNs) regarding global dependencies and efficiency in wireless image transmission tasks.

Overview of WITT

The foundation of WITT involves integrating Swin Transformers, known for efficient global dependency capture, as the backbone for semantic image transmission. In doing so, WITT aims at enhancing the JSCC performance for high-resolution images, where traditional CNN-based methods fall short due to rapid performance degradation with increased image dimensions.

Specifically, WITT distinguishes itself by introducing a spatial modulation approach that tailors the latent image representations concerning channel state information, thereby optimizing the transmission model's adaptability to diverse channel conditions. This adaptability is particularly vital for real-world applications demanding robust transmission across varying bandwidth and noise levels.

Core Architecture and Methods

WITT's workflow initiates with an input image divided into non-overlapping patches (tokens), which are processed through Swin Transformer blocks forming a hierarchy of feature maps. Swin Transformers' window-based self-attention mechanism offers scalability and linear complexity, advantageous for processing high-resolution data streams.

To adapt the Transformer architecture to fluctuating wireless channel conditions, the authors propose a supplementary component called Channel ModNet. This plug-in module dynamically modulates Transformer outputs in response to real-time Channel State Information (CSI), ensuring the model retains robustness against transmission impairments without necessitating frequent retraining.

Experimental Evaluation

Extensive experiments demonstrate WITT's superior performance over existing CNN-based JSCC frameworks and conventional separate source-channel coding paradigms. WITT is shown to significantly enhance Peak Signal-to-Noise Ratio (PSNR) and Multi-Scale Structural Similarity (MS-SSIM) metrics across diverse channel conditions including AWGN and Rayleigh fading.

Importantly, results on large datasets such as CIFAR10, Kodak, and CLIC2021 underscore WITT's ability to maintain image fidelity even at lower channel bandwidths, a critical factor for applications such as augmented reality and autonomous driving requiring high-resolution image transmission under stringent latency constraints.

Implications and Future Directions

The utilization of Transformer architectures, particularly Swin Transformers, within the wireless communication domain paves the way for future research into adaptive coding techniques viable for semantic communications. The promising results presented suggest potential extensions into multi-modal data transmission systems, where further optimization of spatial and temporal dependencies could be explored.

Furthermore, the paper highlights the importance of developing context-aware modulation mechanisms like Channel ModNet. Research into more granular modulation based on real-time feedback could yield advancements in model-based adaptation strategies, enabling seamless and efficient communication across increasingly congested wireless environments.

In summary, WITT demonstrates a significant advancement in semantic communication technology, setting a precedent for further exploration into the cross-disciplinary integration of deep learning innovations and communication systems.