Semi-Supervised Goal-Oriented Semantic Communication Framework for Foreground Classification

Abstract: Wireless goal-oriented semantic communication (GSC) has emerged as a promising paradigm by directly optimizing task performance. However, existing GSC frameworks typically operate on entire images and rely on labeled data for classification tasks, which can limit their compression efficiency and increase the risk of overfitting. This paper proposes a novel semi-supervised wireless GSC framework for the unlabeled image foreground classification task. In our proposed framework, a foreground-aware masked autoencoder (MAE) is developed to prioritize semantically important foreground objects, thereby reducing transmission overhead. To enable accurate reconstruction and classification under a limited data size, we further propose a semi-supervised autoencoder (SSAE) that decodes the semantic latent tensor and refines image details by leveraging three complementary information sources, followed by fine-tuning a pre-trained image classification model. The entire pipeline, from foreground masking to classification, is trained in a semi-supervised manner to significantly reduce the need for manual labeling. Simulation results validate that the proposed GSC framework achieves over 90% image classification accuracy while reducing the original image data size by 95%, and demonstrate its strong potential for practical tasks in resource-constrained wireless scenarios.

• The paper introduces a GSC architecture that combines ViT-based masked autoencoders with semi-supervised learning to extract and compress foreground semantic information.
• It optimizes transmission efficiency and accuracy by reducing data size by 95% and maintaining over 90% classification accuracy under low noise.
• The framework incorporates adaptive patch attention and palette-based quantization to balance bitrate reduction with image reconstruction quality.

Semi-Supervised Goal-Oriented Semantic Communication Framework for Foreground Classification

Overview

This paper presents a goal-oriented semantic communication (GSC) architecture that integrates semi-supervised learning and vision Transformer-based masked autoencoders (ViT-MAEs) to address the task of wireless image foreground classification under limited labeling. The framework optimizes classification performance and transmission efficiency by extracting and compressing semantically important foreground content, leveraging self-supervised and semi-supervised components to minimize labeled data dependency.

Proposed Framework and System Model

The GSC system processes images through a pipeline of modules at both the transmitter and receiver. The transmitter applies a ViT-based MAE to isolate foreground regions, which are then encoded using a semi-supervised autoencoder (SSAE). The resulting compressed latent is channel-encoded and transmitted over AWGN channels. At the receiver, decoding, reconstruction, and classification are performed using a symmetric framework, utilizing only a small fraction of labeled data for final classification.

Figure 1: The system pipeline: ViT-based detection of semantic foreground, encoding and wireless transmission, and semi-supervised (limited-label) classification after reconstruction.

The system model formalizes the end-to-end transformation from original image $\mathbf{I}$, through semantic encoding and quantization, wireless transmission, decoding, and image classification, optimizing the transmission of goal-relevant (foreground) information.

ViT-Based Foreground-Aware Masked Autoencoder

Foreground extraction is realized using a ViT-based MAE, which constructs semantic masks to remove background content, relying solely on unlabeled data. Images are patchified and converted into tokens using the ViT architecture, from which the "[CLS]" token encodes global semantics without position embedding. The method adopts a DINO-inspired teacher-student strategy: a fixed teacher network generates coarse semantic foreground masks, while a student model, trained with augmented views, learns to output teacher-consistent semantics. Loss minimization between teacher and student outputs ensures the student network distills foreground-aware representations robustly, even in the absence of labels.

Figure 2: MAE-processed image pairs, illustrating the original image versus the detected and isolated foreground.

Foreground extraction achieves a masking ratio where only approximately 49% of pixels are retained as semantically relevant, establishing a compressed image suitable for further semantic encoding and transmission.

Semi-Supervised Semantic Autoencoder with Fine Detail Refinement

To maximize compression and reconstruction quality, the framework introduces a CNN-based SSAE after foreground masking. The encoder reduces the image to a compact latent tensor, with quantization for transmission over noisy channels. The decoder reconstructs the image, leveraging the segmentation mask to focus on the foreground.

Patch attention, derived from ViT self-attention maps, is used to adaptively select high-attention patches that warrant per-pixel detail refinement. These are further compressed using palette-based quantization and RLE, efficiently coding both core and fine-grained visual details. The combination of semantic latent and refined patch stream achieves flexible fidelity control, optimizing the tradeoff between bitrate and visual quality/classification utility.

Classification is conducted by fine-tuning a ViT-based head on the reconstructed images, using only a limited subset of labeled data, minimizing manual annotation requirements and aligning with the semi-supervised paradigm.

Numerical Results

The framework is evaluated on the STL-10 dataset, emphasizing unlabeled training for self-supervised modules and using minimal labeled data for classification. Key performance metrics include classification accuracy and peak signal-to-noise ratio (PSNR) under various transmission conditions and channel bit error rates (BER).

Notable empirical results:

• The proposed approach reduces transmitted data size by ~95% (from 27KB original to 0.9KB), while classification accuracy degrades by only 4–5% compared to the raw image baseline.
• Over 90% classification accuracy is maintained after compression and foreground extraction under low BER; performance remains robust even as BER increases (substantial accuracy preserved below BER of $10^{-2}$).

PSNR metrics also demonstrate superior image reconstruction robustness compared to JPEG, standard MAE, and prior deep joint source-channel coding approaches, especially under AWGN scenarios and channel-induced errors.

Theoretical and Practical Implications

This work demonstrates that coupling ViT-MAE-based self-supervised foreground extraction with an end-to-end semi-supervised autoencoding and classification pipeline substantially reduces both labeling and communication overhead without significant loss in task performance. The explicit use of attention-derived semantic masking, joint learning, and adaptively refined latent coding introduces a principled methodology for goal-oriented semantic communication in resource-constrained, noisy wireless environments.

The framework distinguishes itself by combining semantic compression with minimal supervision and a hybrid design that gate-keeps classification-relevant information while discarding background redundancies. This is critical for real-world federated learning, edge inference, IoT, and bandwidth-constrained scenarios where annotation cost and radio resources are bottlenecks.

Future Directions

Potential advancements include the integration of adaptive, context-aware refinement schemes and task-conditioned dynamic masking policies. Further exploration of cross-modal semantic communication and generalization to multi-task scenarios (e.g., detection, retrieval, multimodal reasoning) is suggested. Advancing this paradigm may enable scalable, label-efficient deployment of semantic communication systems for autonomous vehicles, sensor networks, and beyond.

Conclusion

This paper establishes a robust and label-efficient wireless semantic communication pipeline for image foreground classification. By leveraging ViT-based masking for self-supervised semantic extraction, a hybrid semi-supervised autoencoder for efficient latent description, and adaptive patch refinement, the framework achieves significant transmission savings and strong classification under stringent resource constraints. The methodology and empirical findings highlight the value of joint, goal-driven semantic representation learning in the context of next-generation wireless networks.