Design Self-Supervised Pre-Training Strategies for Wireless Foundation Models Across Diverse Downstream Tasks

Develop self-supervised pre-training objectives and procedures—such as next-sample prediction, masking, and denoising—for a physical-layer wireless foundation model that can effectively support multiple downstream wireless tasks with heterogeneous data representations, enabling efficient fine-tuning with minimal additional data across varied telecom applications.

Background

The white paper proposes building a wireless physical-layer foundation model using self-supervised training to reduce reliance on large labeled datasets and to facilitate adaptation to new tasks. However, telecom downstream tasks vary widely in data representation and objectives, making pre-training design challenging.

While self-supervised tasks like next-sample prediction, masking, and denoising are suggested, the paper explicitly acknowledges that determining effective pre-training strategies that generalize across multiple wireless tasks with different data formats remains unresolved and requires targeted research.

References

Nevertheless, designing effective pre-training strategies for multiple wireless downstream tasks with different data representations remains an open research question.

— Large-Scale AI in Telecom: Charting the Roadmap for Innovation, Scalability, and Enhanced Digital Experiences  (2503.04184 - Shahid et al., 6 Mar 2025) in Section 13.1.13, LTM pre-training of a physical-layer foundation model

Current gains are difficult to attribute to a single design choice. The reviewed models differ simultaneously in modality, pretraining objective, backbone, dataset composition, and adaptation protocol. As a result, it remains unclear whether performance gains come from the wireless representation itself, the learning objective, the amount and diversity of data, or the downstream task setup.

— Wireless Foundation Models: State-of-the-Art and Open Challenges  (2609.04707 - Huachaca et al., 4 Sep 2026) in Section 4.6, “Key Insights and Open Challenges” (Signal Recognition and Demodulation)

Overall, task count alone is a weak signal of foundation-model generality. What is still missing across all three groups is controlled evidence isolating the source of transfer: none of the surveyed works report ablations that separate gains due to the shared representation from gains due to task-specific conditioning or context injection. Establishing this distinction is arguably the key open question for validating cross-category PHY WFMs.

— Wireless Foundation Models: State-of-the-Art and Open Challenges  (2609.04707 - Huachaca et al., 4 Sep 2026) in Section 9, “Multi-Task Physical-Layer WFMs”