Network Foundation Models

• Network Foundation Models are large-scale pretrained models that learn general-purpose network representations from diverse data sources using self-supervised techniques.
• They employ domain-specific tokenization and Transformer-based architectures tailored to protocol hierarchies, enabling effective adaptation for tasks like traffic classification and anomaly detection.
• NFMs reduce the need for extensive labeled data while achieving high performance, though challenges remain in scaling sequence lengths, ensuring real-time inference, and handling encrypted or noisy payloads.

Searching arXiv for the cited Network Foundation Models papers to ground the article in current literature. arxiv_search({"query":"id:(Mayhoub et al., 6 Feb 2026) OR id:(Le et al., 2022) OR id:(Guthula et al., 2023) OR id:(Qian et al., 2024) OR id:(Sylee et al., 27 Sep 2025) OR id:(Chen et al., 2023) OR id:(Abdisarabshali et al., 3 Sep 2025) OR id:(Zanouda et al., 2024) OR id:(Zhang et al., 6 Jan 2026) OR id:(Liang et al., 4 Jun 2026)", "max_results": 10}) Network Foundation Models (NFMs) are large-scale pretrained neural models that learn general-purpose representations from network data such as packet headers, payload bytes, flow sequences, telemetry streams, and, in wireless settings, channel state information, raw I/Q streams, interference graphs, and multimodal sensing inputs. Across the literature, the term denotes a shift from task-specific supervised models toward self-supervised pretraining on abundant unlabeled network traces followed by adaptation through fine-tuning, prompting, adapters, or related parameter-efficient techniques (Le et al., 2022). In traffic analysis, NFMs have been instantiated with Transformer backbones over bytes, packet fields, bursts, flows, patches, and image-like representations, and have been evaluated on traffic classification, traffic characteristic prediction, and traffic generation (Mayhoub et al., 6 Feb 2026). In security, benchmarking, federated wireless deployment, and telecom orchestration, the same concept expands into hierarchical, multimodal, and cross-tier systems that seek reusable representations and unified task handling under the constraints of protocol structure, long temporal context, heterogeneity, privacy, and latency (Guthula et al., 2023).

1. Definition, scope, and conceptual foundations

The foundational definition recurring across the literature is that an NFM is a large-scale pretrained model designed to learn general-purpose network representations from raw or minimally processed network data via self-supervision, after which it can be fine-tuned or prompted for downstream tasks with minimal labeled data (Le et al., 2022). In network traffic analysis, this includes packet headers, flow sequences, telemetry, and traffic metadata; in wireless networking, it extends to CSI tensors, raw I/Q streams, interference graphs, and multimodal sensing signals (Liang et al., 4 Jun 2026). In wireless and telecom systems, NFMs are additionally framed as network-tier-deployed models spanning cloud, fog/edge, and client nodes, where inference and training must satisfy per-node compute, memory, and bandwidth constraints (Chen et al., 2023).

A central motivation is that network data shares several properties with data domains in which foundation models have proved effective, while also introducing domain-specific complications (Le et al., 2022). The literature identifies rich semantic content, correspondence between NLP-style pretraining tasks and networking tasks, and the possibility of few-shot adaptation as opportunities. At the same time, it emphasizes protocol hierarchy, statefulness, long and interleaved sequences, privacy-sensitive or encrypted payloads, domain shift, rare events, and real-time inference requirements as constraints that make direct transfer of generic FM practice insufficient (Le et al., 2022).

The design principles attached to NFMs are correspondingly specific. They include self-supervised pretraining on massive unlabeled traces; contextualized representations that interpret fields relative to packet and flow context; modularity through common tokenizers and encoders with task-specific heads; and domain alignment through protocol-aware tokenization, hierarchical positional structure, and architecture choices that reflect packet order, protocol syntax, and cross-packet dependencies (Le et al., 2022). This suggests that NFMs are best understood not as a single model family but as a design program: general-purpose representation learning adapted to the ontological structure of networked systems.

2. Data modalities, tokenization, and representational units

A defining feature of NFMs is that the representational unit is not fixed. The surveyed traffic models cover byte-level sequences, burst- or packet-level tokenization, protocol-field-aware packet decomposition, flow-level multimodal inputs, and image-style patching of flows (Mayhoub et al., 6 Feb 2026). PERT treats each payload byte as a “word” and applies a dynamic embedding table $E\in\mathbb{R}^{|V|\times d}$ to raw bytes. ET-BERT groups unidirectional bursts, converts each burst into a hex string, applies a bi-gram tokenizer plus Byte-Pair Encoding, and embeds the resulting tokens with learned vectors. netFound and MLETC split each packet into header fields and payload, use separate embedding tables for each field type plus learned header-type embeddings, and, in MLETC, mask entire header fields as atomic units (Mayhoub et al., 6 Feb 2026).

Flow-level multimodality introduces additional signals beyond tokens. netFound concatenates packet embeddings with flow metadata such as timestamps and inter-arrival times and projects them into a shared $d$-dimensional space (Mayhoub et al., 6 Feb 2026). In image-style formulations, YaTC and Flow-MAE transform flows into 2D or 1D visual-like objects, such as Multi-level Flow Representation matrices or 1D patches of bursts, which are then embedded by linear projection (Mayhoub et al., 6 Feb 2026). NetBench formalizes a benchmark tokenization pipeline in which anonymized flows are converted into hexadecimal strings, tokenized with WordPiece over four hex digits per token, and augmented with special tokens such as <head>, <pkt>, and </s> (Qian et al., 2024).

Protocol awareness is treated as essential rather than optional. The netFound architecture extracts 13 fields per packet into a fixed 279-bit vector, splits that vector into 16-bit tokens, groups same-direction packets with inter-arrival gaps of at most $10$ ms into bursts, pads or truncates to 6 packets per burst and 12 bursts per flow, and introduces special tokens [CLS-B], [CLS-F], [PAD], and [MASK] (Guthula et al., 2023). Its embeddings sum token, positional, and metadata embeddings, where metadata captures burst direction, bytes-per-burst, packets-per-burst, and burst timestamp delta. The broader NFM literature similarly advocates protocol-aware segmentation into sub-fields, byte-pair encoding with header boundaries, and hybrid tokenizers that respect protocol grammar while retaining fallback mechanisms for unknown payload structure (Le et al., 2022).

A recurrent implication is that tokenization is not merely preprocessing. It encodes the ontological stance of the model toward the network: whether the network is treated primarily as text-like bytes, structured protocol records, burst-level temporal behavior, or multimodal telemetry. Many reported gains in NFMs are tied explicitly to this representational alignment rather than to scale alone (Guthula et al., 2023).

3. Architectures and self-supervised objectives

Transformer backbones dominate the current traffic-NFM literature, with encoder-only, decoder-only, and encoder-decoder variants appearing in BERT-style, GPT-style, and T5-style forms (Mayhoub et al., 6 Feb 2026). A representative BERT-style traffic foundation model uses $L=12$ encoder layers, hidden dimension $d=768$, $H=12$ attention heads per layer, intermediate feed-forward size $4d$, and learned positional embeddings $P\in\mathbb{R}^{(L\times d)}$, with self-attention

$$A^{(\ell)}=\mathrm{softmax}\!\left(\frac{Q^{(\ell)}(K^{(\ell)})^T}{\sqrt{d/H}}\right)V^{(\ell)},$$

where $Q,K,V=XW_Q,W_K,W_V$ (Mayhoub et al., 6 Feb 2026). The architecture space also includes hierarchical transformers, graph-enhanced variants, and multimodal fusion modules (Le et al., 2022).

The self-supervised objectives used in NFMs largely mirror those of language and vision models, but are adapted to protocol and flow structure. The principal losses summarized in recent work are masked packet or token modeling, masked patch reconstruction, next-token prediction, same-origin flow prediction, and packet order prediction (Mayhoub et al., 6 Feb 2026). With token sequence $d$0 and masked positions $d$1, masked modeling takes the form

$d$2

For MAE-style continuous inputs, the reconstruction objective is

$d$3

For decoder-only models such as NetGPT and TrafficGPT, autoregressive next-token prediction is

$d$4

Same-origin flow prediction is formulated as a binary loss over pairs of segments, and Lens-style packet order prediction is cast as cross-entropy over permutations (Mayhoub et al., 6 Feb 2026).

At the conceptual level, the NFM design literature further identifies masked packet modeling, next-packet prediction, and contrastive session-level losses as canonical objectives (Le et al., 2022). The contrastive objective is written as

$d$5

where $d$6 are flow embeddings, $d$7 is dot-product similarity, and $d$8 is a temperature (Le et al., 2022). This broadens the NFM objective space beyond masked recovery toward session and context discrimination.

netFound provides a prominent example of architecture-level specialization. It uses a two-level hierarchical transformer in which a burst encoder operates on burst-local tokens and a flow encoder attends only to burst summaries plus a flow summary token, followed by a projection that restores token-level outputs for masked prediction (Guthula et al., 2023). The masked-token objective randomly selects 30% of tokens, replacing 80% with [MASK], 10% unchanged, and 10% with random tokens, and optimizes

$d$9

This design explicitly trades quadratic token-token attention across an entire long flow for hierarchical factorization aligned with burst and flow structure (Guthula et al., 2023).

In wireless NFMs, the architecture space broadens further to graph transformers, multimodal cross-attention modules, and reinforcement-learning-integrated agentic models (Liang et al., 4 Jun 2026). Inputs may be CSI matrices $10$0, raw I/Q samples, graphs $10$1, or textual intents; objectives include self-supervised masked reconstruction, autoregressive next-token prediction, contrastive learning, and RL reward maximization (Liang et al., 4 Jun 2026). This suggests that the “foundation” property in NFMs is increasingly tied to cross-modality reuse and control readiness as much as to static representation learning.

4. Downstream tasks, adaptation, and empirical performance

The traffic-analysis literature presents a unified pre-training and fine-tuning pipeline in which pretrained backbones are adapted to traffic classification, traffic characteristic prediction, and traffic generation (Mayhoub et al., 6 Feb 2026). For classification, a pretrained encoder plus MLP head produces logits $10$2, probabilities $10$3, and cross-entropy loss

$10$4

For traffic characteristic prediction, an encoder plus regression head predicts values such as flow volume, trained with mean squared error and often reported with MAPE. For traffic generation, a prompt template and partial flow tokens condition a frozen decoder with a small prefix-tuning adapter, optimized with next-token cross-entropy (Mayhoub et al., 6 Feb 2026).

The reported results support the claim that pretrained traffic models transfer across heterogeneous tasks. On CICIoT2023, YaTC+MLP achieved 96.9% overall accuracy and average $10$5, compared with an MLP-only baseline at 72.5% accuracy and average $10$6; the summary notes that the Transformer backbone consistently boosts $10$7 by 20–40 points across classes (Mayhoub et al., 6 Feb 2026). On CIC-IDS-2017 benign-flow traffic volume prediction, NetFound plus a regression head achieved MAE of 18 bytes versus 21.63 bytes for an MLP baseline, MAPE of 0.082%, and $10$8 versus 0.840 (Mayhoub et al., 6 Feb 2026). In traffic generation on the CICIoT2023 Upload Attack case, a prefix-tuned ChatGLM2-6B-based TrafficLLM generated TTL and IP packet length distributions whose CDFs closely matched real packets (Mayhoub et al., 6 Feb 2026).

The same work emphasizes label efficiency. In the IoT classification use case, adding just 1 K fine-tuning flows, identified as 0.1%, achieves $10$9 accuracy, and even a shallow MLP head on a frozen encoder outperforms a fully trained end-to-end MLP by $L=12$0 $L=12$1 points (Mayhoub et al., 6 Feb 2026). This directly instantiates the core NFM premise described earlier at the conceptual level: pretraining can reduce labeled-data requirements while improving generalization (Le et al., 2022).

NetBench extends this empirical picture by standardizing evaluation across seven public datasets and twenty tasks, comprising fifteen classification tasks and five generation tasks (Qian et al., 2024). It evaluates ET-BERT and YaTC as foundation models against six traditional deep-learning methods. The benchmark reports that ET-BERT and YaTC consistently achieve $L=12$2 accuracy and $L=12$3 on many tasks; for example, in VPN detection, ET-BERT at flow level reaches accuracy $L=12$4 and $L=12$5, while YaTC reaches accuracy $L=12$6 and $L=12$7, compared with DataNet at accuracy $L=12$8 and $L=12$9 (Qian et al., 2024). On a 17-class VPN application task, ET-BERT at packet level achieved accuracy $d=768$0 and $d=768$1, versus FS-Net at accuracy $d=768$2 and $d=768$3 (Qian et al., 2024). The benchmark also notes that packet-level fine-tuning can outperform flow-level fine-tuning when flow sequences are truncated to 512 tokens, underscoring the practical impact of context-window limits (Qian et al., 2024).

In security-specific evaluation, netFound is presented as a network foundation model for disparate learning tasks under sparse, noisy, and skewed labels (Guthula et al., 2023). It reports $d=768$4 for 11-class campus traffic classification, compared with 81.4% for nPrintML, 68.6% for ET-BERT, and 41.3% for Curtains. For CIC-IDS-2017 intrusion detection, it reports $d=768$5, and for APT detection in a multi-cloud setting, $d=768$6 (Guthula et al., 2023). The same study reports robustness to dropped and corrupted labels: with 40% missing labels, traffic-classifier $d=768$7 falls from 88.3% to 86.2%, and with 40% random label corruption, to 83.6% (Guthula et al., 2023).

These results collectively support a stable empirical pattern: NFMs are most clearly advantageous when labels are limited, task families are diverse, and tokenization plus architecture are matched to network structure. A plausible implication is that gains attributed to “foundation models” in networking are inseparable from domain-shaped representation choices.

5. Benchmarks, intrinsic analysis, and model limitations

Evaluation methodology is itself a major topic in the NFM literature. Early conceptual work proposed benchmark suites analogous to GLUE, spanning tasks such as device classification, anomaly detection, and performance estimation, with ML metrics including accuracy, precision/recall, $d=768$8, ROC-related measures, perplexity, token-prediction accuracy, throughput/latency impact, and flow completion time prediction (Le et al., 2022). NetBench operationalizes part of this agenda through a unified preprocessing and split pipeline intended to prevent data leakage by splitting raw PCAPs at the PCAP/flow level before tokenization, with an 8:1:1 train/validation/test ratio (Qian et al., 2024).

A later line of work shifts attention from downstream performance to intrinsic representation analysis. “Demystifying Network Foundation Models” evaluates YaTC, ET-BERT, netFound, and NetMamba on five datasets using three diagnostics: Embedding Geometry Analysis, Metric Alignment Assessment, and Causal Sensitivity Testing (Sylee et al., 27 Sep 2025). Its anisotropy score

$d=768$9

measures how collapsed embedding spaces are, while Mean Cosine Contribution assesses axis dominance. The reported finding is that all evaluated NFMs exhibit significant anisotropy, with average $H=12$0 across all models and datasets; NetMamba is described as almost always collapsed, with $H=12$1 in several cases (Sylee et al., 27 Sep 2025).

Metric alignment is assessed via linear CKA between learned embeddings $H=12$2 and CICFlowMeter features $H=12$3: $H=12$4 Among the four models, netFound shows the highest average CKA at approximately 0.143, followed by YaTC at approximately 0.093, NetMamba at approximately 0.066, and ET-BERT at approximately 0.029 (Sylee et al., 27 Sep 2025). The same work reports sharply lower alignment on real-world CAIDA and MAWI than on controlled CIC datasets, suggesting reduced correspondence between NFM latent structure and conventional flow statistics under more heterogeneous operational conditions (Sylee et al., 27 Sep 2025).

Causal sensitivity testing reveals further limitations. Using protocol-relevant perturbations, the study reports that most models are robust to header changes, except that YaTC is selectively sensitive to sequence/acknowledgment numbers and window size. All models except netFound undergo drastic shifts under payload perturbation; for example, YaTC’s cosine similarity under payload perturbation is 0.18, ET-BERT’s is 0.48, and NetMamba’s is 0.62, while netFound remains at 0.99 (Sylee et al., 27 Sep 2025). The authors interpret this as excessive payload dependency, which is problematic when payloads are encrypted or unavailable. They also report weak geometric separation of high-level network context, even when linear probes can extract some contextual information (Sylee et al., 27 Sep 2025).

Importantly, the paper shows that simple post hoc embedding isotropification can improve downstream performance. A decorrelation transform applied to frozen embeddings yields gains up to $H=12$5 $H=12$6 without architectural changes, with NetMamba showing $H=12$7 on Crossmarket (Sylee et al., 27 Sep 2025). This suggests that some limitations of current NFMs lie less in representational capacity than in latent-space geometry and training objectives.

6. Deployment topologies, federated wireless systems, and telecom-scale NFMs

Outside traffic analysis and security, the NFM concept is extended to distributed wireless and telecom systems in which model placement and communication topology become first-class design variables. In wireless-network federated learning, an NFM is defined as a large-scale, general-purpose AI model, often with billions of parameters, pretrained on massive multimodal data and deployed across cloud, fog/edge, and client tiers to deliver adaptive, low-latency network intelligence (Chen et al., 2023). The corresponding FL framework broadcasts global parameters $H=12$8 or lightweight updates, performs local fine-tuning or PEFT, compresses client updates, and aggregates them with weighted averaging. Communication cost is formalized through uplink and downlink rates and model-size terms, and secure aggregation is treated as a standard primitive (Chen et al., 2023).

A more specialized hierarchical federated design appears in hierarchical federated foundation models over wireless networks. There, edge devices, fog servers, and cloud form a three-tier infrastructure; D2D links can provide horizontal module relaying and intra-cluster consensus; and foundation models are decomposed into modality-specific encoders $H=12$9, prompt layers $4d$0, MoE blocks $4d$1, adapters $4d$2, and task heads $4d$3 (Abdisarabshali et al., 3 Sep 2025). The local objective at edge device $4d$4 is

$4d$5

fog aggregation is weighted by local sample sizes, and cloud aggregation composes fog-level aggregates in the same manner (Abdisarabshali et al., 3 Sep 2025). In a case study with 40 edge devices, 10 fog servers, and 1 cloud, using ViLT with inserted Double-Adapter modules, HF-FM reduces latency and energy relative to star-FFM while improving VQA accuracy: HF-FM with $4d$6 reports latency $4d$7, energy $4d$8, Art VQA 61.5, and GQA 65.2, compared with star-FFM at 1.00×, 1.00×, 59.2, and 62.8 (Abdisarabshali et al., 3 Sep 2025). D2D-enabled HF-FM matches HF-FM accuracy with further communication savings (Abdisarabshali et al., 3 Sep 2025).

Telecom Foundation Models generalize the idea further into a multimodal graph-transformer framework over configuration files, PM time series, KPIs and alarms, logs, standardization documents, user measurements, and geospatial auxiliary data (Zanouda et al., 2024). The conceptual architecture combines a radio-node component, a network component represented as a graph of RAN nodes, and a network-journey component capturing expert rules, upgrade actions, and maintenance tickets. Pretraining is described conceptually through multimodal graph pretraining objectives such as

$4d$9

followed by continued pretraining, instruction or adapter tuning, prompting, RAG or Graph-RAG, and orchestration among specialized TFMs (Zanouda et al., 2024). The paper is explicitly conceptual and does not provide concrete comparative accuracy or latency numbers, but it frames deployment and orchestration of multiple specialized TFMs as a central systems problem (Zanouda et al., 2024).

Wireless-specific surveys then organize NFMs into off-the-shelf FM adaptation, wireless-native foundation models, and agentic foundation models (Liang et al., 4 Jun 2026). They describe shared patterns: modality-specific encoders projected into a common token space, transformer or graph-transformer backbones, lightweight adaptation through LoRA or prompts, and downstream tasks spanning beam management, power control, resource allocation, congestion prediction, semantic communications, and autonomous orchestration (Liang et al., 4 Jun 2026, Zhang et al., 6 Jan 2026). This broadening suggests that the term “NFM” now covers both models of network traffic and models embedded within networking infrastructure itself.

7. Challenges, controversies, and future directions

Several limitations recur across the NFM literature. One is sequence-length scaling. In traffic models, Transformer attention incurs $P\in\mathbb{R}^{(L\times d)}$0 complexity in sequence length $P\in\mathbb{R}^{(L\times d)}$1, while real flows may exceed 1 K tokens; proposed mitigations include sparse or linear attention and chunking (Mayhoub et al., 6 Feb 2026). NetBench identifies truncation at 512 tokens as a concrete source of degraded flow-level performance (Qian et al., 2024). netFound addresses this through hierarchical decomposition (Guthula et al., 2023), and conceptual work points to sliding windows, hierarchical positional encoding, and session demultiplexing as broader strategies (Le et al., 2022).

A second challenge is reliance on payloads versus metadata and headers. Privacy-sensitive or encrypted environments often restrict payload availability, motivating metadata-centric training and anonymized traces (Le et al., 2022). The intrinsic-analysis literature argues that several NFMs remain overly payload-dependent and insufficiently robust to protocol perturbation, which complicates claims of universal traffic understanding (Sylee et al., 27 Sep 2025). This is not a contradiction so much as a boundary condition: pretrained models may generalize across tasks yet still encode brittle causal dependencies.

A third challenge is real-time deployment. In traffic analysis, inference latency must meet sub-second constraints in NIDS, and future work is said to require benchmarking end-to-end throughputs and exploring distillation and quantization (Mayhoub et al., 6 Feb 2026). In wireless and telecom settings, even tighter latency, bandwidth, and energy constraints motivate split learning, hierarchical fine-tuning, quantization, distillation, and on-device adapter tuning (Chen et al., 2023, Zanouda et al., 2024). Efficient inference is therefore not ancillary; it is part of what determines whether a model can function as a network component rather than as an offline analytics tool.

Interpretability and robustness are also persistent concerns. The design literature highlights attention visualization, concept-based explanations over packet fields, and token-level attributions such as Integrated Gradients as possible strategies (Le et al., 2022). Traffic-analysis work suggests using attention maps to highlight influential header fields or payload bytes, and combining symbolic constraints such as protocol state machines with Transformer backbones for more robust anomaly inference (Mayhoub et al., 6 Feb 2026). These proposals reflect a broader tension: NFMs promise a unified framework across tasks, but network operators often require field-level causal justification, especially in security and control applications.

Finally, there is an open question about what should count as a “network” in Network Foundation Models. In traffic and telecom papers, NFMs refer to models over network traffic, telemetry, and control data (Mayhoub et al., 6 Feb 2026, Zanouda et al., 2024). In federated wireless work, they denote foundation models deployed over network tiers (Chen et al., 2023, Abdisarabshali et al., 3 Sep 2025). In one separate usage, foundation models are cast as hypernetworks that generate the weights of other neural networks and are described as NFMs in that sense (Gu et al., 2 Mar 2025). This terminological breadth suggests that the field is still stabilizing around the network object of study: packet and flow behavior, communications infrastructure, multimodal wireless environments, or neural-network parameter spaces. A plausible implication is that future usage will increasingly reserve the term for pretrained models whose inductive biases and deployment context are explicitly shaped by network structure, protocol semantics, and system constraints, rather than for any foundation model merely applied somewhere within networking.

Across these variants, the clearest common thread is methodological: pretrain on abundant unlabeled network-relevant data, align tokenization and architecture with domain structure, and adapt a reusable backbone across heterogeneous downstream tasks. The technical agenda now extends beyond performance improvement toward latent-space geometry, causal feature use, scalable context handling, privacy-aware distributed training, and orchestration in real networked systems (Sylee et al., 27 Sep 2025).