NSS-Tokenizer: Traffic-Aware Semantic Tokenization

• The paper’s main contribution is NSS-Tokenizer, which resects network protocol fields using a novel delimiter to maintain semantic integrity.
• It combines supervised joint-dataset fine-tuning with parameter-efficient LoRA adaptation to achieve state-of-the-art classification accuracy and robust cross-domain performance.
• Experimental results demonstrate high resilience against adversarial noise and scalability for real-world IDS deployments in dynamic network environments.

BERTector is a deep learning-based intrusion detection system (IDS) that addresses the challenges of generalization and robustness in the presence of highly heterogeneous network traffic and diverse attack types. By combining a traffic-aware tokenization scheme (NSS-Tokenizer), supervised joint-dataset fine-tuning, and a parameter-efficient low-rank adaptation (LoRA) mechanism, BERTector establishes a unified and scalable solution for modern IDS deployments in large-scale, dynamic environments. Extensive benchmarking demonstrates state-of-the-art classification accuracy, strong cross-domain generalization, and high robustness to adversarial perturbations (Hu et al., 14 Aug 2025).

1. NSS-Tokenizer: Traffic-Aware Semantic Tokenization

The NSS-Tokenizer is designed to mitigate the deficiencies of standard BERT tokenizers when applied to network flow data. Standard approaches such as WordPiece are optimized for natural language processing and tend to fragment structured protocol fields (e.g., IP addresses, port numbers) into multiple subword pieces. This leads to lengthy, noisy, and semantically ambiguous token sequences.

NSS-Tokenizer addresses these limitations by:

• Resecting the semantic boundaries of protocol fields using an out-of-vocabulary delimiter (“⊞”) that does not appear in native datasets.
• Dropping empty or redundant tokens, truncating or padding each flow to a uniform length $L_{\max}$, and maintaining the semantic integrity of flows from heterogeneous sources.
• Enabling seamless concatenation of traffic flows from different datasets by ensuring that the tokenized output is a fixed-length, flat stream, regardless of the original schema.

Given a flow $f$ comprised of fields $\{v_1, v_2, \dots, v_n\}$, NSS-Tokenizer constructs a delimited string

$$f_{\rm delimited} = v_1 \oplus ⊞ \oplus v_2 \oplus ⊞ \oplus \ldots \oplus v_n$$

which is then split at the character level and truncated or padded to $$L_{\max} = \min(\max_{f \in D_{\rm train}} \mathrm{len}(f), 512)$$. Tokens are mapped to integer IDs via vocabulary lookup, resulting in tokenized representations $T(f) \in \{0, \ldots, V-1\}^{L_{\max}}$ suitable for direct BERT ingestion. The approach obviates the need for explicit feature alignment or manual handling of missing fields.

2. Joint-Dataset Supervised Fine-Tuning

BERTector employs a hybrid-dataset training paradigm in which flows from multiple benchmark IDS datasets are pooled and used for supervised fine-tuning. The datasets used include KDD-99, NSL-KDD, UNSW-NB15, and X-IIoTID, each contributing 100,000 flows.

The preprocessing pipeline involves converting each source’s traffic data to NetFlow-style tuples, followed by NSS-Tokenizer application to ensure uniformity. The resultant dataset MIX is split 80/20 into training and validation sets per source, while test sets consist of 10,000 never-before-seen flows per dataset. Class balancing is achieved by uniform class sampling across datasets but without specific over- or under-sampling by attack category.

The optimization objective is the standard label-weighted cross-entropy:

$$\mathcal{L}_{\mathrm{CE}} = -\frac{1}{N} \sum_{i=1}^{N} \sum_{k=1}^{C} y_{i,k}\,\log \hat{y}_{i,k}$$

with L2 regularization on all trainable parameters:

$$\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \lambda \|\Theta\|_2^2$$

No explicit domain adaptation losses (adversarial, MMD) are introduced. Regularization is implicitly achieved through dropout ($p=0.1$), early stopping on validation loss, and L2 weight decay.

3. Low-Rank Adaptation (LoRA) for Efficient Training

Full fine-tuning of BERT’s ~110M parameters for every new dataset is computationally burdensome. BERTector leverages LoRA-based adaptation to reduce the number of trainable parameters by two orders of magnitude, facilitating rapid re-training and mitigating the risk of overfitting.

Within any BERT feed-forward or projection layer (weight matrix $W \in \mathbb{R}^{d \times k}$), LoRA parameterizes weight updates as

$$\Delta W = B\,A,\quad A \in \mathbb{R}^{r \times k},\, B \in \mathbb{R}^{d \times r},\, r \ll \min(d,k)$$

At inference, only $A$ and $B$ are updated; $W$ itself remains frozen. The effective weight is $W_{\rm eff} = W + BA$, with no overhead in forward computation. The typical LoRA rank $r$ used is 8. This strategy enables parameter-efficient fine-tuning, allowing BERTector to adapt effectively across domains with a minimal memory/compute footprint.

4. Model Architecture and Training Protocol

BERTector preserves the core characteristics of the standard BERT encoder while specializing for IDS traffic analysis:

• Input: $T(f) \in \mathbb{Z}^{L_{\max}}$, $L_{\max} \leq 512$.
• Embedding: Token and positional embeddings of size $L_{\max} \times d$, $d=768$.
• Encoder: Twelve Transformer encoder blocks ($d=768$, $H=12$ heads, feedforward dimension = $3072$).
• Pooling: The [CLS] token is pooled for downstream attack classification.
• Output: Dense softmax layer mapping to $C$ attack classes.

LoRA is integrated into every linear projection and feedforward matrix within the encoder stack.

Training employs the Adam optimizer (with L2 weight decay), a constant learning rate $2 \times 10^{-5}$, batch size 64, and up to ten epochs with early stopping triggered after three consecutive epochs without validation loss improvement. No learning rate warmup or gradient clipping is reported.

5. Experimental Results and Performance Assessment

BERTector’s empirical performance is evaluated both on single-source datasets and via zero-shot transfer following mixed-source (MIX) training. For NSL-KDD in isolation, BERTector attains:

• Accuracy: $0.9928$
• Precision: $0.9880$
• Recall: $0.9989$
• F1-score: $0.9934$

Under cross-dataset (MIX→test) evaluation, observed accuracies are:

Dataset	NSL-KDD	KDD-99	UNSW-NB15	X-IIoTID
Accuracy	0.9903	0.9887	0.9610	0.9987

BERTector’s robustness is further validated through adversarial noise injection (Poisson, Uniform, Gaussian, Laplace) on the NSL-KDD test set. Under Poisson perturbations, BERTector achieves $\text{Acc} = 0.9374$, F1 = 0.9437, substantially exceeding the next-best classical ML or deep learning baselines (F1, Acc < 0.83). With stronger Uniform/Gaussian/Laplace noise, BERTector maintains 0.73–0.77 accuracy, F1 ≥ 0.78, outperforming reference alternatives.

6. Design Rationale, Limitations, and Future Directions

BERTector’s competitive advantage derives from three factors:

1. NSS-Tokenizer ensures strict semantic alignment of protocol fields, minimizing subword fragmentation and noisy tokens.
2. Joint-dataset supervised fine-tuning promotes cross-domain pattern learning and prevents overfitting to spurious dataset-specific artifacts.
3. LoRA adaptation makes frequent, scalable re-training practical and avoids catastrophic forgetting or domain-specific degeneration.

Identified limitations include the absence of evaluation on online/streaming data for real-time IDS, lack of mechanisms to address rare class imbalance, and absence of explicit domain-discrepancy losses or unsupervised anomaly modeling. Potential avenues for extension involve dynamic adaptation to evolving network traffic and integration of unsupervised or semi-supervised anomaly detection modules to enhance zero-day attack resilience (Hu et al., 14 Aug 2025).