NEFMind: 5G API Automation

• NEFMind is a domain-specific telecom automation framework that converts natural-language queries into structured NEF API calls in 5G service-based architectures.
• It employs an open-source Phi-2 model fine-tuned via QLoRA and synthetic data generation from YAML-based NEF API specifications to streamline API discovery.
• Evaluation results indicate near-perfect performance with 98–100% accuracy on API call identification and substantial resource savings compared to traditional methods.

NEFMind is a domain-specific telecom automation framework that uses an open-source LLM, Phi-2, and parameter-efficient fine-tuning to turn natural-language user requests into correct Network Exposure Function (NEF) API calls in 5G networks. It is positioned within 5G Service-Based Architecture (SBA), where the growth of Network Functions (NFs) and Application Programming Interfaces (APIs) creates substantial operational complexity in service discovery and management. The framework integrates three core components—synthetic dataset generation from NEF API specifications, model optimization through Quantized-Low-Rank Adaptation, and evaluation via GPT-4 Ref Score and BERTScore—to automate the documentation-driven process by which operators identify an endpoint, method, operation, and parameters for a desired NEF action (Khan et al., 12 Aug 2025).

1. Telecommunications context and task definition

NEFMind is designed for 5G SBA environments standardized by 3GPP. In that setting, network functionality is exposed through NFs, and those functions communicate through service-based interfaces and REST APIs. One important NF is the NEF, which exposes network capabilities to external consumers via APIs. As 5G systems grow, the number of NFs and APIs grows as well, increasing the burden of API discovery and request composition (Khan et al., 12 Aug 2025).

The framework targets a concrete operational problem. Telecom engineers and system administrators often have to inspect NEF API documentation, infer the correct endpoint and HTTP method, identify the relevant operation, and manually assemble the required parameters. NEFMind is presented as an automation layer for that discovery-and-formulation step. In the example given in the paper, a request such as “How can I obtain an access token for future requests?” is mapped to /api/v1/login/access-token together with its associated method, operation, and parameters (Khan et al., 12 Aug 2025).

This framing is narrower than general-purpose question answering. The model is not trained merely to classify an endpoint label; it is trained to produce structured API information associated with a telecom request. That emphasis on structured response generation is central to the framework’s design.

2. Data construction from NEF API specifications

The data pipeline begins with NEF API specification files in YAML format from a telecom API dataset. Because one file references others, the YAML structure is flattened before example generation. GPT-4 is then used as a teacher model to generate synthetic training examples in JSON, and those outputs are manually validated to remove fabricated or invalid objects. Each JSON record contains request, api_call, description, method, operation, and parameters fields (Khan et al., 12 Aug 2025).

A representative record maps a natural-language request to an OAuth2 token endpoint. The paper also describes another example for reading active subscriptions. These examples establish the task format as a request-to-structured-call mapping problem rather than free-form summarization.

Because the initial validated set was small, the dataset was expanded by prompting GPT-4 to generate 100 unique paraphrases of each request. This expansion grew the dataset from 7 validated records to 765 records. The resulting corpus was split roughly 70/30 into about 535 training records and 230 evaluation records. For training, the JSON data was converted into CSV and formatted into Instruct/Output pairs: the Instruct field contains the user request text, and the Output field contains the API call, description, method, operation, and parameters (Khan et al., 12 Aug 2025).

This construction procedure matters methodologically because the supervision is derived from actual NEF API specifications rather than from unconstrained synthetic intents alone. A plausible implication is that the model’s strong task performance depends not only on fine-tuning strategy but also on the tight coupling between synthetic requests and specification-grounded outputs.

3. Model choice and parameter-efficient fine-tuning

The model used in the experiments is Phi-2, an open-source Transformer with 2.7 billion parameters and a 2048-token context window. The paper selects Phi-2 because it is relatively small, open, and efficient enough to adapt for a telecom-specific task. Fine-tuning is performed with Quantized Low-Rank Adaptation (QLoRA), a parameter-efficient fine-tuning (PEFT) method in which low-rank adapter parameters are trained instead of updating all model weights (Khan et al., 12 Aug 2025).

The paper gives several reasons for using PEFT and QLoRA in this setting: lower compute cost than full fine-tuning, lower memory usage, better practicality for domain deployment, and near full-fine-tuning performance on the target task. It further states that NEFMind requires about 60% less computational resources than comparable full fine-tuning approaches (Khan et al., 12 Aug 2025).

The reported QLoRA configuration is specific: LoRA alpha 16, LoRA dropout 0.1, LoRA rank 64, target modules q_proj, k_proj, v_proj, dense, fc1, fc2, bias None, and task type CAUSAL_LM. Training uses 5 epochs, batch size 3, gradient accumulation steps 1, optimizer paged_adamw_32bit, save steps 10, logging steps 10, learning rate $2 \times 10^{-4}$, weight decay 0.001, warmup ratio 0.03, BF16 True, max grad norm 0.3, scheduler Constant, group by length True, and reporting to TensorBoard. The experiments run on Amazon G5 g5.4xlarge instances with a single 24GB GPU, 16 vCPUs, and 64GB RAM. Reported training statistics are training runtime $\sim 595$ seconds, training throughput 4.495 samples/sec, training step rate 1.504 steps/sec, total FLOPs $6.08 \times 10^{15}$, and final training loss 0.1921 (Khan et al., 12 Aug 2025).

These details place NEFMind within the broader PEFT literature as a deployment-oriented specialization pipeline: a compact base model is adapted to a narrow but operationally significant API-selection task without requiring full-parameter retraining.

4. Evaluation protocol and reported performance

Evaluation uses the 230 held-out records and compares the baseline Phi-2 model with the fine-tuned Phi2-NEF model. Two metrics are reported: GPT-4 Ref Score, defined as a reference-based accuracy score in which GPT-4 acts as the expert judge on a 0–100 scale, and BERTScore, which measures semantic similarity on a 0–1 scale using contextual embeddings and cosine similarity. The authors run 25 evaluation iterations and report summary statistics (Khan et al., 12 Aug 2025).

The central quantitative result is that the fine-tuned model moves from weak baseline behavior to near-perfect benchmark performance.

Metric	Baseline Phi-2	Fine-tuned Phi2-NEF
GPT-4 Ref Score (min–max)	4.7826–10.4348	98.2609–100
BERTScore (min–max)	0.7657–0.7699	0.9979–0.9986

The paper repeatedly highlights 98–100% API call identification accuracy for the fine-tuned model, compared to 4–10% for the untuned baseline. It also claims an 85% reduction in communication overhead compared to manual discovery methods and states that the fine-tuned Phi-2 model delivers performance comparable to significantly larger models like GPT-4 while maintaining computational efficiency for telecommunications infrastructure deployment (Khan et al., 12 Aug 2025).

The comparison is deliberately narrow in scope. The claim is not that Phi-2 becomes a general-purpose replacement for GPT-4, but that for 5G NEF API discovery and composition, domain-specific fine-tuning allows a small open model to perform extremely well.

5. Baseline behavior, inference mode, and operational significance

The baseline Phi-2 system is evaluated in a retrieval-augmented generation configuration because the untuned model lacks sufficient telecom-specific knowledge. That baseline pipeline includes semantic splitting of API documentation with the LangChain recursive text splitter, HuggingFace embeddings, FAISS retrieval, a QA chain, and structured JSON output generation. Even with retrieval, the baseline exhibits formatting problems and semantic mismatches (Khan et al., 12 Aug 2025).

By contrast, the fine-tuned model does not require supplementary context during inference. It takes NEF API queries directly and produces the expected structured response. This difference is operationally significant because it shifts the burden from runtime retrieval and composition toward offline domain adaptation. This suggests a deployment model in which telecom operators can use a compact, open-source model for deterministic API-surface tasks while reserving larger proprietary models for data generation or evaluation.

The paper therefore positions NEFMind as an end-to-end pipeline rather than only a fine-tuned checkpoint. Its sequence is explicit: start with NEF YAML API specs; generate synthetic JSON Q&A/API-call records with GPT-4; validate and refine the synthetic records; expand request paraphrases to 765 examples; convert to train/eval splits and format as instruct-output pairs; fine-tune Phi-2 using QLoRA; evaluate on held-out data with GPT-4 Ref Score and BERTScore; compare to baseline Phi-2 and to retrieval-based approaches; and use the model to automate telecom API call selection (Khan et al., 12 Aug 2025).

6. Limitations, scope, and terminological disambiguation

The paper identifies several limitations. The training corpus is small—765 records—which may create overfitting risk. The model is specialized for NEF APIs, so cross-domain transfer is not guaranteed. Direct API integration with agents such as LangChain’s OpenAPI agent showed limitations in deterministic, semantically consistent execution. The conclusion also notes that security implementation remains an open challenge and that expanded datasets are needed to improve robustness (Khan et al., 12 Aug 2025).

These caveats delimit the framework’s current scope. NEFMind is a specialized NEF API automation system, not a general telecom reasoning benchmark, and not a general-purpose agentic platform. Its strongest evidence concerns held-out performance on synthetic-but-validated request formulations grounded in NEF specifications.

The name can also be confused with several unrelated research terms. In “Nonlinear functional mapping of the human brain,” NFM refers to nonlinear functional mapping for resting-state fMRI analysis and is not introduced as “NEFMind” (Allgaier et al., 2015). In “Improving Multimodal Brain Encoding Model with Dynamic Subject-awareness Routing,” MIND denotes a Mixture-of-Experts decoder within the AFIRE framework for naturalistic fMRI encoding (Yin et al., 6 Oct 2025). In “Neural Electromagnetic Fields for High-Resolution Material Parameter Reconstruction,” NEMF refers to a physics-supervised inversion framework for reconstructing $\epsilon_r$ and $\sigma$ (Chen et al., 3 Mar 2026). In “Near Field Electric (NFE): Energy-efficient, High-speed Communication at Decimeter-range,” the named technology is NFE, not NEFMind (Sarkar et al., 8 Dec 2025).

Within the provided literature, therefore, NEFMind specifically denotes the telecom framework for parameter-efficient fine-tuning of open-source LLMs for NEF API automation in 5G SBA systems (Khan et al., 12 Aug 2025).