Chat-Driven Network Management

• Chat-driven network management framework is a system that integrates natural language interfaces, large language models, and optimization techniques to translate user intents into network configurations.
• It employs modular architectures—such as two-stage chat-optimize pipelines and agentic LLM systems—to achieve automated, verified network adjustments with formal correctness guarantees.
• The framework leverages iterative feedback, rigorous verification, and vendor-agnostic intermediate representations to ensure both accuracy and scalability in dynamic network environments.

A chat-driven network management framework is a class of software architectures that integrate natural language interfaces, LLMs, and network management tools to enable operators to monitor, configure, and optimize networks via dialogue-based interaction. Recent frameworks extend traditional intent-based networking by leveraging deep neural NLP methods and agentic workflow orchestration, achieving both accessibility for non-expert users and automation with formal correctness guarantees (Miyaoka et al., 31 Dec 2025, Lin et al., 24 Sep 2025, Huang et al., 2023). Key instantiations include modular agentic systems with structured intermediate representations, optimization-based backends, and feedback loops for iterative refinement.

1. System Architectures and Core Components

Chat-driven network management systems employ modular, pipeline architectures. Three representative frameworks are observed:

• Two-stage Chat–Optimize Pipeline (Miyaoka et al., 31 Dec 2025):
  1. Interpreter: Maps user chat into an “update direction” vector (increase/decrease/maintain) for each managed virtual service.
  2. Optimizer: Solves an integer linear program (ILP) to derive a configuration (e.g., VM placement, routing) fulfilling the interpreted intent subject to hard constraints.

• Agentic LLM Architectures (Lin et al., 24 Sep 2025):

  • Core components: Natural language interface, dialogue manager, LLM agent, state retrievers (vector database), intermediate representation (IR) compiler, external feedback subsystem.
  • Workflow: NL utterance → IR candidate via LLM → automated/human verification → IR compilation → vendor-specific CLI/config → deployment if approved.
• LLM-Orchestrated Modular Frameworks (ChatNet, (Huang et al., 2023)):
  • Modules: Analyzer (intent parser), Planner (stepwise decomposition), Calculator (computational/optimization backend), Executor (command issuance).
  • Central controller orchestrates modules, allowing multi-step reasoning and seamless integration with external tools (optimization solvers, monitoring systems).

These architectures facilitate a closed feedback loop: real-time network state informs intent extraction, which triggers verified configuration actions, yielding an updated state for further dialogue and action. Vendor-agnostic operation is achieved by using IRs and deterministic compilers for translation.

2. Natural Language Processing and Intent Extraction

Intent extraction from user chat leverages advanced NLP methods:

• Sentence-BERT + SVM Pipeline (Miyaoka et al., 31 Dec 2025): Translates chat into SBERT embeddings, then classifies intent using a support vector machine with RBF kernel. Achieves 89.3% accuracy with CPU inference latency ≈6 ms.
• LLM-Based Extraction (LLaMA-7B) (Miyaoka et al., 31 Dec 2025): Implements zero-shot prompt-based classification on GPU, requiring only 200 labeled examples. Attains 93.8% accuracy at ≈220 ms latency, and demonstrates robustness to ambiguous or multi-intent language.
• Pipeline in Agentic Systems (Lin et al., 24 Sep 2025): Uses chained prompting—domain retrieval, IR generation, error correction, and clarification—within an LLM agent (e.g., GPT-4o), flagged by log-probability scoring for confidence estimation.

A standard internal format for intent is a vector $\Delta = (\Delta_1,\ldots,\Delta_K)$ where each $\Delta_k \in \{-1,0,+1\}$ indicates per-VNS update direction. Semantic parsing outputs structured fields, which are subsequently mapped to configuration templates, constraints, or IR actions (Huang et al., 2023).

3. Configuration Synthesis and Optimization

Synthesizing feasible and efficient configurations from parsed intent employs optimization theory and formal constraint systems:

• ILP Formulation Example (Miyaoka et al., 31 Dec 2025):
  • Decision variables: $x_{i,k}$ (VM $k$ on server $i$), $y_{e,k}$ (service $k$ on link $e$).
  • Objective: $$\min_{x,y}\; \sum_{i,k}\alpha x_{i,k} + \sum_{e,k} \beta y_{e,k} + \sum_k \gamma | \sum_i x_{i,k} - \sum_i x^{prev}_{i,k} - \Delta_k |$$
  • Subject to CPU, memory, bandwidth, latency, and update-boundedness constraints.
  • Solved by Gurobi 10.0, achieving $22$–$180$ ms runtime for realistic topologies ($|S|\leq 20$, $|E|\leq 50$, $|K|\leq 15$).
• External Tool Integration (Huang et al., 2023):
  • The calculator invokes solvers (CPLEX, NetworkX, Matplotlib) via JSON function-call interfaces, enabling both optimization (capacity planning, ILP/LP) and visualization steps.
• IR Compilation and Vendor-Agnostic Deployment (Lin et al., 24 Sep 2025):
  • NL → IR via LLM; IR (JSON-YANG fragment) → deterministic translator for vendor CLI (Python mapping function).
  • Modular compilers abstract vendor differences, enabling consistent responses to high-level intents.

This approach ensures that only feasible (resource- and policy-compliant) configurations are produced, and automates rollback or user-side negotiation if infeasibility is detected.

4. Feedback, Verification, and Human-in-the-Loop

Robustness is enhanced by integrating verification and interactive feedback:

• Automated Syntax/Constraint Verification (Lin et al., 24 Sep 2025):
  • Tools like pyang validate generated IR; error summaries are re-injected into LLM prompt until all issues are resolved (up to 5 iterations).
• Human-in-the-Loop Approval (Lin et al., 24 Sep 2025):
  • Dialogue manager queries users for clarification or approval before deployment, diffing new vs. prior config for transparency.
• RLHF and Policy Updates (Lin et al., 24 Sep 2025):
  • Deployment outcomes are logged as binary rewards; periodic RL-based fine-tuning improves generation policies toward “high-reward” (successful) outcomes.

Iterative chat–optimize loops allow for ongoing dialogue, timely user correction, and automatic fallback or mitigation if constraints become unachievable.

5. Evaluation Methodologies and Quantitative Results

Empirical evaluations in the cited frameworks include:

Method/Module	Accuracy/F1	Latency	Context
SBERT+SVM (Intent)	89.3%	6 ms/utterance	Virtual network service update (Miyaoka et al., 31 Dec 2025)
LLM (Intent)	93.8%	220 ms/utterance	Same as above
ILP Engine	—	22–180 ms	Small datacenter to multi-user case
Agentic LLM (F1)	0.59	15 s/intent	Entity recognition (real user data) (Lin et al., 24 Sep 2025)
Lumi baseline	0.61	—	Entity recognition

Further, automated verification increased syntactic validity of IRs from 33.3% (no verifier) to 87.5% (with verification) (Lin et al., 24 Sep 2025). Retrieval-augmented prompting plus verification yielded a statistically significant increase in end-to-end accuracy (76.9%, $p=0.021$).

Complexity-accuracy-latency trade-offs are central. LLM-based interpretation improves overall intent extraction and ambiguity handling but incurs higher latency; hybrid strategies are recommended (default to fast SBERT, escalate to LLM on low-confidence) (Miyaoka et al., 31 Dec 2025).

6. Intermediate Representations and Vendor Agnosticism

A central design feature is the use of structured intermediate representations (IR). The dominant models are:

• JSON-based YANG IR (Lin et al., 24 Sep 2025):
  • Each IR entry: $\langle$device, [ (action, path, value) ]$\rangle$, with “append”/“remove” actions and YANG path scoping.
  • Enables a strict separation of parsing (NL → IR) and generation (IR → CLI/config), facilitating correctness checks and vendor-neutral workflows.
• Template/Fine-Tuned LLM Mapping (Huang et al., 2023):
  • Semantic fields drive template or code synthesis, which can be separately validated before deploying.

Vendor-specific compilers are minimal and deterministic, mapping paths and values to CLI or config fragments via static rulebases.

7. Limitations, Open Challenges, and Best Practices

Identified limitations and operational recommendations:

• Scalability: ILP and LLM inference latency grow with network and user population scale. Precomputed warm-starts, partial solution caches, and incremental solvers are advised (Miyaoka et al., 31 Dec 2025).
• Expressivity: Some complex, multi-step or vague intents challenge current NL interfaces; iterative clarification and logging help mitigate.
• Misclassification Risk: Misparsed intents may trigger unintended reallocation; hybrid pipelines and enforced “update-bound” constraints reduce risk.
• Auditability: All chat–to–action mappings should be logged for post hoc review and rollback.
• Data Collection and Evaluation: Large-scale data logging (session, intent, IR, verifier, CLI, deployment outcome) is essential for future fine-tuning and traceability (Lin et al., 24 Sep 2025).
• Privacy: User identifiers are anonymized; sensitive fields (e.g., IP, MAC) are redacted from logs (Lin et al., 24 Sep 2025).
• System Integration: Tight coupling with orchestration platforms (Kubernetes, ONAP, etc.) is necessary for rapid VM/config change propagation.

A hybrid interpreter, strict feasibility verification, and explicit human feedback points represent current best practices.

---

References:

• "Chat-Driven Optimal Management for Virtual Network Services" (Miyaoka et al., 31 Dec 2025)
• "An LLM-based Agentic Framework for Accessible Network Control" (Lin et al., 24 Sep 2025)
• "LLMs for Networking: Applications, Enabling Techniques, and Challenges" (Huang et al., 2023)