Knowledge Graph Fusion with Large Language Models for Accurate, Explainable Manufacturing Process Planning (2506.13026v1)

Abstract: Precision process planning in Computer Numerical Control (CNC) machining demands rapid, context-aware decisions on tool selection, feed-speed pairs, and multi-axis routing, placing immense cognitive and procedural burdens on engineers from design specification through final part inspection. Conventional rule-based computer-aided process planning and knowledge-engineering shells freeze domain know-how into static tables, which become limited when dealing with unseen topologies, novel material states, shifting cost-quality-sustainability weightings, or shop-floor constraints such as tool unavailability and energy caps. LLMs promise flexible, instruction-driven reasoning for tasks but they routinely hallucinate numeric values and provide no provenance. We present Augmented Retrieval Knowledge Network Enhanced Search & Synthesis (ARKNESS), the end-to-end framework that fuses zero-shot Knowledge Graph (KG) construction with retrieval-augmented generation to deliver verifiable, numerically exact answers for CNC process planning. ARKNESS (1) automatically distills heterogeneous machining documents, G-code annotations, and vendor datasheets into augmented triple, multi-relational graphs without manual labeling, and (2) couples any on-prem LLM with a retriever that injects the minimal, evidence-linked subgraph needed to answer a query. Benchmarked on 155 industry-curated questions spanning tool sizing and feed-speed optimization, a lightweight 3B-parameter Llama-3 augmented by ARKNESS matches GPT-4o accuracy while achieving a +25 percentage point gain in multiple-choice accuracy, +22.4 pp in F1, and 8.1x ROUGE-L on open-ended responses.

• The paper presents ARKNESS, a novel framework integrating LLMs and KGs to deliver precise, verifiable CNC process planning.
• It employs zero-shot entity extraction and beam search retrieval to convert machining documents into structured data for enhanced query response.
• The approach achieves up to 25 percentage points improvement in multiple-choice accuracy and an 8.1× boost in ROUGE-L scores for open-ended queries.

Knowledge Graph Fusion with LLMs for Accurate, Explainable Manufacturing Process Planning

This paper proposes an innovative framework, ARKNESS, which augments LLMs with knowledge graphs (KGs) to enhance manufacturing process planning in CNC machining. ARKNESS aims to provide numerically precise and verifiable answers through the combination of zero-shot KG construction with retrieval-augmented generation.

Introduction

Computer Numerical Control (CNC) machining process planning imposes significant cognitive demands on engineers, particularly due to complex, multifaceted decisions like tool selection and feed-speed optimization. Current solutions, such as rule-based systems, present static and often contextually rigid information, while LLMs, despite their flexibility, tend to hallucinate numerical values and lack traceability. This work proposes ARKNESS, a hybrid system employing a novel KG-LNN fusion strategy to address these issues by delivering numerically precise and contextually enriched insights.

Methodology

Knowledge Graph Construction

ARKNESS automates the KG creation process by utilizing heterogeneous machining documents. Using a zero-shot entity-relation extraction process powered by LLMs, such as GPT-4o, the system distills documents into structured triples (head, relation, tail) enriched with contextual information.

Graph Retrieval and Augmentation

The framework retrieves relevant subgraphs through a semantic search based on cosine similarity of embedded representations. A beam search expansion is used to include relevant neighbors, adapting to diverse query complexities and data richness, ensuring the retrieval of rich and relevant context for subsequent query processing by LLMs.

Figure 1: Overview of the framework with incorporating knowledge graph with LLM, illustrating the process from graph creation and storage to semantic search, match, and retrieval, and finally to LLM response.

Experimental Design and Results

Experiments were conducted using several LLMs, including both open-source models like Llama 3 and closed-source models like GPT-4o, to assess the impact of KG integrating on the performance of multiple-choice and open-ended questions across content and machining-specific categories.

Multiple-Choice Question Performance

ARKNESS-enhanced models demonstrated notable improvements, with accuracy gains of up to 25 percentage points in machining-specific questions. This exhibits the system's strength in grounding LLM outputs with domain-specific precision.

Figure 2: Graph of LLM accuracy as a function of graph traversal depth and Top K triple retrieval for a) content specific queries and b) machining specific queries.

Open-Ended Question Performance

For open-ended queries, large performance gains were achieved, particularly highlighted by up to an 8.1× increase in ROUGE-L scores. This underscores the utility of KGs in enhancing the structural and semantic fidelity of generative responses from LLMs.

Figure 3: Average performance over 10 runs for evaluated LLMs, comparing baseline outputs with those augmented by knowledge graph data.

Conclusion

The ARKNESS framework successfully integrates LLMs with KGs, yielding improved accuracy and explainability in CNC process planning tasks. By effectively anchoring LLM responses in validated, context-rich knowledge, ARKNESS addresses domain-specific challenges inherent in machining environments, making strides toward privacy-preserving, real-time inference on the shop floor. Future work should explore multimodal KG constructions to further enhance decision support in diverse manufacturing scenarios.