FlowExtract: Procedural Knowledge Extraction from Maintenance Flowcharts

Abstract: Maintenance procedures in manufacturing facilities are often documented as flowcharts in static PDFs or scanned images. They encode procedural knowledge essential for asset lifecycle management, yet inaccessible to modern operator support systems. Vision-LLMs, the dominant paradigm for image understanding, struggle to reconstruct connection topology from such diagrams. We present FlowExtract, a pipeline for extracting directed graphs from ISO 5807-standardized flowcharts. The system separates element detection from connectivity reconstruction, using YOLOv8 and EasyOCR for standard domain-aligned node detection and text extraction, combined with a novel edge detection method that analyzes arrowhead orientations and traces connecting lines backward to source nodes. Evaluated on industrial troubleshooting guides, FlowExtract achieves very high node detection and substantially outperforms vision-LLM baselines on edge extraction, offering organizations a practical path toward queryable procedural knowledge representations. The implementation is available athttps://github.com/guille-gil/FlowExtract.

• The paper introduces FlowExtract, a hybrid pipeline that separates node detection from edge reconstruction in maintenance flowcharts.
• It employs YOLOv8 for element detection, EasyOCR for text extraction, and geometric analysis for precise edge detection, achieving node F1 of 98.8% and edge F1 of 66.7%.
• The approach supports human-in-the-loop digitization of ISO 5807-standardized diagrams, significantly improving the automation of legacy maintenance documentation.

Procedural Knowledge Extraction from Maintenance Flowcharts: A Technical Review of FlowExtract

Introduction and Context

Industrial maintenance procedures are frequently encoded in flowcharts within static PDF files or scanned images. These diagrams serve as repositories of procedural knowledge vital to asset lifecycle management. However, the information encapsulated in such legacy documentation remains largely inaccessible to automated systems. While state-of-the-art vision-LLMs (VLMs) have substantially advanced document image understanding, they consistently underperform in reconstructing the connection topology of flowcharts—limiting the conversion of procedural diagrams to actionable, queryable knowledge.

FlowExtract (2604.06770) addresses this gap by proposing a hybrid extraction pipeline that separates node (element) detection from connectivity (edge) reconstruction, leveraging both deep neural techniques and classical image processing. FlowExtract is explicitly architected for ISO 5807-standardized flowcharts and prioritizes connection precision, aligning its outputs with human-in-the-loop digitization workflows prevalent in industrial contexts.

Figure 1: Human-in-the-loop workflow for knowledge extraction from legacy maintenance documentation.

Pipeline Architecture and Methods

The FlowExtract pipeline consists of three modular stages: node detection, text extraction, and edge detection.

Node Detection

Core flowchart elements—including process boxes, decision diamonds, terminators, connectors, documents, and arrowheads—are identified using YOLOv8 (small variant), trained with extensive mosaic and brightness augmentation strategies to mitigate class imbalance. Arrowheads form a distinct detection class; their accurate localization is crucial for downstream edge extraction.

Text Extraction

Text within detected node regions is recognized using EasyOCR, taking advantage of its robust multilingual capabilities and resilience to technical fonts and imperfect scan quality, prevalent in industrial documentation.

Edge Detection

Connectivity reconstruction is performed using a novel method anchored on arrowhead detection. The pipeline first identifies arrowheads and determines their orientation to ascertain flow direction. Using the orientation, it performs backward line tracing (via the Probabilistic Hough Transform) to designate the source node, while the arrowhead points to the target. This approach sharply contrasts with approaches that only extract connectivity based on generic line detection, eschewing directionality, which is semantically significant in procedural flows.

Figure 2: FlowExtract pipeline overview and edge detection methodology, including arrowhead orientation analysis and complex connection tracing.

Figure 3: Flowchart symbols taxonomy (ISO 5807:1985), including arrowheads.

Empirical Evaluation

The system is evaluated on a manually annotated dataset sourced from real-world industrial troubleshooting guides, comprising over one thousand flowchart elements and connection edges. The evaluation delineates individual pipeline stages, with particular focus on the limitations of competing VLM-based systems.

Node Detection and Text Extraction Performance

FlowExtract achieves node detection F1 of 98.8%, with per-class rates above 97% except for arrowheads (73.1% accuracy), and a text extraction character-level accuracy of 99.2%. These figures showcase the efficacy of leveraging YOLOv8 and robust augmentation for symbol detection in low-resource, imbalanced settings.

Edge Detection

The core bottleneck—the extraction of directed edges—is addressed with the arrowhead-anchored methodology, yielding edge F1 of 66.7% with a pronounced precision of 85.5% and recall of 54.6%. This stands in stark contrast to VLM pipelines (Qwen2-VL-7B, Pixtral-12B), whose edge F1 rates do not exceed 10.7%, evidencing an order-of-magnitude improvement in topological correctness.

Figure 4: Example extraction results; original textual node content redacted for proprietary reasons—visualization demonstrates structure preservation and accurate connectivity assignment.

Architectural Rationales and Design Implications

FlowExtract is shaped by several core design imperatives:

• Separation of Element and Connectivity Extraction: The results empirically substantiate the importance of isolating node detection from edge recovery, particularly for standardized diagrams where node shapes and connectivity cues (arrowheads) are visually distinct and semantically orthogonal.
• Arrowhead-Anchored Precision: The pipeline's prioritization of explicit directional cues ensures high-confidence connectivity graphs, minimizing spurious or hallucinated edges that would burden human validators. This is especially pertinent in mission- and safety-critical industrial domains, where commission errors (false positives) bear a higher operational cost than omission errors (false negatives).
• Alignment with Human-in-the-Loop Workflows: High precision, even at the expense of recall, harmonizes with the cognitive strengths of human validators—enabling them to augment incomplete but reliable graph skeletons, rather than perform error detection on noisy, fully-automated outputs.
• Transfer and Generalization Limits: While effective for ISO 5807 flowcharts, FlowExtract requires adaptation for domains lacking standardized symbology or relying on line-only connectivity. The exclusive focus on arrowheads inherently limits recall in dense or degraded documents where arrowheads are poorly rendered or occluded.

Comparative Analysis with Vision-LLMs

FlowExtract's empirical supremacy over VLMs is attributable to the latter's inability to reliably interpret two-dimensional spatial relationships and directional cues in crowded diagrams. VLMs, optimized for node classification and text recognition, lose crucial positional and topological data during textual serialization of spatial layouts. Even explicit spatial prompting or hybrid VLM+segmentation architectures (e.g., arrow-guided prompts, Segment Anything) do not close this gap, as demonstrated by edge F1 scores consistently below 0.3 across various literature benchmarks. FlowExtract—by limiting neural methods to bounded-object detection and delegating connectivity to geometric analysis—sidesteps these intrinsic VLM limitations.

Future Prospects and Open Challenges

The study identifies several priority directions for further research:

• Arrowhead Detection Enhancement: The main bottleneck remains arrowhead recall. Incorporating architectures with explicit keypoint detection or context-aware augmentation may improve robustness against occlusion and scale variance.
• Extensibility to Non-Standard/Degraded Documents: Broadening applicability requires adaptation to document types which either lack standardized arrowheads or present significant image quality challenges.
• Cross-Document Reference Resolution: Integrating connector logic to enable spanning of partial flowcharts across multiple files or pages remains unaddressed and is crucial for industrial pipeline completeness.
• Human-AI Collaboration Optimization: Further investigation into optimal integration strategies—task allocation, interface design, feedback mechanisms—will be essential for maximizing efficiency and reliability in human-in-the-loop extraction regimes.

Conclusion

FlowExtract establishes a high-precision, hybrid pipeline for extracting procedural knowledge graphs from standardized maintenance flowcharts, outperforming both open and closed VLMs by a wide margin on connection extraction tasks. Its design principles—anchoring connectivity on explicit directional cues and modularizing neural and geometric analysis—position it as a robust foundation technology for organizations aiming to unlock and operationalize procedural knowledge entombed in legacy documentation. Adoption of such hybrid precision-oriented pipelines will play an increasingly central role in industrial digital transformation, enabling the creation of actionable knowledge graphs from previously inaccessible visual procedural artifacts.