FlowExtract: Hybrid Flowchart Extraction

• FlowExtract is a hybrid computer vision pipeline that converts ISO 5807-compliant flowcharts into queryable directed graphs through sequential node detection, OCR, and edge reconstruction.
• It leverages YOLOv8 and EasyOCR to achieve near-perfect node detection and robust text extraction, ensuring high classification accuracy and minimal OCR errors.
• The arrowhead-anchored edge reconstruction method delivers precise graph connectivity and a six-fold increase in edge F1 score over traditional vision-language approaches.

FlowExtract is a hybrid computer vision pipeline for transforming scanned or PDF-embedded maintenance flowcharts, specifically those compliant with the ISO 5807 standard, into queryable directed graphs. The system addresses the challenge of extracting procedural knowledge from flowchart diagrams, which are widely used in manufacturing for asset lifecycle management but remain inaccessible to modern operator-support and knowledge-retrieval systems. Unlike vision-LLMs that struggle with connection topology recovery, FlowExtract decomposes the task into three sequential stages—node detection, text extraction, and edge reconstruction—applying specialized methods to each. The result is a high-precision, robust workflow that converts industrial flowcharts into structured, machine-interpretable formats suitable for downstream applications (Avalle et al., 8 Apr 2026).

1. Architectural Overview

FlowExtract consists of three distinct stages:

1. Node Detection: Flowchart symbols—including Process, Decision, Document, Terminator, Connector, and Arrowhead types—are localized and classified within the input diagram.
2. Text Extraction: Each detected node, excluding arrowheads, is cropped and subjected to optical character recognition (OCR) to recover embedded text. Labels that appear along arrows (e.g., “ja”/“nee”) are also extracted.
3. Edge Reconstruction: Directed graph edges are established by anchoring on detected arrowheads. Arrowhead orientation is leveraged to infer edge directionality, and line tracing is performed to connect arrowheads to source and target nodes. Decision labels are assigned to appropriate edges.

This modular separation of symbol identification and connectivity parsing is foundational: it enables FlowExtract to circumvent the limitations of monolithic approaches, where a single model attempts to conflate perceptual and relational reasoning (Avalle et al., 8 Apr 2026).

2. Node Detection Methodology

The node detection component utilizes YOLOv8s, a small-variant object detector pre-trained on COCO and fine-tuned on a domain-specific dataset. The pipeline is trained to classify six symbol types relevant to ISO 5807 flowcharts.

• Training Regimen: 25 annotated training images comprising 823 symbol instances are used, with a batch size of 8, input resolution of 640×640, and a cosine annealing learning rate schedule starting at 0.01. Default YOLOv8 loss functions—binary cross-entropy for classification and CIoU for bounding boxes—are employed.
• Data Augmentation: To counter strong class imbalance (notably a 15-fold difference between arrowheads and connectors), mosaic augmentation and HSV brightness jitter (+/- 30%) are used. Spatial augmentations (e.g., flips, rotations) are excluded to preserve symbol spatial integrity.
• Detection Performance: The model attains a node detection F1 of 98.8% (precision = 98.4%, recall = 99.2%), node classification accuracy of 97.6%, and arrowhead detection accuracy of 73.1%. High precision in classifying and localizing nodes is instrumental in the accuracy of subsequent stages (Avalle et al., 8 Apr 2026).

3. Text Extraction via OCR

Text extraction is handled using EasyOCR, which provides language flexibility and robustness to variations in font and scan artifacts prevalent in maintenance documentation.

• Procedure:

1. Detected nodes are cropped to bounding boxes, and EasyOCR is executed to recognize the included text.
2. A separate OCR pass is conducted on the full diagram to capture edge-embedded labels, such as “ja” and “nee,” potentially crucial for decision nodes.
3. Recognized text is systematically linked with the associated symbol or, in the case of edge labels, with the respective graph edge.

• Accuracy: The OCR component achieves a character-level recognition rate of 99.2%, indicating negligible semantic loss due to text extraction errors (Avalle et al., 8 Apr 2026).

4. Edge Reconstruction: Arrowhead-Anchored Line Tracing

FlowExtract introduces an arrowhead-anchored edge reconstruction algorithm, a key innovation that prioritizes edge precision by using arrowheads as direction anchors rather than generic line detection.

• Arrowhead Orientation: For each arrowhead, a binary mask is computed and the convex hull’s two most distant points are determined. The endpoint nearest a node bounding box is classified as the tip (“pointy” end), while the other as the base (“blunt” end). The direction vector is defined by the ordered pair.
• Line Tracing: The full image is preprocessed (grayscale conversion, Canny edge binarization, optional dilation) and subjected to the Probabilistic Hough Transform, yielding a set of line segments. A graph is constructed where segment endpoints serve as nodes and touching/intersecting segments are linked.
• Source-Node Association: A breadth-first search originates at the arrowhead blunt point, traversing line-segment connectivity to a node within a threshold distance (δ, e.g., 10 pixels). The target node is assigned as the nearest to the arrowhead tip.
• Edge Labeling: If an edge label (“ja”/“nee”) is detected within a radius ε of the edge midpoint, it is assigned to the reconstructed edge.
• Output Format: The final graph is structured as a JSON list of nodes (with IDs, types, bounding boxes, text) and directed edges (with source/target IDs and labels).

This algorithmic separation—anchoring on arrowheads and tracing connectivity backward—delivers high-precision edge extraction and a practical performance advantage over vision-LLM paradigms (Avalle et al., 8 Apr 2026).

5. Empirical Evaluation and Comparative Performance

FlowExtract is evaluated on a corpus of 35 Dutch industrial troubleshooting flowcharts, manually annotated with 1,145 nodes, 498 arrowheads, and 62 decision labels. The dataset is split into 25 training, 3 validation, and 7 test diagrams.

Task/Metric	FlowExtract	Qwen2-VL-7B	Pixtral-12B
Node F1 (%)	98.8	34.0	29.5
Node Classification (%)	97.6	—	—
OCR Accuracy (%)	99.2	—	—
Edge Precision (%)	85.5	—	—
Edge Recall (%)	54.6	—	—
Edge F1 (%)	66.7	10.7	1.5
Edge Label Accuracy (%)	73.8	—	—

FlowExtract achieves a six-fold increase in edge F1 over vision-language baselines, with near-perfect node recovery. The system is characterized by high edge precision but moderate recall, attributed to missed arrowheads. There are negligible OCR errors. This performance profile suggests particular suitability for applications where spurious edge introduction is unacceptable (Avalle et al., 8 Apr 2026).

6. Constraints and Potential Extensions

• Standards and Generalization: The method assumes ISO 5807-compliant diagrams with standardized shapes and clear arrowheads. Diagrams deviating from this standard (for example, hand-drawn or non-standard layouts) would likely reduce performance, necessitating retraining or alternative detection models.
• Arrowhead Bottleneck: Arrowhead detection constitutes the principal source of recall errors for edge reconstruction. Missed or occluded arrowheads preclude corresponding edge recovery.
• Page Connectivity: The system does not resolve cross-page connectors (circular symbols with labels), which are present in multi-document procedures.
• Future Directions: Proposed enhancements include multi-scale detection or segmentation heads for improved arrowhead recall, a fallback “lines-first” mode to augment recall at some cost to precision, cross-document connector resolution, and empirical studies of cognitive load reductions in operator workflows (Avalle et al., 8 Apr 2026).

7. Technical Significance and Applications

By employing domain-adapted object detection (YOLOv8), robust OCR (EasyOCR), and a custom arrowhead-anchored geometric reasoning algorithm, FlowExtract delivers procedural graph structures from industrial flowcharts with high fidelity. This enables maintenance engineers and organizations to digitize legacy procedures for automated querying, troubleshooting support, and integration into broader asset management systems. The approach bridges a critical gap left by vision-LLMs, advancing the practical extraction of structured procedural knowledge from document images (Avalle et al., 8 Apr 2026).