SynGen-Vision Pipeline for Rust Detection

• SynGen-Vision Pipeline is a modular framework integrating generative models, style transfer, noise removal, 3D simulation, and automated annotation for creating high-quality synthetic rust data.
• It employs advanced techniques such as Stable Diffusion and YOLOv5 training, achieving an impressive mAP50 accuracy of 0.87 in real-world rust detection tasks.
• The end-to-end automation and configurability of the pipeline enable rapid adaptation to simulate diverse industrial degradation scenarios with minimal manual intervention.

The SynGen-Vision Pipeline is a synthetic data generation and vision model training framework designed for industrial wear and tear detection, with particular focus on facilitating robust predictive maintenance tasks in data-scarce scenarios. By integrating generative vision LLMs, style transfer, advanced noise removal, 3D simulation, rendering, and automated annotation, it enables the production of high-quality, labeled training sets that significantly reduce the reliance on real-world manual data collection and annotation. The pipeline has demonstrated notable efficacy for rust detection in industrial objects, achieving superior model performance on real test images compared to alternative synthetic data pipelines.

1. Synthetic Data Generation Methodology

The SynGen-Vision Pipeline operationalizes the production of accurate training datasets through a sequential mechanism involving text-conditioned generative models and 3D simulation. The process is as follows:

• Prompt Refinement: A user provides a prompt describing the target wear and tear state (e.g., “complete rust,” “rust streaks”). The system refines the prompt to optimize the text input for the generative model.
• Texture Synthesis via Vision LLM: A vision LLM—specifically, stable-diffusion—is queried with the refined prompt to generate a base texture image simulating the desired rust or wear pattern.
• Style Transfer: The generated base texture is processed through a style transfer algorithm that merges it into the original texture of the 3D object model. This preserves crucial surface details such as symbols or pre-existing patterns, enabling accurate transfer of the generative texture without obliterating annotation-critical features.
• Noise Removal: The stylized texture often contains superfluous artifacts (such as watermarks or embedded text) from the generative process. An explicit noise removal stage is applied to produce a clean texture fit for rendering and annotation.

This texture synthesis and refinement is central to maintaining fidelity in downstream detection and ensures the generated images retain both realism and essential semantic details.

2. 3D Simulation, UV Mapping, and Rendering

After synthetic texture creation, the pipeline undertakes 3D data preparation and simulation:

• UV Mapping: The clean, stylized texture is mapped onto the 3D object mesh by updating the UV map, translating the 2D synthetic texture into the proper spatial correspondence on the 3D surface.
• 3D Scene Assembly: The textured 3D model is placed within a virtual industrial environment created in Blender, leveraging multiple variations of camera position, distance, and lighting to capture realistic and diverse object perspectives.
• Rendering and Automated Annotation: Each rendered scene is paired with automatically generated bounding box annotations. Annotation is tightly integrated with the rendering stage, ensuring each image aligns perfectly with ground-truth rust or defect labels.

This modular approach maximizes coverage of possible observational conditions, expanding the variability of data available for model training and validation.

3. Model Training and Evaluation Pipeline

The synthetic dataset produced is used to train object detection models on tasks such as rust recognition and localization:

• Training Protocol: Generated images and computed annotations are used to train a YOLOv5 detection model, supporting class labels such as “complete rust,” “rust streaks,” and “default” (no rust).
• Evaluation: The trained models are validated on real-world, manually annotated images of industrial components containing rust. The primary evaluation metric is mAP50 (mean average precision at IoU threshold 0.5), capturing localization and classification performance. Additional metrics such as Precision and Recall are reported.

Experimental results demonstrate that the full pipeline, including generative vision LLMing, style transfer, and noise removal, achieves a mAP50 score of 0.87, outperforming approaches using GenAI-only or GenAI with style transfer but without noise filtering.

4. Pipeline Integration and Modularity

The SynGen-Vision Pipeline is characterized by the following integration principles:

• End-to-End Workflow: The system unifies texture synthesis, 3D mesh texture application, scene simulation, rendering, and annotation in a fully automated process.
• Component Interchangeability: Each stage (prompt refinement, texture generation, style transfer, noise removal, UV mapping, rendering) can be recalibrated or substituted as new advances in generative modeling or simulation become available.
• Configurability and Extension: By varying text prompts and tuning generative model parameters, SynGen-Vision readily adapts to simulate and label an array of industrial defect or aging patterns (e.g., cracks, corrosion, general surface aging) with minimal manual intervention.

A LaTeX-style algorithmic summary is:

Input: 3D_Model, User_Prompt Output: Annotated_Synthetic_Images 1. Refined_Prompt ← Refine(User_Prompt) 2. Base_Texture ← StableDiffusion(Refined_Prompt) 3. Stylized_Texture ← StyleTransfer(Base_Texture, Texture_of(3D_Model)) 4. Clean_Texture ← NoiseRemoval(Stylized_Texture) 5. UV_Map ← UpdateUVMap(3D_Model, Clean_Texture) 6. For each variation in {Viewpoints, Lighting, Distance}: a. Render_Image ← Render(3D_Model_with(UV_Map), Current_Variation) b. Annotate(Render_Image) 7. Save Rendered Images and Annotations

5. Empirical Performance and Core Findings

Performance quantification centers on the mAP50 metric under real-world testing. Key findings include:

• Accuracy: The YOLOv5 model trained on SynGen-Vision synthetic data achieves mAP50 = 0.87. The pipeline’s highest performance is obtained via sequential application of Stable Diffusion, style transfer, and explicit noise removal.
• Ablative Analysis: Omission of style transfer or noise removal yields inferior detection accuracy, highlighting the necessity of all refinement steps for robust data generation.
• Generalization: The synthetic data generated enables detection models to generalize effectively to unseen, real rusted objects, implying successful transfer from synthetic to real domains.

A comparative table of pipeline components and their impact on detection accuracy, as reported in the data, is as follows:

Approach	mAP50 Score	Key Features
GenAI Only	Lower	Prompt→Stable Diffusion
GenAI + Style Transfer	Moderate	+ Alignment with 3D base texture
GenAI + Style Transfer + NoiseR	0.87	+ Noise/artifact removal

6. Extensions, Limitations, and Future Work

The design of SynGen-Vision accommodates future enhancements:

• Advanced Generative Models: As vision LLMs improve, greater realism and alignment in generated textures are anticipated.
• Attribute Localization: Enhancements may target pixel-level mapping of wear and tear attributes rather than bounding box detection alone.
• Broad Applicability: Extension to diverse forms of industrial degradation (cracks, abrasion, chemical corrosion) is straightforward via prompt and parameter modification.
• Noise Filtering Improvements: Research into more robust noise/artifact cleaning methods could yield further improvements in the fidelity and utility of synthetic data.

The pipeline’s flexibility and demonstrated impact on predictive maintenance position it as a significant modular framework for industrial computer vision, reducing the high costs and practical barriers associated with traditional data curation and manual annotation (Dubey et al., 5 Sep 2025).