SynGen-Vision: Synthetic Data Generation for training industrial vision models (2509.04894v1)

Abstract: We propose an approach to generate synthetic data to train computer vision (CV) models for industrial wear and tear detection. Wear and tear detection is an important CV problem for predictive maintenance tasks in any industry. However, data curation for training such models is expensive and time-consuming due to the unavailability of datasets for different wear and tear scenarios. Our approach employs a vision LLM along with a 3D simulation and rendering engine to generate synthetic data for varying rust conditions. We evaluate our approach by training a CV model for rust detection using the generated dataset and tested the trained model on real images of rusted industrial objects. The model trained with the synthetic data generated by our approach, outperforms the other approaches with a mAP50 score of 0.87. The approach is customizable and can be easily extended to other industrial wear and tear detection scenarios

• The paper introduces SynGen-Vision, a pipeline that integrates generative AI, style transfer, and noise removal to produce synthetic, photorealistic data for industrial rust detection.
• It leverages Stable Diffusion-based text prompts, 3D rendering with Blender, and automated annotation to efficiently generate datasets for training YOLOv5 models.
• Empirical results demonstrate substantial improvement (mAP50 up to 0.87) over baseline methods, highlighting robust generalization to real-world scenarios.

SynGen-Vision: Synthetic Data Generation for Training Industrial Vision Models

Introduction

The paper presents SynGen-Vision, an end-to-end pipeline for generating synthetic data to train computer vision (CV) models for industrial wear and tear detection, with a focus on rust detection. The motivation stems from the scarcity and high cost of acquiring and annotating real-world datasets for such tasks, especially for rare or progressive conditions like varying degrees of rust. The proposed approach leverages generative AI (GenAI) models, 3D simulation, and rendering engines to create photorealistic, annotated datasets that can be used to train object detection models. The pipeline is designed to be extensible to other industrial wear and tear scenarios beyond rust.

Methodology

Texture Generation from User Prompts

The pipeline begins with user prompts describing the desired wear and tear conditions. Stable Diffusion, a latent diffusion model, is used for text-to-image generation, producing texture images corresponding to different rust conditions. The authors note that prompt engineering is critical: appending terms like "texture" or "surface" yields cleaner, more usable outputs, while modifiers such as "complete," "streaks," or "spots" control the degree and pattern of rust.

Texture Synthesis with Style Transfer

Direct application of GenAI-generated textures to 3D models often results in loss of important object details (e.g., patterns, symbols). To address this, the pipeline incorporates a style transfer step, combining the original object texture (content) with the generated rust texture (style). This preserves the semantic and structural details of the base object while imparting the desired wear characteristics, resulting in more realistic and domain-appropriate textures.

Intelligent Texture Application and 3D Scene Generation

Noisy or unusable textures (e.g., those with watermarks, random text, or incorrect rust patterns) are filtered using image processing techniques. The selected textures are mapped onto 3D models via UV mapping in Blender. The models are then placed in 3D scenes, and multiple images are rendered from varying camera angles, lighting conditions, and distances to simulate real-world variability.

Synthetic Data Annotation

Rendered images are automatically annotated with bounding boxes and class labels corresponding to the rust condition, leveraging Blender's scripting capabilities. The annotation files include image paths, class labels, and bounding box coordinates, formatted for compatibility with standard object detection frameworks.

Model Training and Evaluation

A YOLOv5 object detection model is trained exclusively on the synthetically generated dataset (2000 samples, three classes: complete rust, rust streaks, no rust). Evaluation is performed on a manually annotated set of 100 real-world images. The pipeline is benchmarked against three synthetic data generation strategies: (a) GenAI only, (b) GenAI + Style Transfer, and (c) GenAI + Style Transfer + Noise Removal.

Results

The quantitative evaluation demonstrates a clear progression in performance across the three approaches. The final pipeline (GenAI + Style Transfer + Noise Removal) achieves a mean average precision at IoU 0.5 (mAP50) of 0.87 on real images, with high precision and recall across all classes. This is a substantial improvement over the GenAI-only baseline (mAP50 = 0.28) and the intermediate style transfer approach (mAP50 = 0.45). The results indicate that style transfer and noise filtering are essential for generating high-quality, domain-relevant synthetic data that generalizes to real-world conditions.

Key numerical results:

Approach	mAP50 (all classes)
GenAI only	0.28
GenAI + Style Transfer	0.45
GenAI + Style Transfer + Noise Removal	0.87

The model trained on synthetic data generalizes well to real images, including objects of different shapes, supporting the claim that the pipeline produces robust and transferable features.

Implications and Discussion

The SynGen-Vision pipeline addresses a critical bottleneck in industrial CV applications: the lack of annotated data for rare or progressive wear conditions. By combining GenAI, style transfer, and 3D rendering, the approach enables rapid, low-cost generation of large, diverse, and accurately labeled datasets. The pipeline's modularity and reliance on prompt-based control make it adaptable to a wide range of industrial inspection tasks beyond rust detection, such as crack detection, corrosion, or other forms of material degradation.

The strong empirical results—particularly the high mAP50 on real data—underscore the viability of synthetic data as a substitute for real-world data in industrial settings, provided that domain-specific realism is maintained through style transfer and noise filtering. The pipeline's use of open-source tools (Stable Diffusion, Blender, YOLOv5) further enhances its accessibility and reproducibility.

Potential limitations include the dependence on the quality of 3D models and the representativeness of the synthetic scenes. The approach may require further adaptation for highly complex or non-rigid objects, or for wear types with subtle visual cues. Additionally, while the pipeline automates annotation, the accuracy of bounding boxes and class labels is contingent on the fidelity of the 3D rendering and scene setup.

Future Directions

The authors suggest several avenues for future work:

• Extending the pipeline to simulate and annotate other forms of wear and tear, including cracks, dents, or multi-modal degradation.
• Incorporating more advanced generative models as they become available, to further improve texture realism and diversity.
• Enhancing the annotation process to support instance segmentation or pixel-wise labeling for more granular tasks.
• Integrating domain adaptation techniques to further bridge the gap between synthetic and real data distributions.
• Scaling the pipeline for large-scale industrial deployment, including automated 3D model acquisition and scene generation.

Conclusion

SynGen-Vision demonstrates that a carefully designed synthetic data generation pipeline, leveraging GenAI, style transfer, and 3D rendering, can produce high-quality datasets for training industrial vision models. The approach achieves strong performance on real-world rust detection tasks, validating the utility of synthetic data in domains where real data is scarce or costly to obtain. The pipeline's extensibility and reliance on open-source components position it as a practical solution for a broad range of industrial CV applications, with significant implications for predictive maintenance and automated inspection.