Rethinking Text-to-Image as Semantic-Aware Data Augmentation for Indoor Scene Recognition

Abstract: In the realm of computer vision, indoor image recognition presents challenges due to the intricate interplay of lighting conditions, occlusions, and diverse object arrangements within confined spaces. To address the lacks of training indoor images, we introduce a novel approach leveraging Stable Diffusion (SD) for the generation of synthetic images, which serve as a powerful data augmentation tool. The utilization of SD offers a principled framework for synthesizing diverse and realistic indoor scenes, thereby enriching the training data pool for robust indoor image recognition models. Experimental findings on the MIT Indoor Scene dataset reveal the potential of our proposed approach in enhancing the training of deep models when authentic data is limited. Furthermore, to prevent the misuse of SD synthetic images, we introduce a counter measure based on DIffusion Reconstruction Error (DIRE). The powerful DIRE presentation enables training robust classifiers only using lightweight deep models. Experiments show that our approach can perfectly recognize SD generated images with the accuracy of 100% using MobilenetV3.

• The paper introduces a semantic-aware data augmentation pipeline using Stable Diffusion to generate high-fidelity indoor scenes.
• It employs CLIP-based prompt generation and cosine similarity measures to ensure contextual fidelity and remove duplicate or outlier images.
• Quantitative evaluations show a notable accuracy boost with EfficientNetV2 and perfect synthetic image detection using DIffusion Reconstruction Error.

Semantic-Aware Synthetic Data Augmentation via Stable Diffusion for Indoor Scene Recognition

Introduction

The paper "Rethinking Text-to-Image as Semantic-Aware Data Augmentation for Indoor Scene Recognition" (2606.18555) presents a principled framework for leveraging generative diffusion models, specifically Stable Diffusion (SD), as a tool to synthesize semantically relevant indoor scenes for data augmentation. This approach addresses significant challenges in indoor image recognition, particularly data scarcity, heterogeneity in indoor environments, and the limitations of conventional augmentation techniques in capturing complex visual attributes of indoor scenes. Furthermore, the paper introduces a mechanism for robust detection of synthetic data using DIffusion Reconstruction Error (DIRE), ensuring that SD-generated content can be reliably distinguished from authentic imagery, which is critical for both security and ethical deployment.

Semantic Data Augmentation Pipeline

The synthetic data augmentation pipeline is structured into three main phases: prompt generation, image synthesis, and duplicate/outlier removal. Prompts are derived using a CLIP-based text decoder to ensure contextual fidelity; these are then used to steer SD’s text-to-image module, with the Realistic Vision checkpoint producing high-fidelity, visually realistic images. Subsequently, image similarity is analyzed via CLIP feature embedding and cosine similarity measures, removing near-duplicates or outliers to yield a dataset with both novelty and class consistency.

Figure 1: Dataset synthesis process integrating prompt generation, image generation, and duplicate/outlier removal for enriched indoor scene diversity.

Empirical results on the MIT Indoor Scene dataset exhibit that the distribution of augmented images closely mirrors that of authentic scenes, maintaining semantic and statistical consistency. Feature space visualization (using t-SNE on CLIP features) demonstrates strong overlap between original and synthetic domains, evidencing that generative augmentation preserves class-relevant manifold structure.

Figure 2: Feature space visualization via t-SNE, showing overlapping distributions between original and SD-augmented images.

Numerical Performance and Data Scarcity Analysis

Quantitative evaluation demonstrates that EfficientNetV2 trained with SD-augmented images achieves an accuracy of 84.2% on the MIT dataset, marking a notable increase (up to +1.9% in low-data regimes) compared to training with only authentic data. When the training set is supplemented by synthetic imagery at a 1:1 ratio, models consistently outperform those trained under traditional augmentation, substantiating the hypothesis that diffusion-based synthesis offers richer intra-class variation and improved generalization.

Figure 3: Visual examples from the MIT dataset (top) and SD-generated augmented images (bottom) highlighting semantic and visual fidelity.

This result is significant for applications in settings where data collection is impractical or prohibitive, confirming that semantic-aware synthetic augmentation can bridge gaps in sampling and enable robust model training with reduced annotation or acquisition overhead.

Defense Mechanisms: Detection of Synthetic Images Using DIRE

A potential risk with generative augmentation is the misuse or undetected deployment of synthetic data in operational systems. The paper proposes a defense mechanism that utilizes DIRE as a discriminative feature representation. DIRE quantifies reconstruction error differences between original and synthetic images processed by diffusion inversion and reconstruction. Training lightweight deep models (e.g., MobilenetV3) on DIRE representations yielded perfect classification accuracy (100%) in distinguishing SD-generated from real images, with only 4M parameters and requiring minimal fine-tuning.

Figure 4: Two-phase pipeline for SD image recognition, employing DIRE image creation followed by fake image classification.

This approach sharply outperforms RGB-only or combined representations, highlighting that diffusion-specific artifacts are both detectable and highly separable with compact architectures. Such techniques serve as effective safeguards against synthetic media manipulation and support responsible generative model deployment.

Practical and Theoretical Implications

From a practical standpoint, semantic-aware synthetic augmentation via SD fundamentally expands the diversity and complexity of training data in indoor scene recognition, enabling effective model training in low-sample or sensitive domains. Theoretical implications include reinforcement of the hypothesis that diffusion models can generate sample distributions that closely approximate real class manifolds, and that specialized reconstructions (DIRE) offer a robust methodology for synthetic content detection.

Further work could explore domain adaptation between synthesized and real distributions, adversarial robustness, and expansion of the DIRE framework to multi-modal synthetic media types. Integration of generative and discriminative modules suggests future architectures for automated, secure data pipelines in computer vision.

Conclusion

This work formalizes a framework for using text-to-image diffusion models as a semantically rich, context-aware augmentation strategy for indoor image recognition. Experimental evidence validates performance enhancement, especially under data-scarce conditions. The introduction of DIRE for synthetic content detection achieves high efficiency and robustness, providing critical support for ethical and secure generative AI practice. The methodologies outlined expand the toolkit for supervised learning across vision domains, and form a basis for future research on generative data augmentation and synthetic media regulation.