MureCom Dataset for Image Composition

• MureCom dataset is a large-scale image composition benchmark that supports reference-based object insertion and semantic harmonization in diverse scenes.
• It comprises 32 object categories with 640 annotated backgrounds and 480 multi-view reference images, enabling comprehensive evaluation of generative models.
• The dataset emphasizes compositional fidelity by using hand-annotated placement regions to challenge models with varied lighting, clutter, and scene complexity.

The MureCom (Multi-reference Composition) dataset is a large-scale image composition benchmark introduced to study reference-based object insertion, out-of-context editing, and multi-view composition. It is widely used as an evaluation platform for generative compositing algorithms, especially those leveraging few-shot visual references to achieve realistic and semantically controlled foreground placement within diverse backgrounds. The dataset consists of RGB images covering 32 everyday object categories, annotated with precise rectangular regions for object placement, and is structured to provide rich scene diversity as well as challenging harmonization scenarios for contemporary inpainting and generative diffusion models (Lu et al., 2023, Tong et al., 15 Apr 2026).

1. Dataset Origin and Core Objectives

MureCom was originally introduced in "DreamCom: Finetuning Text-guided Inpainting Model for Image Composition" (Lu et al., (Lu et al., 2023) as a testbed for compositional image generation (Lu et al., 2023). Its design centers around evaluating models' ability to insert a reference foreground object into an arbitrary target background, guided by a placement annotation but without leveraging explicit pixel-level background masks or part-level pose labels. In subsequent research—most notably "PostureObjectStitch" (Tong et al., 15 Apr 2026)—MureCom has been adopted as a general-domain composition baseline for evaluating industrial image generation systems, though its imagery and annotations are not specialized for industrial objects or assembly semantics.

The primary focus is compositional fidelity: ensuring that inserted objects match both semantic and appearance constraints defined by a reference set, while achieving seamless harmonization with backgrounds that display significant diversity in lighting, scene content, and clutter.

2. Data Collection Protocol and Image Statistics

Construction Pipeline

• Background selection: Backgrounds are sourced from the R-FOSD dataset, covering diverse natural scenes with 32 semantic categories (including objects such as horse, guitar, train, and cup).
• Bounding box annotation: For each background, a single rectangular region is hand-annotated, denoting a plausible insertion zone.
• Foreground acquisition: For each category, three distinct foreground instances are collected. Each is photographed in isolation from five different viewpoints, resulting in 480 total reference images.
• Composition pairing: Any foreground of a given category can be paired with any of its 20 corresponding category backgrounds, leading to 1,920 unique pairings.

Key Statistics

Attribute	Value	Details
Categories	32	Everyday objects
Backgrounds	640	20 per category
Foreground objects	96	3 per category
Reference images	480	5 per object; multi-view
Total pairs	1,920	32 × 20 × 3
Modality	RGB only	No depth, IR, or non-RGB sensors

Original image resolutions range from 256×256 up to 1024×768; all processing, evaluation, and model training reshape or center-crop images to 512×512 pixels to ensure uniformity and compatibility with contemporary diffusion pipelines (Lu et al., 2023, Tong et al., 15 Apr 2026).

3. Annotation Structure and Semantic Scope

Annotation in MureCom is optimized for reference-based composition tasks:

• Backgrounds: Each has a single (xmin, ymin, xmax, ymax) rectangle specifying the compositional region.
• Foregrounds: Each is provided as isolated reference crops with tight bounding boxes and binary masks (for masking during model training, not benchmark evaluation).
• Labels: Both backgrounds and foregrounds are tagged with a semantic category identifier (1–32).
• No anomaly, part, or pose labels: There are no annotations for assembly, part segmentation, or anomalous structure, making the dataset composition-centric and unsuited for direct use in anomaly-detection benchmarks or part relationship modeling.
• No pixel-level masks for backgrounds: Only bounding-box "placement regions" per background.

This structure targets compositional realism and reference-based object fidelity, not fine-grained spatial reasoning or semantic parsing.

4. Image Diversity and Scene Complexity

The dataset encompasses substantial intra-class and scene diversity:

• Lighting: Outdoor and indoor, bright, dusk, mixed/shadowed.
• Viewpoints and poses: Multi-angle, including side-, top-, and foreshortened views.
• Occlusion: Some foreground references include partial occlusion (e.g., object behind scene elements).
• Scale and aspect ratio: Placement boxes vary from small (≈10% image area) to large (≈50%).
• Background complexity: From minimal texture (skies, walls) to high clutter (indoors, vegetation, natural scenes).
• Scene ambiguity: Backgrounds may feature category-matching objects or distractors, necessitating accurate identity preservation by the compositing algorithm.

All of these factors present compositional and harmonization challenges, directly testing the limits of generative models' semantic and photometric adaptation capacities (Lu et al., 2023).

5. Evaluation Protocols and Benchmark Metrics

No standardized train/validation/test splits are distributed with the dataset. In published protocols (Lu et al., 2023, Tong et al., 15 Apr 2026), typical usage includes:

• Utilizing all reference views for model finetuning.
• Testing on all 20 backgrounds, or sampling 2 backgrounds per category for user studies and analysis (yielding 64 evaluation cases).
• Generating multiple samples per pair (e.g., 4 per pair) to account for stochasticity in diffusion-based generation.

Quantitative Metrics

MureCom evaluation adopts and extends benchmarks from DreamEditBench:

• CLIP-I: Cosine similarity between CLIP image-encoder embeddings of generated and reference images. Higher values denote stronger foreground fidelity.
• DINO: Cosine similarity between DINO ViT embeddings, emphasizing self-supervised match of foreground characteristics.
• LPIPS: Learned Perceptual Image Patch Similarity, computed between generated and reference images; lower values indicate stronger perceptual similarity.

$$\mathrm{LPIPS}(G,R) = \sum_{\ell}\frac{1}{H_\ell W_\ell}\sum_{h,w} \| w_\ell \odot ( \phi_\ell(G)_{h,w} - \phi_\ell(R)_{h,w} ) \|_2^2$$

where $w_\ell$ are learned weights, $\phi_\ell$ are deep-network activations.

• SSIM: Structural Similarity Index, computed on the background region (foreground masked out) to measure background preservation.

$$\mathrm{SSIM}(x,y) = \frac{(2\mu_x\mu_y + C_1)\,(2\sigma_{xy} + C_2)}{(\mu_x^2+\mu_y^2+C_1)\,(\sigma_x^2+\sigma_y^2+C_2)}$$

No anomaly-, detection-, or classification-specific metrics are natively defined, nor does the dataset support instance-segmentation or keypoint evaluation.

6. Applications, Limitations, and Usage in Contemporary Research

MureCom is leveraged as a standard benchmark for compositional realism and semantic fidelity in object insertion models, particularly those employing diffusion-based architectures or text-guided inpainting. Its design enables methodical evaluation of:

• Appearance and pose transfer from limited reference images.
• Model capacity for semantic harmonization and illumination transfer.
• Background preservation capabilities in complex and distractor-rich environments.

In "PostureObjectStitch" (Tong et al., 15 Apr 2026), MureCom is employed to establish that a proposed industrial-object image synthesis pipeline generalizes beyond the industrial domain and does not overfit to assembly-specific cues. The absence of anomaly and industrial assemblies in MureCom positions it as an unbiased, real-world baseline.

Limitations

• No anomalies: The dataset contains no anomalous samples or anomaly annotations; anomaly detection studies require alternative or supplemented datasets.
• No industrial components/assembly information: The dataset is composition-specific and does not reflect industrial or manufacturing assembly relationships.
• No fixed data splits: Users must define experimental splits, which can introduce protocol-dependent variability across papers.

A plausible implication is that while MureCom enables rigorous evaluation of composition methods, applications to industrial anomaly detection or assembly reasoning necessitate additional, domain-specific datasets and annotation schemas.

7. Summary Table

Attribute	Value
Categories	32
Backgrounds per category	20 (total 640)
Foreground instances per category	3 (total 96)
Multi-view refs per object	5 (total 480 reference images)
Image modality	RGB only
Anomalies	None
Placement annotation	Rectangular region in each background
Resolution (preprocessed)	512 × 512 px
Train/Val/Test split	None provided
Metrics	CLIP-I, DINO, LPIPS (↓), SSIM (↑)

MureCom constitutes a comprehensive general-domain benchmark for image composition, supporting research on reference-driven object insertion, generative harmonization, and quantitative evaluation of compositional fidelity within diverse and challenging scenes (Lu et al., 2023, Tong et al., 15 Apr 2026).