InsertPipe: Automated Video Insertion Pipeline

• InsertPipe is an automated data curation pipeline that generates diverse subject–scene pairs for mask-free video insertion without the need for manual mask annotation.
• It integrates three sub-pipelines—RealCapture, SynthGen, and SimInteract—to ensure high data diversity, quality control, and natural subject-scene equilibrium.
• The pipeline supports end-to-end training in the OmniInsert framework, enabling photorealistic insertion harmonization and improved performance benchmarks.

InsertPipe is an automated data curation pipeline designed for mask-free video insertion tasks within the OmniInsert framework. Its primary purpose is to construct diverse, high-quality cross-pair training data that enables large-scale, robust training of video insertion models from single or multiple visual references. InsertPipe’s innovations address three major challenges in this domain: data scarcity, subject–scene equilibrium, and robust insertion harmonization, thereby supporting highly generalizable video generation architectures.

1. Motivation and Conceptual Underpinnings

InsertPipe confronts the severe lack of paired training data required for mask-free video insertion, which involves seamlessly integrating arbitrary reference subjects into diverse source videos without reliance on precomputed spatial masks. Unlike previous approaches that either depend on hand-curated pairings or expensive mask annotation, InsertPipe automatically curates large and diverse datasets by synthesizing subject–scene cross-pairs, enabling end-to-end supervised training for video insertion architectures. The pipeline leverages multiple data sources, modalities, and AI systems—including detection, vision–LLMs (VLMs), LLMs, generative models, and rendering engines—with the explicit goal of covering the full spectrum of subject–scene interactions and visual variability encountered in the wild (Chen et al., 22 Sep 2025).

2. Architecture and Component Pipelines

InsertPipe constructs datasets through three principal sub-pipelines, each targeting a different aspect of cross-pair diversity:

Sub-Pipeline	Data Source	Key Processing Steps	Coverage/Strength
RealCapture	Unstructured real videos	Scene segmentation, VLM captioning, subject extraction, detection/tracking, subject erasure, cross-video matching (CLIP, face embeddings)	Captures real-world scene/subject configuration, avoids copy–paste artifacts
SynthGen	LLM/T2I/I2V generators	Prompt bucketization (≈300 subject, ≈1000 scene categories), prompt-driven I2V, VLM scoring for consistency, subject removal	Massive scene and subject diversity, detail preservation via VLM scoring
SimInteract	Physically-based rendering	Houdini-based synthetic actors, layout priors, motion assets, paired rendering with and without subject	Complex subject–scene interactions, highly controllable simulation

RealCapture processes long real-world videos by dividing them into scene-consistent clips, generating detailed captions, extracting subject categories, and deploying erasure techniques to construct source videos absent the subject. Target videos result from matching appropriate external subjects via multimodal embeddings, ensuring natural cross-pairings. SynthGen covers breadth by algorithmically constructing prompt–pair sets, using template-driven LLM prompting and generative models; consistency is enforced using VLM-based similarity scoring. SimInteract leverages physically accurate rendering (e.g., built in Houdini) to simulate complex subject–scene interactions with full control, providing detailed ground truth for challenging cases.

This ensemble creates comprehensive datasets containing {prompt, reference images, source video, target video}, supporting supervised training of video insertion models with rich variability across subject attributes, scene types, and dynamic interactions (Chen et al., 22 Sep 2025).

3. Role in Model Training and Mask-Free Insertion

By providing paired source and target videos for a reference subject and scene text prompt, InsertPipe enables direct application of end-to-end learning for the mask-free video insertion objective:

• The source video offers background and scene context without the target subject.
• Reference images encode subject appearance and identity.
• The target video is used as ground truth for supervising subject insertion (appearance, pose, dynamics, lighting) into the source context.

OmniInsert, the accompanying framework, is specifically constructed to exploit InsertPipe’s outputs. Its architecture leverages InsertPipe's paired data in a progressive, multi-stage training strategy:

• Initial subject–scene separation and focused subject representation learning (from text-to-video samples and subject images).
• Introduction of source videos for mask-free insertion training—requiring fusion of subject and video features.
• Refinement phases with high-fidelity and synthetic data to overcome identity drift, followed by preference-driven finetuning for perceptual harmonization.
• Pooling of diverse InsertPipe data enables balancing of subject-scene equilibrium and robust handling of varied interactions during training.

4. Data Quality Control and Diversity

InsertPipe employs several mechanisms to ensure diversity and quality:

• VLMs (e.g., GPT-4 or similar) generate and cross-check detailed captions, provide category assignments, and score generated samples for semantic consistency.
• CLIP and specialized face embedding models are used to prevent trivial copy–paste artifacts by enforcing visual and identity-level diversity between source and reference frames.
• For synthetic and rendered content, prompt-bucketization ensures all major subject and scene types are represented, and VLM scoring at sample selection time reinforces detailed attribute coverage.
• SimInteract’s renderings use motion priors and layout libraries to sample rare but physically plausible subject–scene interactions, augmenting otherwise hard-to-acquire real data in the supervision set (Chen et al., 22 Sep 2025).
• For both generated and rendered samples, subject erasure/replacement is performed using state-of-the-art video erasing and inpainting techniques to avoid biasing the model with mask boundaries.

The result is automatic large-scale generation of high-fidelity, diverse cross-pairs suitable for robust mask-free insertion training.

5. Comparative Impact and Integration with Benchmarks

InsertPipe directly addresses a gap in the field by supporting the first large-scale benchmark—InsertBench—specifically constructed for mask-free video insertion evaluation. InsertBench consists of 120 videos and controlled subject cross-pairs, each lasting about 5 seconds with 121 frames, covering domains such as indoor/outdoor settings, interaction-intense scenes, wearable cases, and animation.

• Evaluations using InsertBench demonstrate that models trained on InsertPipe-derived data (within the OmniInsert system) outperform closed-source commercial solutions in both objective (e.g., CLIP-I, DINO-I, FaceSim for subject consistency) and subjective (human rating on realism, consistency, and dynamics) metrics (Chen et al., 22 Sep 2025).
• The approach supports not only visually plausible insertions but also insertion harmonization—achieved via cross-modal prompt refinement (Context-Aware Rephraser) and preference-driven optimization (Insertion Preference Optimization).

6. Mathematical and Algorithmic Formulation

InsertPipe’s data curation process enables the definition and optimization of losses for mask-free video insertion using paired supervision. In training, OmniInsert minimizes a combination of global flow-matching and subject-focused loss terms, with explicit down-weighting of trivial background regions and up-weighting of subject features:

$$L_{SL} = \mathbb{E} \left[ \left\| M \cdot \left( (z_0 - \epsilon) - V_{\theta}(z_t, t, y) \right) \right\|^2 \right]$$

$L = \lambda_1 L_{FM} + \lambda_2 L_{SL}$

where $M$ is a spatial mask (from InsertPipe ground truth), $z_0$ and $\epsilon$ are latent variables in the diffusion process, and $V_\theta$ denotes the learned denoising function. InsertPipe provides explicit masks (by construction during subject removal/simulation); these facilitate the subject-focused loss without reliance on manual annotation or extra overhead.

7. Broader Implications and Potential Extensions

InsertPipe’s pipeline of automated cross-pair curation with a blend of real, synthetic, and physically simulated data presents a scalable strategy for overcoming data scarcity in any generative modeling scenario where paired examples are rare or unattainable. While designed for video insertion, its methodology of leveraging VLMs, generative synthesis, simulation, and programmatic matching may generalize to related problems in video editing, scene composition, and multimodal data augmentation.

A plausible implication is that further advances in generative modeling and video understanding will benefit from similar large-scale, synthetic, and auto-curated paired datasets, particularly in applications requiring photorealistic, coherent edits or insertions into unconstrained visual environments.

---

InsertPipe thus constitutes a crucial foundational layer enabling mask-free, high-fidelity video insertion at scale by synthesizing, refining, and quality-controlling the critical paired data necessary for modern conditional generative architectures (Chen et al., 22 Sep 2025).