VOID: Video Object and Interaction Deletion

Abstract: Existing video object removal methods excel at inpainting content "behind" the object and correcting appearance-level artifacts such as shadows and reflections. However, when the removed object has more significant interactions, such as collisions with other objects, current models fail to correct them and produce implausible results. We present VOID, a video object removal framework designed to perform physically-plausible inpainting in these complex scenarios. To train the model, we generate a new paired dataset of counterfactual object removals using Kubric and HUMOTO, where removing an object requires altering downstream physical interactions. During inference, a vision-LLM identifies regions of the scene affected by the removed object. These regions are then used to guide a video diffusion model that generates physically consistent counterfactual outcomes. Experiments on both synthetic and real data show that our approach better preserves consistent scene dynamics after object removal compared to prior video object removal methods. We hope this framework sheds light on how to make video editing models better simulators of the world through high-level causal reasoning.

• The paper introduces a method to generate realistic counterfactual videos by removing objects and recalculating dynamic interactions.
• It employs a two-stage diffusion pipeline with quadmask conditioning and VLM-guided masking to capture both direct and indirect effects.
• Empirical evaluations show superior performance in physical plausibility and interaction consistency compared to state-of-the-art inpainting methods.

VOID: Video Object and Interaction Deletion – An Expert Analysis

Motivation and Problem Definition

The challenge addressed by "VOID: Video Object and Interaction Deletion" (2604.02296) is the synthesis of physically plausible counterfactual videos following object removal, transcending conventional inpainting approaches. Standard video object removal methods typically excel at reconstructing occluded backgrounds and correcting photometric artifacts, yet they fail dramatically when the object participates in dynamic interactions—such as supporting, colliding, or manipulating other entities in the scene. The paper argues that a robust video editing system must accommodate these high-level, causal dependencies, effectively "rewriting" the physical narrative of the scene post-removal to maintain realism.

Data Construction and Counterfactual Supervision

To train models capable of such causal reasoning, the paper curates a novel paired dataset leveraging Kubric for simulated rigid-body physics and HUMOTO for articulated human-object interactions. Each video pair consists of a source clip containing the target object and its interactions, and a counterfactual resimulation with the object excised, forcing physical outcomes to diverge (e.g., domino chains that halt, objects entering free-fall, or supported items collapsing).

Figure 1: Counterfactual supervision examples illustrating downstream motion changes in Kubric and natural gravity transitions in HUMOTO upon object removal.

This supervision facilitates learning not only visual inpainting but also the underlying evolution of object states and their causal impact—a prerequisite for realistic counterfactual generation.

Methodology

VOID is architected as a two-stage video diffusion pipeline, guided by interaction-aware masking strategies and powered by VLM reasoning:

1. Quadmask Conditioning: The method refines traditional trimasks, introducing a fourth "dark grey" category to resolve ambiguities in regions with overlapping object and effect domains. Mask assignment is dynamically expanded using VLM inference, enabling pixel-level identification of indirect downstream effects.
2. VLM-Guided Interaction Masking: At inference, the user specifies a sparse binary mask for object removal, which the VLM expands into an interaction-aware quadmask. This is achieved via iterative querying and spatial grid overlays, extracting affected object regions and counterfactual positions.
3. Diffusion Backbone & Trajectory Synthesis: The backbone is initialized from Generative Omnimatte, then fine-tuned with quadmask supervision for dynamic trajectory rewriting. The first diffusion pass generates plausible hypotheses for scene evolution. Where structural deformation arises (due to new trajectories not present in the source video), the second pass uses flow-warped noise—as in "Go-with-the-Flow"—to stabilize object rigidity and mitigate drift.

   Figure 2: VOID pipeline overview: user input, VLM mask expansion, counterfactual generation, and deformation stabilization.

Empirical Evaluation

VOID is evaluated on both synthetic benchmarks (with ground truth) and a real-world interaction dataset. Metrics span pixel-level measures (PSNR, LPIPS), feature-based perceptual similarity (DreamSim, DINOv2), temporal fidelity (FVD), and VLM-judge assessments.

Human and VLM Judge Studies

A human preference study shows VOID is preferred in 64.8% of cases, far exceeding both state-of-the-art inpainting and commercial text-guided editors such as Runway (18.4%). VLM-judge evaluations (Gemini 3 Pro, GPT-5.2, Qwen 3.5-32B) verify superior "Interaction Physics" and overall plausibility, with consistent ranking alignment between human and automated judges.

Figure 3: Qualitative comparisons on real-world edits demonstrating VOID’s superior scene structure and plausible motion relative to baseline models.

Generalization and Unseen Effects

VOID demonstrates robust extrapolation to domain shifts and unseen interaction types. Notably, it infers balloon ascent after removal of the holder, and disables blender contents upon removal of the operator, despite lacking explicit supervision for these phenomena.

Figure 4: VOID generalizes to diverse object interactions, correctly handling causal consequences such as falling, trajectory prevention, and photometric effect correction.

Ablation Studies

Ablations affirm the benefit of dual synthetic data sources (Kubric+HUMOTO) and quadmask detail, validating the necessity of interaction-aware mask guidance and dataset diversity. The second-pass refinement step further enhances interaction realism and minimizes deformation.

Figure 5: Mask interface illustrates sparse user selection and VLM mask expansion.

Figure 6: Interface employed in human preference evaluation studies.

Limitations

Despite strong results, VOID is subject to domain gaps (e.g., unusual camera angles), constrained video duration, and resolution fidelity. Model performance remains sensitive to VLM mask generation quality; Gemini 3-Pro yields optimal mask guidance. The reliance on synthetic supervision implies further benefit from real-world paired data or advanced simulators.

Theoretical and Practical Implications

VOID substantively raises the bar for video editing and world modeling, demonstrating that causal counterfactual reasoning can be successfully integrated with large-scale generative models. The framework leverages world knowledge from VLMs to structure spatial guidance, informing physical state transitions—a crucial innovation toward more accurate, interaction-aware generative editing. This methodology portends future developments in compositional video synthesis, autonomous film visual effects, simulation-based editing, and the broader intersection of generative models and causal reasoning.

Figure 7: Standard similarity metrics may favor visually implausible outputs, underscoring the necessity of interaction-aware evaluation for physically plausible edits.

Conclusion

"VOID: Video Object and Interaction Deletion" introduces a principled approach for interaction-aware object removal, embedding high-level causal reasoning in generative video editing. By orchestrating physics-based counterfactual supervision, VLM mask expansion, and a two-pass diffusion process, VOID achieves superior plausibility and generalization relative to prior methods. The paper provides a foundation for future research aiming to conjoin physical simulation and generative editing, underscoring the value of world modeling in video synthesis.