FlowDrag: 3D-Aware Drag Image Editing

• FlowDrag is a 3D-aware drag-based image editing method that reconstructs a 3D mesh from an input image to enforce geometric consistency and prevent distortions.
• It constructs either a DepthMesh or DiffMesh from the image and applies a progressive SR-ARAP deformation to achieve smooth, coherent edits that maintain object structure.
• FlowDrag improves drag accuracy by coupling point supervision with explicit geometric structure, outperforming previous methods on VFD-Bench and DragBench benchmarks.

FlowDrag is a 3D-aware drag-based image editing method that addresses geometric inconsistency in point-driven diffusion editing by reconstructing a 3D mesh from an input image, deforming that mesh under drag constraints, and projecting the resulting mesh displacements into 2D vector flow fields that guide UNet denoising. It is designed for the setting in which drag-based editing allows precise object manipulation through point-based control, but prior systems optimize primarily for handle-to-target alignment in 2D feature space and can therefore move the dragged point correctly while distorting the rest of the object. FlowDrag targets that failure mode by coupling point supervision to explicit geometric structure and is reported to outperform existing drag-based editing methods on both VFD Bench and DragBench (Koo et al., 11 Jul 2025).

1. Problem formulation and conceptual scope

FlowDrag is defined around a specific weakness of prior drag-editing systems: they usually optimize only the user-specified handle-to-target correspondence in 2D feature space, so they can move the dragged point correctly while distorting the rest of the object. The paper characterizes this failure as “geometric inconsistency,” and notes that it is especially visible for rigid edits such as rotations, pose changes, or relocations, where nearby parts should move coherently with the handle. Existing motion-based methods named in this context include DragDiffusion, FreeDrag, DragNoise, and GoodDrag; these methods can match the point trajectory but often ignore the object’s broader shape, leading to broken structure, missing parts, or artifacts (Koo et al., 11 Jul 2025).

The method is organized into three stages: constructing a 3D mesh from the input image, deforming that mesh using a progressive SR-ARAP energy under drag constraints, and projecting the deformation into 2D vector flow fields that are injected into a diffusion UNet. This division is central to the method’s identity. The first stage supplies an explicit geometric prior, the second stage enforces rigidity-preserving deformation, and the third stage converts the 3D-aware deformation into a form that can guide denoising and handle-to-target alignment within a standard drag-editing diffusion pipeline (Koo et al., 11 Jul 2025).

A plausible implication is that FlowDrag should be understood not as a replacement for diffusion editing, but as a geometry-conditioned control layer over diffusion editing. The paper’s emphasis is not on changing the generative backbone itself, but on making drag supervision object-aware rather than point-only.

2. Mesh construction and geometric representation

FlowDrag begins by converting the image into a mesh and presents two mesh-construction options. The first is DepthMesh, built from a monocular depth map, and the second is DiffMesh, built from an image-to-3D diffusion model. For the depth-based route, the method uses Marigold to estimate a depth map $D$ from a single image. Each pixel $(x,y)$ with depth value $D(x,y)$ is mapped to a 3D vertex

$v_i = (x_i, y_i, z_i),$

where $z_i$ comes from the depth value. Neighboring pixels are then connected into triangles if their depth values are sufficiently similar, and connections are removed if the depth difference is too large. The paper states that this artifact-reduction step helps separate foreground from background and suppress discontinuities, yielding a mesh $M=(V,F)$ with vertices $V$ and triangular faces $F$ (Koo et al., 11 Jul 2025).

The alternative DiffMesh is motivated by the limitation that depth maps can miss occluded or unseen geometry. The paper states that DiffMesh tends to give a more complete shape estimate, especially for complex objects, while DepthMesh is simpler and faster. This produces a methodological split within FlowDrag itself: one route prioritizes simplicity and speed, and the other prioritizes completeness of the inferred shape (Koo et al., 11 Jul 2025).

After mesh construction, the user’s 2D drag annotation is mapped onto the 3D mesh. The 2D handle point becomes a handle vertex $v_h$, the 2D target point becomes a target vertex $v_t$, the editable region defines movable vertices, and all vertices outside the mask become constraints. This translation from 2D annotation to 3D constrained deformation is the key step by which FlowDrag turns local drag input into a global geometric optimization problem (Koo et al., 11 Jul 2025).

3. Progressive SR-ARAP deformation

The deformation stage is designed around the claim that directly moving the handle vertex to the target can cause abrupt stretching, local foldovers, or bad local minima when the drag distance is large. FlowDrag therefore uses a progressive update rule rather than a one-step relocation. The handle is moved gradually over $(x,y)$0 iterations according to

$(x,y)$1

The paper states that this reduces sudden geometric shocks and makes the deformation smoother (Koo et al., 11 Jul 2025).

The underlying deformation model is Smoothed Rotation As-Rigid-As-Possible (SR-ARAP). In the paper’s formulation, ARAP is augmented by a smoothing term over adjacent rotations,

$(x,y)$2

where $(x,y)$3 denotes neighboring vertices, $(x,y)$4 is the local rotation at vertex $(x,y)$5, and the scale factor is set to $(x,y)$6 to prevent scaling. The added term discourages abrupt rotation differences and is intended to yield smoother, more coherent deformation. The paper’s interpretation is explicit: the handle is not moved in isolation, and the surrounding mesh is deformed in a rigid-consistent way so correlated parts move together (Koo et al., 11 Jul 2025).

FlowDrag further adds inter-step smoothness to the progressive variant,

$(x,y)$7

This penalizes large jumps between consecutive deformation steps. In ablation, the paper reports that $(x,y)$8 works best, giving the best balance of rigidity and smoothness, and also reports highest mean edge length ratio and lowest mean ARAP error at that setting. The stated significance of this term is that it preserves shape under large drags by stabilizing the progressive optimization (Koo et al., 11 Jul 2025).

4. Projection to 2D flow and diffusion guidance

Once the final deformed mesh is obtained, FlowDrag projects both the original and deformed meshes into the image plane, $(x,y)$9 and $D(x,y)$0. The projected per-vertex 2D displacement defines the vector flow field

$D(x,y)$1

where $D(x,y)$2 and $D(x,y)$3 are the 2D coordinates of the same vertex before and after deformation. The paper treats this flow field as the bridge between 3D geometry and 2D drag editing: instead of supervising only the user’s handle, the method obtains a structured set of displacement vectors covering the object region (Koo et al., 11 Jul 2025).

The full vector field is not used directly. FlowDrag samples an $D(x,y)$4 grid of candidate vectors inside the edit mask and then selects a smaller subset $D(x,y)$5 of about 5–30 vectors. The paper compares magnitude-based sampling and uniform sub-sampling and states that magnitude-based sampling performs better, giving lower mean distance and better fidelity. It also reports that around 10 sampled vectors gives the best overall performance. This part of the method establishes that the geometry prior is not injected as a dense optical-flow substitute, but as a compact set of informative deformation cues (Koo et al., 11 Jul 2025).

These vectors are used in motion supervision during latent optimization. The latent is updated by gradient descent,

$D(x,y)$6

so that denoising follows the mesh-induced deformation. FlowDrag also uses the 2D projection of the deformed mesh as a layout feature. This projection is inverted with DDIM inversion, and attention features from the corresponding latent are injected into the main edit branch of the UNet during early or intermediate denoising steps. The paper’s stated intuition is that early denoising controls global layout, while later steps refine details, so early/intermediate feature injection transfers broad structural information without over-constraining fine appearance details (Koo et al., 11 Jul 2025).

5. Benchmarks, implementation, and empirical results

A major contribution accompanying FlowDrag is VFD-Bench (VidFrameDrag), introduced because existing drag-editing benchmarks provide no ground truth and therefore make it difficult to assess how accurately edits match the intended transformation. VFD-Bench is built from consecutive video frames selected from DAVIS, LOVEU-TGVE, TVR, and copyright-free Pexels clips. The dataset contains 250 input–GT pairs: 140 animal images, 65 human images, and 45 object images. For each pair, 1–5 drags are annotated by comparing the input frame and the later ground-truth frame. FlowDrag is also evaluated on DragBench, described here as 205 images and 349 drag pairs with no ground truth. The implementation details reported in the paper include Stable Diffusion 1.5, $D(x,y)$7 resolution, VQ-VAE, LoRA fine-tuning with rank 16 for 200 steps, DDIM inversion up to step 38 out of 50, layout injection at timestep 30, and a preference for DiffMesh unless it has strong artifacts, in which case DepthMesh is used (Koo et al., 11 Jul 2025).

Benchmark	FlowDrag	Selected comparison
DragBench	MD $D(x,y)$8	GoodDrag $D(x,y)$9; DiffEditor $v_i = (x_i, y_i, z_i),$0; DragDiffusion $v_i = (x_i, y_i, z_i),$1
VFD-Bench	PSNR $v_i = (x_i, y_i, z_i),$2; 1-LPIPS $v_i = (x_i, y_i, z_i),$3; MD $v_i = (x_i, y_i, z_i),$4	GoodDrag: $v_i = (x_i, y_i, z_i),$5, $v_i = (x_i, y_i, z_i),$6, $v_i = (x_i, y_i, z_i),$7; DragDiffusion MD $v_i = (x_i, y_i, z_i),$8

On DragBench, the paper states that FlowDrag achieves the best Mean Distance, with $v_i = (x_i, y_i, z_i),$9, slightly better than GoodDrag at $z_i$0. On VFD-Bench, FlowDrag is reported as best on all three metrics: PSNR $z_i$1, 1-LPIPS $z_i$2, and MD $z_i$3. The paper notes that DiffEditor attains the highest 1-LPIPS on DragBench because it often changes the image least, but argues that this does not imply successful dragging. Qualitatively, the reported advantage is that FlowDrag moves objects coherently while preserving dependent structures such as hats, arms, torches, or body contours; the cited example is rotating a woman’s face while also moving the hat and hand in a natural way. A user study on 50 images with 25 volunteers is also reported to prefer FlowDrag for both drag accuracy and image quality (Koo et al., 11 Jul 2025).

6. Research position, limitations, and adjacent directions

Within the drag-based editing literature summarized by the paper, FlowDrag occupies the position of a geometry-aware alternative to methods that supervise only point trajectories. Its distinctive move is to replace handle-only motion supervision with mesh-guided deformation vectors and to combine those vectors with layout feature injection inside the diffusion UNet. This suggests a hybridization of image-based diffusion editing with explicit geometric regularization: the edit remains diffusion-based, but the displacement prior is produced by a separate 3D deformation process rather than by 2D feature matching alone (Koo et al., 11 Jul 2025).

The paper is explicit about several limitations. FlowDrag is best for moderate, rigid edits and is not ideal for very large drag distances. It struggles with content creation or removal and with edits requiring major topological change. Because the 3D mesh is ultimately projected into 2D, some 3D structural detail is lost, and the method still relies on Stable Diffusion, which is not inherently 3D-aware. Efficiency is also mesh-dependent: the full depth-based mesh is reported as slower, around 5 seconds per sample on average, while reducing faces speeds deformation at the cost of geometric detail. Robustness tests further show that the method is fairly robust to depth-mesh reduction ratios from 1 down to 0.001, and that DiffMesh is robust across a range of diffusion sampling steps, but very low sampling or overly simplified meshes degrade quality (Koo et al., 11 Jul 2025).

A plausible implication is that FlowDrag defines one branch of a broader evolution from point-only drag editing toward structure-aware interactive generation. In that broader space, FlowDrag addresses still-image editing through mesh-guided deformation, whereas DragStream addresses streaming drag-oriented interactive video manipulation in autoregressive video diffusion by targeting latent distribution drift and context interference rather than explicit mesh geometry (Zhou et al., 3 Oct 2025).