3DObjectEditor: Sequential 3D-Aware Editing

• 3DObjectEditor is a hybrid dataset that combines real and synthetic domains to enable multi-round, 3D-aware object editing with sequence-level supervision.
• The dataset supports history-aware transformations by preserving object identity, background effects, and lighting consistency during translation, scaling, and rotation edits.
• Empirical evaluations demonstrate improved PSNR, SSIM, and user preference metrics, establishing its effectiveness for training advanced 3D editing models.

3DObjectEditor is a hybrid, two-domain dataset of simulated multi-round edits introduced with the Free-Form Scene Editor (FFSE), a 3D-aware autoregressive framework for object manipulation on real-world images (Shuai et al., 17 Nov 2025). It is designed for editing sessions that unfold as sequences of transformations rather than isolated before/after pairs. In the published configuration, it contains approximately $86\,\text{K}$ sequences with average round-length $32$, split between a realistic domain of approximately $40\,\text{K}$ sequences and a synthetic domain of approximately $46\,\text{K}$ sequences (Shuai et al., 17 Nov 2025). Its central role is to provide training data for models that must preserve object identity, background effects, and cross-round consistency while applying translation, scaling, and rotation edits.

1. Conceptual role within 3D-aware editing

3DObjectEditor was proposed in response to a limitation of many image-editing systems: strong semantic editing performance in 2D does not automatically yield 3D-aware object manipulation. FFSE formulates editing as a sequence of latent scene states $s \in S$, projected to images $x \in X$ by a renderer $f_m : S \to X$, with edits represented as 3D transformations $o \in O = \{o^T,o^S,o^X,o^Y,o^Z\}$, corresponding to translation, uniform scaling, and rotation about the $x,y,z$ axes (Shuai et al., 17 Nov 2025). Under this formulation, supervision must reflect both geometric change and sequential dependence.

The dataset therefore differs in emphasis from earlier language-guided editing corpora that focus on single-step editing. OBJECT, for example, consists of $400\text{K}$ editing examples created from procedurally generated 3D scenes, with exactly four atomic operations—translation, rotation, insertion, and removal—and each example contains an input image, a language instruction, and an edited image (Michel et al., 2023). By contrast, 3DObjectEditor is organized around multi-round edit sequences under dynamic conditions (Shuai et al., 17 Nov 2025). This suggests a shift from isolated edit synthesis toward persistent scene-state modeling.

2. Dataset composition and domain structure

3DObjectEditor is explicitly described as a hybrid, two-domain dataset. One domain emphasizes realism through compositing and large-scale sequence generation; the other emphasizes physically grounded rendering, broader 3D transformation coverage, and lighting effects (Shuai et al., 17 Nov 2025).

Domain	Scale	Core construction
$32$0	$32$1 sequences of length $32$2	MS-COCO cutouts, random backgrounds, 3–5 objects, painter’s algorithm
$32$3	$32$4 sequences of length $32$5	HDR panoramas, $32$6 3D meshes, Blender Cycles rendering
Total	$32$7 sequences	Hybrid supervision across real and synthetic domains

In the realistic domain $32$8, assets are RGBA “cutouts” from MS-COCO obtained via MULAN. Scenes are created from a random background plus $32$9–$40\,\text{K}$0 objects, each assigned a random $40\,\text{K}$1 and depth $40\,\text{K}$2. At each step, one object is selected and transformed either by translation $40\,\text{K}$3, implemented as pixel shift and depth offset, or by scaling $40\,\text{K}$4, with factor $40\,\text{K}$5. Rendering is performed by painter’s algorithm, with objects sorted by depth (Shuai et al., 17 Nov 2025).

In the synthetic domain $40\,\text{K}$6, assets are HDR panoramas from PolyHaven/Sketchfab and more than $40\,\text{K}$7 3D meshes from Objaverse-LVIS/XL. Scenes use random lighting, specifically directional and point lights, with objects placed on a ground plane. At each step, a random object is transformed by one of $40\,\text{K}$8, with resampling until the transformation is collision-free. These scenes are rendered with Blender Cycles to capture physically accurate shadows, reflections, and occlusions (Shuai et al., 17 Nov 2025).

The reported synthetic-domain statistics further specify approximately $40\,\text{K}$9 “everyday” object classes, including furniture, vehicles, and tools, and approximately $46\,\text{K}$0 scene types, including living rooms, offices, outdoor, and industrial backgrounds (Shuai et al., 17 Nov 2025).

3. Sequential edit representation

The defining structural property of 3DObjectEditor is that supervision is sequential. FFSE models the $46\,\text{K}$1-th edited image $46\,\text{K}$2 conditioned on the entire edit history

$46\,\text{K}$3

and training and sampling proceed over the noisy trajectory $46\,\text{K}$4 (Shuai et al., 17 Nov 2025). The dataset is therefore not merely a collection of edited images; it is a corpus of ordered editing sessions.

Each edit $46\,\text{K}$5 is decomposed into two parts. The first is a source-region descriptor $46\,\text{K}$6, defined by a centroid and a bounding box. The second is a transformation vector $46\,\text{K}$7. FFSE Fourier-embeds these quantities, maps them through MLPs, and injects the resulting sequence into the UNet through an “operation self-attention” module (Shuai et al., 17 Nov 2025). This indicates that 3DObjectEditor provides not only image-level supervision but also a structured representation of object identity and edit parameters across time.

A practical consequence is that the dataset supports history-aware editing rather than stateless instruction following. In FFSE’s autoregressive loop, the system initializes a frame buffer $46\,\text{K}$8 and an operation buffer $46\,\text{K}$9, appends a new operation $s \in S$0 at each step, constructs history $s \in S$1 from the last $s \in S$2 pairs, and generates only the new frame $s \in S$3 (Shuai et al., 17 Nov 2025). This makes 3DObjectEditor particularly relevant to models that must withstand drift, temporary occlusion, and repeated manipulation.

4. Rendering regimes and physical cues

The two domains serve different supervisory functions. The realistic domain provides large-scale sequences assembled from real-image cutouts and backgrounds. The synthetic domain provides scenes rendered with Blender Cycles, specifically to encode shadow, reflection, and occlusion behavior under repeated object motion (Shuai et al., 17 Nov 2025).

This division is directly reflected in FFSE’s training strategy. Stage 1 uses the union $s \in S$4 to optimize the denoising model and both Domain-LoRA branches, while Stage 2 uses only $s \in S$5, freezing LoRA and fine-tuning the base network for better lighting and shadow realism (Shuai et al., 17 Nov 2025). The paper states that photorealism is enforced by fine-tuning on $s \in S$6, whose Blender-rendered scenes contain accurate shadow and reflection cues.

Ablation results clarify why the hybrid design matters. Training on real only produces poor shadows, whereas training on synthetic only yields an over-saturated style; removing dual LoRA causes failures either in edit obedience or artifact suppression; dropping the Stage 2 fine-tune leads to unrealistic lighting (Shuai et al., 17 Nov 2025). These observations locate 3DObjectEditor at the intersection of two supervision regimes: appearance statistics drawn from realistic compositing and physically consistent lighting behavior drawn from synthetic rendering.

5. Use in FFSE and learned consistency mechanisms

3DObjectEditor is the training substrate for FFSE’s 3D-aware autoregressive diffusion framework. FFSE augments a denoising backbone with four principal conditioning mechanisms: an operation encoder, a frame encoder, context self-attention (CSA), and Domain-LoRA (Shuai et al., 17 Nov 2025). The dataset’s sequential structure is what makes these modules meaningful.

The frame encoder processes previous renderings $s \in S$7, optionally with a binary target-region mask, and injects spatial context into the down-sampling blocks. CSA re-aligns features of the same object between rounds $s \in S$8 and $s \in S$9, with attention masked to lie only within object regions. The paper states that the frame encoder plus CSA explicitly passes the previous frame’s object appearance into the current denoiser, preventing drift over repeated edits (Shuai et al., 17 Nov 2025). Operation self-attention, in turn, ensures that the model “understands” which object is to be moved, rotated, or scaled, and by how much.

Notably, FFSE is trained with the standard denoising-diffusion reconstruction loss and adds no explicit adversarial or 3D-specific regularizers. The reported explanation is that 3D coherence emerges from four factors: the simulated edit data, the autoregressive history encoding, context self-attention, and fine-tuning on physically rendered scenes (Shuai et al., 17 Nov 2025). In this formulation, 3DObjectEditor is not ancillary supervision; it is the primary source from which cross-round and cross-effect consistency is learned.

6. Empirical outcomes, scope, and relation to adjacent datasets

When trained on 3DObjectEditor, FFSE reports strong quantitative performance in both single-round and multi-round editing. In Table 1, FFSE achieves $x \in X$0 PSNR, $x \in X$1 SSIM$x \in X$2, $x \in X$3 DINO-Score, and $x \in X$4 CLIP-Score in the single-round setting, and $x \in X$5, $x \in X$6, $x \in X$7, and $x \in X$8, respectively, in the multi-round setting; the best multi-round baseline reports $x \in X$9, $f_m : S \to X$0, $f_m : S \to X$1, and $f_m : S \to X$2 (Shuai et al., 17 Nov 2025). In a user study with $f_m : S \to X$3 raters, normalized preference scores are $f_m : S \to X$4 for image quality, $f_m : S \to X$5 for object effects, $f_m : S \to X$6 for background effects, and $f_m : S \to X$7 for scene consistency, compared with next-best scores of $f_m : S \to X$8, $f_m : S \to X$9, $o \in O = \{o^T,o^S,o^X,o^Y,o^Z\}$0, and $o \in O = \{o^T,o^S,o^X,o^Y,o^Z\}$1 (Shuai et al., 17 Nov 2025).

The qualitative claims are equally tied to the dataset’s design. For single-round editing, FFSE is reported to handle large rotations, including out-of-plane rotations, without geometry artifacts and to plausibly re-cast shadows. For multi-round editing, the paper states that only FFSE correctly recovers objects that have been briefly occluded, while other methods degrade over repeated steps and lose track of hidden elements (Shuai et al., 17 Nov 2025). These behaviors are consistent with supervision from long edit sequences containing occlusion, reappearance, and relighting events.

In relation to adjacent resources, 3DObjectEditor occupies a distinct position. OBJECT provides a large synthetic corpus for language-guided 3D-aware image editing with exactly four atomic operations and supports both single-task and multi-task models (Michel et al., 2023). 3DObjectEditor instead focuses on multi-round object manipulation with sequential history and hybrid-domain supervision (Shuai et al., 17 Nov 2025). Its operation set, as described in the dataset construction, covers translation and scaling in the realistic domain and translation, scaling, and axis-aligned rotations in the synthetic domain; insertion and removal are not part of the reported construction. This distinguishes it from OBJECT’s translation, rotation, insertion, and removal formulation (Michel et al., 2023).

A plausible implication is that 3DObjectEditor marks a transition from isolated 3D-aware edit examples to datasets explicitly engineered for temporally extended manipulation. Within the evidence reported for FFSE, its significance lies less in raw scale than in the combination of hybrid rendering, parameterized 3D transformations, and sequence-level supervision needed to train models that behave “like in a 3D engine” across multiple rounds of editing (Shuai et al., 17 Nov 2025).