HOI-Edit-44K: Identity-Preserving HOI Editing Dataset

• The paper introduces HOI-Edit-44K with 44,117 paired examples to enable identity-preserving changes in human–object interactions.
• It employs an automated pipeline using HOI detectors and DINOv2 features to ensure both interaction correctness and subject/object identity retention.
• The dataset supports layout-guided and layout-free HOI editing in the unified OneHOI framework, bridging the gap between HOI detection and image editing.

Searching arXiv for the specific dataset and closely related HOI editing work to ground the article in current papers. HOI-Edit-44K is a large-scale, paired dataset for human–object interaction (HOI) editing, introduced with OneHOI as a resource for learning identity-preserving changes in interaction rather than only recognizing or generating static HOI labels. It consists of 44,117 identity-preserving HOI editing pairs, where each pair couples a source image with a target interaction triplet, an edited image realizing that target interaction, and a corresponding HOI layout for the edited image (Hoe et al., 15 Apr 2026). Its stated purpose is to address the scarcity of paired data for HOI editing, especially supervision that specifies how an interaction should change while preserving the identities of the subject and object. A central source of confusion in the literature is nomenclature: HOI-Edit-44K is not the same resource as HOI-Edit, a separate 705-sample benchmark for instruction-based image HOI editing introduced in a different paper (Gao et al., 17 Jun 2026).

1. Definition and intended problem scope

HOI-Edit-44K was created for HOI editing, not for conventional HOI detection alone. The target setting is an edit in which a model must change how a person interacts with an object without changing who the person is or what the object is. The dataset is described as a core contribution of OneHOI: “A new large-scale paired dataset, HOI-Edit-44K, addressing the scarcity of paired data, with 44K identity-preserving examples, for training of robust HOI editing” (Hoe et al., 15 Apr 2026).

The motivation is defined against three limitations of prior resources. First, HOI recognition and detection datasets provide single images with HOI labels and boxes, but no explicit before/after editing pairs. Second, HOI generation settings condition on triplets and layouts, but again do not provide paired source and edited images. Third, general image editing datasets rarely include nuanced changes in human–object interaction while preserving identity. HOI-Edit-44K is intended to fill this gap by supplying paired supervision for interaction change, identity preservation, and layout-aware grounding within a single dataset (Hoe et al., 15 Apr 2026).

The tasks it supports are stated explicitly. It is used to supervise layout-free HOI editing, layout-guided HOI editing, and the broader unified HOI generation+editing formulation used by OneHOI. The paper further identifies it as particularly important for single-HOI editing, identity-preserving edits, and training the model’s structured interaction representation (Hoe et al., 15 Apr 2026).

2. Data composition and annotation schema

The supplementary material states that “The final dataset consists of 44,117 high-quality, paired HOI editing examples” (Hoe et al., 15 Apr 2026). Each sample contains four elements:

1. a source image containing at least one HOI,
2. a target interaction triplet,
3. an edited image realizing the target interaction,
4. the corresponding HOI layout for the edited image.

The paper describes the core HOI representation as the classical triplet

$$\langle \text{subject}, \text{action}, \text{object} \rangle,$$

while the main method denotes a set of instances as

$\{\langle s,o,a\rangle_n\}_{n=1}^N.$

For HOI-Edit-44K specifically, the supplement states that each sample includes the source image, the target interaction triplet, the edited image, and the edited-image HOI layout (Hoe et al., 15 Apr 2026).

Dataset-wide category statistics are also given. HOI-Edit-44K contains 79 unique object categories, 92 unique target actions, and 372 unique HOI triplets. The treemap visualizations described in the supplement indicate a long-tailed distribution, although the paper does not enumerate the frequency of each category in text (Hoe et al., 15 Apr 2026).

A structurally important property is that HOI-Edit-44K is a single-HOI editing dataset. The paper states that the authors “synthesise potential single-HOI edits,” and that a pair is retained if it passes the verification pipeline. This means that each sample focuses on one target interaction change per pair, even if the underlying image may contain multiple humans or objects (Hoe et al., 15 Apr 2026).

The paper does not specify train/validation/test splits for HOI-Edit-44K. Instead, it is described primarily as a training dataset. Evaluation in the OneHOI paper is conducted on IEBench and MultiHOIEdit, rather than on a held-out subset of HOI-Edit-44K itself (Hoe et al., 15 Apr 2026).

3. Construction and automatic verification pipeline

The collection pipeline combines synthetic and real-image sources, then filters candidate edits with automated HOI and identity checks. Two types of source images are used. One stream consists of Flux.1 generations that realise a verified source interaction. The other stream consists of HICO-DET images, adding real photographs to the source pool (Hoe et al., 15 Apr 2026).

For each source image, the authors generate candidate edits using Flux.1 Kontext and InteractEdit. These candidates are intended to realize a target HOI triplet given the source image, but only a small fraction are retained after verification (Hoe et al., 15 Apr 2026).

The first automated check is HOI correctness. The edited image is processed by PViC, described in the paper as a state-of-the-art HOI detector. A candidate pair is retained only if the predicted HOI matches the target HOI triplet. When this match succeeds, the detected subject/object layout is recorded and becomes the layout annotation stored with the pair (Hoe et al., 15 Apr 2026).

The second automated check is identity preservation. The pipeline extracts DINOv2 features from subject and object crops in both the source and edited images, computes cosine similarity for both entities, and requires both similarities $\ge 0.75$. Only pairs meeting this threshold for both subject and object are kept (Hoe et al., 15 Apr 2026).

The rejection rate is high: the paper reports that around 90% of initial candidate pairs are rejected. This is presented as evidence that constructing high-precision paired HOI editing data is difficult and that the final dataset is strongly filtered for both interaction correctness and identity preservation (Hoe et al., 15 Apr 2026).

4. Spatial layout, structured representation, and training use

Within OneHOI, HOI-Edit-44K is not treated as unstructured text-image editing data. Its triplets and layouts are converted into a structured conditioning interface for the Relational Diffusion Transformer (R-DiT). Let $\mathcal{S}_n$, $\mathcal{A}_n$, and $\mathcal{O}_n$ denote the T5-encoded token sequences for the subject, action, and object of instance $n$. The union of all HOI tokens is written as

$$\mathcal{H} = \bigcup_{n=1}^N \{\mathcal{S}_n, \mathcal{A}_n, \mathcal{O}_n\}.$$

For each role $r \in \{s, a, o\}$ and instance $n$, a T5 vector is augmented with a role embedding, an instance embedding, and a box/layout embedding derived from Fourier features of the corresponding bounding box (Hoe et al., 15 Apr 2026).

HOI-Edit-44K provides subject and object boxes for the edited image. These are denoted $\{\langle s,o,a\rangle_n\}_{n=1}^N.$0 and $\{\langle s,o,a\rangle_n\}_{n=1}^N.$1, and they are used to define role-specific regions $\{\langle s,o,a\rangle_n\}_{n=1}^N.$2 and $\{\langle s,o,a\rangle_n\}_{n=1}^N.$3. The paper defines the action region as

$\{\langle s,o,a\rangle_n\}_{n=1}^N.$4

This union region is reported to better match the attention footprint of action tokens than the “between” region used in prior work (Hoe et al., 15 Apr 2026).

These layouts drive Structured HOI Attention. For a query HOI token $\{\langle s,o,a\rangle_n\}_{n=1}^N.$5 and image token $\{\langle s,o,a\rangle_n\}_{n=1}^N.$6, the mask is defined so that subject tokens attend to subject regions, object tokens attend to object regions, and action tokens attend to the union region: $\{\langle s,o,a\rangle_n\}_{n=1}^N.$7 The intended effect is to make the action token mediate relation change, while subject and object tokens preserve role-specific identity and appearance (Hoe et al., 15 Apr 2026).

The dataset is also central to the editing branch of OneHOI’s unified training regime. In editing, the model receives a source image latent, noise latent, target HOI triplet, and target layout boxes, and learns the edited image with a flow-matching objective. During training, modality dropout is applied: layout is dropped with probability $\{\langle s,o,a\rangle_n\}_{n=1}^N.$8, HOI labels with $\{\langle s,o,a\rangle_n\}_{n=1}^N.$9, and the global text prompt with $\ge 0.75$0, while at least one modality is always kept. Because HOI-Edit-44K provides both triplets and layouts, it can support the full-conditioned and layout-dropped variants needed for unified editing and generation (Hoe et al., 15 Apr 2026).

A further technical boundary is explicit: HOI-Edit-44K itself provides bounding boxes, not arbitrary segmentation masks. OneHOI supports arbitrary-shape masks through SA-1B, using the minimum enclosing bounding box for Fourier embedding. That capability is architectural and multi-dataset rather than a direct property of HOI-Edit-44K (Hoe et al., 15 Apr 2026).

5. Relation to adjacent resources and common misidentifications

The most common misconception is to conflate HOI-Edit-44K with HOI-Edit. They are distinct resources introduced in different papers, for different purposes.

Resource	Primary purpose	Scale
HOI-Edit-44K	Paired dataset for training identity-preserving HOI editing	44,117 paired HOI editing examples
HOI-Edit	Benchmark for instruction-based image HOI editing with three cognitive levels	705 samples

The HOI-Edit benchmark is organized into three progressive cognitive levels and contains 357 L1 samples, 202 L2 samples, and 146 L3 samples, for a total of 705 samples. The paper introducing HOI-Edit states explicitly that it “consistently uses the name HOI-Edit, not ‘HOI-Edit-44K’,” and that the term “HOI-Edit-44K” does not appear in its text, figures, or tables (Gao et al., 17 Jun 2026). By contrast, OneHOI defines HOI-Edit-44K as a 44,117-pair training dataset (Hoe et al., 15 Apr 2026).

HOI-Edit-44K also differs from HOI detection resources. Detection-oriented work such as HOI Transformer formulates HOI detection as set prediction of quintuples

$\ge 0.75$1

predicted directly from an image, with no before/after editing supervision (Zou et al., 2021). More recent DETR-based detectors such as DQEN refine query design for HOI localization and classification, again operating in the detection regime rather than paired editing (Li et al., 26 Aug 2025). HOI-Edit-44K is therefore best understood not as a detection benchmark, but as paired supervision for changing interactions while preserving identity.

6. Limitations, biases, and significance

Several limitations are either stated directly or follow immediately from the collection protocol. First, the dataset has a low yield: after candidate generation and filtering, only 44,117 pairs remain, with around 90% of initial candidates rejected. This indicates that automated HOI editing pair construction is difficult even when powerful edit generators are available (Hoe et al., 15 Apr 2026).

Second, there is potential teacher-model bias. Edited images are produced using Flux.1 Kontext and InteractEdit, so some bias toward the behaviors of these systems is possible. The paper presents two mitigating steps: mixing in real HICO-DET images as sources and using strict HOI and identity thresholds during filtering (Hoe et al., 15 Apr 2026).

Third, HOI-Edit-44K is primarily a single-HOI dataset. Multi-HOI editing is addressed separately through MultiHOIEdit, and the paper notes that multi-HOI editing is not explicitly present in HOI-Edit-44K itself (Hoe et al., 15 Apr 2026). Fourth, although the dataset is diverse, its coverage remains bounded at 79 objects, 92 actions, and 372 triplets, which implies selective rather than exhaustive ontology coverage (Hoe et al., 15 Apr 2026).

Its significance within OneHOI is stated directly: “This resource was critical for jointly training our unified model, providing the necessary supervision for robust, identity-preserving HOI editing” (Hoe et al., 15 Apr 2026). A plausible implication is that HOI-Edit-44K fills a structural gap between HOI detection corpora and generic image editing datasets: it supplies paired supervision for how an interaction changes, not merely which interaction is present. The paper states that the dataset “will release publicly,” with the project page at https://jiuntian.github.io/OneHOI/, although license terms are not specified in the paper itself (Hoe et al., 15 Apr 2026).