---
title: 'HOI-Edit-44K: Identity-Preserving HOI Editing Dataset'
url: https://www.emergentmind.com/topics/hoi-edit-44k
type: topic
---

# HOI-Edit-44K: Identity-Preserving HOI Editing Dataset

Searching arXiv for the specific dataset and closely related HOI editing work to ground the article in current papers.
HOI-Edit-44K is a large-scale, paired dataset for human–object interaction (HOI) editing, introduced with OneHOI as a resource for learning identity-preserving changes in interaction rather than only recognizing or generating static HOI labels. It consists of **44,117 identity-preserving HOI editing pairs**, where each pair couples a source image with a target interaction triplet, an edited image realizing that target interaction, and a corresponding HOI layout for the edited image [2604.14062]. Its stated purpose is to address the scarcity of paired data for HOI editing, especially supervision that specifies how an interaction should change while preserving the identities of the subject and object. A central source of confusion in the literature is nomenclature: HOI-Edit-44K is not the same resource as **HOI-Edit**, a separate 705-sample benchmark for instruction-based image HOI editing introduced in a different paper [2606.19073].

## 1. Definition and intended problem scope

HOI-Edit-44K was created for **HOI editing**, not for conventional HOI detection alone. The target setting is an edit in which a model must change **how a person interacts with an object** without changing **who** the person is or **what** the object is. The dataset is described as a core contribution of OneHOI: “A new large-scale paired dataset, HOI-Edit-44K, addressing the scarcity of paired data, with 44K identity-preserving examples, for training of robust HOI editing” [2604.14062].

The motivation is defined against three limitations of prior resources. First, HOI recognition and detection datasets provide single images with HOI labels and boxes, but no explicit before/after editing pairs. Second, HOI generation settings condition on triplets and layouts, but again do not provide paired source and edited images. Third, general image editing datasets rarely include nuanced changes in human–object interaction while preserving identity. HOI-Edit-44K is intended to fill this gap by supplying paired supervision for **interaction change**, **identity preservation**, and **layout-aware grounding** within a single dataset [2604.14062].

The tasks it supports are stated explicitly. It is used to supervise **layout-free HOI editing**, **layout-guided HOI editing**, and the broader **unified HOI generation+editing** formulation used by OneHOI. The paper further identifies it as particularly important for **single-HOI editing**, **identity-preserving edits**, and training the model’s **structured interaction representation** [2604.14062].

## 2. Data composition and annotation schema

The supplementary material states that “The final dataset consists of **44,117** high-quality, paired HOI editing examples” [2604.14062]. Each sample contains four elements:

1. a **source image** containing at least one HOI,
2. a **target interaction triplet**,
3. an **edited image** realizing the target interaction,
4. the **corresponding HOI layout** for the edited image.

The paper describes the core HOI representation as the classical triplet
\[
\langle \text{subject}, \text{action}, \text{object} \rangle,
\]
while the main method denotes a set of instances as
\[
\{\langle s,o,a\rangle_n\}_{n=1}^N.
\]
For HOI-Edit-44K specifically, the supplement states that each sample includes the source image, the target interaction triplet, the edited image, and the edited-image HOI layout [2604.14062].

Dataset-wide category statistics are also given. HOI-Edit-44K contains **79 unique object categories**, **92 unique target actions**, and **372 unique HOI triplets**. The treemap visualizations described in the supplement indicate a long-tailed distribution, although the paper does not enumerate the frequency of each category in text [2604.14062].

A structurally important property is that HOI-Edit-44K is a **single-HOI editing** dataset. The paper states that the authors “synthesise potential single-HOI edits,” and that a pair is retained if it passes the verification pipeline. This means that each sample focuses on one target interaction change per pair, even if the underlying image may contain multiple humans or objects [2604.14062].

The paper does **not** specify train/validation/test splits for HOI-Edit-44K. Instead, it is described primarily as a **training dataset**. Evaluation in the OneHOI paper is conducted on **IEBench** and **MultiHOIEdit**, rather than on a held-out subset of HOI-Edit-44K itself [2604.14062].

## 3. Construction and automatic verification pipeline

The collection pipeline combines synthetic and real-image sources, then filters candidate edits with automated HOI and identity checks. Two types of source images are used. One stream consists of **Flux.1 generations that realise a verified source interaction**. The other stream consists of **HICO-DET images**, adding real photographs to the source pool [2604.14062].

For each source image, the authors generate candidate edits using **Flux.1 Kontext** and **InteractEdit**. These candidates are intended to realize a target HOI triplet given the source image, but only a small fraction are retained after verification [2604.14062].

The first automated check is **HOI correctness**. The edited image is processed by **PViC**, described in the paper as a state-of-the-art HOI detector. A candidate pair is retained only if the predicted HOI matches the target HOI triplet. When this match succeeds, the detected **subject/object layout** is recorded and becomes the layout annotation stored with the pair [2604.14062].

The second automated check is **identity preservation**. The pipeline extracts **DINOv2** features from subject and object crops in both the source and edited images, computes cosine similarity for both entities, and requires **both similarities \(\ge 0.75\)**. Only pairs meeting this threshold for both subject and object are kept [2604.14062].

The rejection rate is high: the paper reports that **around 90%** of initial candidate pairs are rejected. This is presented as evidence that constructing high-precision paired HOI editing data is difficult and that the final dataset is strongly filtered for both interaction correctness and identity preservation [2604.14062].

## 4. Spatial layout, structured representation, and training use

Within OneHOI, HOI-Edit-44K is not treated as unstructured text-image editing data. Its triplets and layouts are converted into a structured conditioning interface for the **Relational Diffusion Transformer (R-DiT)**. Let \(\mathcal{S}_n\), \(\mathcal{A}_n\), and \(\mathcal{O}_n\) denote the **T5-encoded** token sequences for the subject, action, and object of instance \(n\). The union of all HOI tokens is written as
\[
\mathcal{H} = \bigcup_{n=1}^N \{\mathcal{S}_n, \mathcal{A}_n, \mathcal{O}_n\}.
\]
For each role \(r \in \{s, a, o\}\) and instance \(n\), a T5 vector is augmented with a **role embedding**, an **instance embedding**, and a **box/layout embedding** derived from Fourier features of the corresponding bounding box [2604.14062].

HOI-Edit-44K provides **subject** and **object** boxes for the edited image. These are denoted \(b^s\) and \(b^o\), and they are used to define role-specific regions \(R_n^s\) and \(R_n^o\). The paper defines the **action region** as
\[
R_n^a = R_n^s \cup R_n^o.
\]
This union region is reported to better match the attention footprint of action tokens than the “between” region used in prior work [2604.14062].

These layouts drive **Structured HOI Attention**. For a query HOI token \(q\) and image token \(k\), the mask is defined so that subject tokens attend to subject regions, object tokens attend to object regions, and action tokens attend to the union region:
\[
M_{\mathcal{HI}}(q, k) =
\begin{cases}
0, & q \in \mathcal{S}_n \text{ and } k \in R^s_n \\
0, & q \in \mathcal{O}_n \text{ and } k \in R^o_n \\
0, & q \in \mathcal{A}_n \text{ and } k \in R^a_n \\
-\infty, & \text{otherwise.}
\end{cases}
\]
The intended effect is to make the action token mediate relation change, while subject and object tokens preserve role-specific identity and appearance [2604.14062].

The dataset is also central to the **editing** branch of OneHOI’s unified training regime. In editing, the model receives a source image latent, noise latent, target HOI triplet, and target layout boxes, and learns the edited image with a **flow-matching** objective. During training, **modality dropout** is applied: layout is dropped with probability \(p_{\text{layout}} = 0.25\), HOI labels with \(p_{\text{hoi}} = 0.25\), and the global text prompt with \(p_{\text{txt}} = 0.30\), while at least one modality is always kept. Because HOI-Edit-44K provides both triplets and layouts, it can support the full-conditioned and layout-dropped variants needed for unified editing and generation [2604.14062].

A further technical boundary is explicit: HOI-Edit-44K itself provides **bounding boxes**, not arbitrary segmentation masks. OneHOI supports arbitrary-shape masks through **SA-1B**, using the minimum enclosing bounding box for Fourier embedding. That capability is architectural and multi-dataset rather than a direct property of HOI-Edit-44K [2604.14062].

## 5. Relation to adjacent resources and common misidentifications

The most common misconception is to conflate **HOI-Edit-44K** with **HOI-Edit**. They are distinct resources introduced in different papers, for different purposes.

| Resource | Primary purpose | Scale |
|---|---|---|
| HOI-Edit-44K | Paired dataset for training identity-preserving HOI editing | 44,117 paired HOI editing examples |
| HOI-Edit | Benchmark for instruction-based image HOI editing with three cognitive levels | 705 samples |

The HOI-Edit benchmark is organized into **three progressive cognitive levels** and contains **357** L1 samples, **202** L2 samples, and **146** L3 samples, for a total of **705 samples**. The paper introducing HOI-Edit states explicitly that it “consistently uses the name HOI-Edit, not ‘HOI-Edit-44K’,” and that the term “HOI-Edit-44K” does not appear in its text, figures, or tables [2606.19073]. By contrast, OneHOI defines HOI-Edit-44K as a 44,117-pair training dataset [2604.14062].

HOI-Edit-44K also differs from HOI **detection** resources. Detection-oriented work such as **HOI Transformer** formulates HOI detection as set prediction of quintuples
\[
(c_h, c_r, c_o, b_h, b_o),
\]
predicted directly from an image, with no before/after editing supervision [2103.04503]. More recent DETR-based detectors such as **DQEN** refine query design for HOI localization and classification, again operating in the detection regime rather than paired editing [2508.18896]. HOI-Edit-44K is therefore best understood not as a detection benchmark, but as paired supervision for changing interactions while preserving identity.

## 6. Limitations, biases, and significance

Several limitations are either stated directly or follow immediately from the collection protocol. First, the dataset has a **low yield**: after candidate generation and filtering, only **44,117** pairs remain, with **around 90%** of initial candidates rejected. This indicates that automated HOI editing pair construction is difficult even when powerful edit generators are available [2604.14062].

Second, there is potential **teacher-model bias**. Edited images are produced using **Flux.1 Kontext** and **InteractEdit**, so some bias toward the behaviors of these systems is possible. The paper presents two mitigating steps: mixing in real **HICO-DET** images as sources and using strict HOI and identity thresholds during filtering [2604.14062].

Third, HOI-Edit-44K is primarily a **single-HOI** dataset. Multi-HOI editing is addressed separately through **MultiHOIEdit**, and the paper notes that multi-HOI editing is not explicitly present in HOI-Edit-44K itself [2604.14062]. Fourth, although the dataset is diverse, its coverage remains bounded at **79 objects**, **92 actions**, and **372 triplets**, which implies selective rather than exhaustive ontology coverage [2604.14062].

Its significance within OneHOI is stated directly: “This resource was critical for jointly training our unified model, providing the necessary supervision for robust, identity-preserving HOI editing” [2604.14062]. A plausible implication is that HOI-Edit-44K fills a structural gap between HOI detection corpora and generic image editing datasets: it supplies paired supervision for **how** an interaction changes, not merely **which** interaction is present. The paper states that the dataset “will release publicly,” with the project page at **https://jiuntian.github.io/OneHOI/**, although license terms are not specified in the paper itself [2604.14062].

Source: https://www.emergentmind.com/topics/hoi-edit-44k