RPHarmony: Synthetic Image Harmonization Dataset

• RPHarmony is a synthetic dataset for image harmonization that employs Random Poisson Blending to introduce localized color, texture, and lighting variations.
• It addresses limitations of global color transfer by capturing spatially varying phenomena such as shadows and interreflections for more realistic composites.
• The dataset, integrated in the R2R architecture, provides benchmark metrics and improved generalization for harmonization models on both synthetic and real images.

Searching arXiv for papers mentioning RPHarmony and related uses. I’m checking arXiv for papers directly associated with the term and its surrounding context. Searching for "RPHarmony" on arXiv. RPHarmony most directly denotes a synthetic image harmonization dataset introduced alongside the region-to-region synthesis method Random Poisson Blending (RPB) in "Region-to-Region: Enhancing Generative Image Harmonization with Adaptive Regional Injection" (Zhang et al., 13 Aug 2025). It is constructed to produce composites with realistic, spatially localized disharmonies by transferring local color, texture, layout, and lighting cues from a randomly selected region in a random reference image into the foreground of a target real image. The resulting dataset contains 12,787 training images and 1,422 test images, each paired with a ground-truth real image and a foreground mask, and is used both to stress-test harmonization models and to improve their generalization to real composites (Zhang et al., 13 Aug 2025). In adjacent literature summaries, the same string also appears in broader senses related to preference harmonization, whereas it does not appear in "The Harmony Platform" (Falleri et al., 2013), and the acronym RPH in "Oracle Separations for RPH" refers instead to the real polynomial hierarchy (Hamm et al., 13 Feb 2025).

1. Terminology and scope

Within image harmonization, RPHarmony is the name of a new synthetic dataset generated via Random Poisson Blending, a region-to-region technique that transfers local appearance cues from a randomly sampled region in a random reference image into the foreground of a target real image (Zhang et al., 13 Aug 2025). The motivation is that prevailing synthetic training datasets, such as iHarmony4, are largely produced by global color transfer, which manipulates global foreground statistics by referencing semantically similar objects and therefore lacks local variation. The stated consequence is underrepresentation of complex real-world conditions such as spatially varying lighting, interreflections, shading, and shadows.

The term is potentially ambiguous. In the Harmony platform paper, the term used throughout is "Harmony platform"; the "RP" qualifier does not appear in the paper and is described as a likely external alias or search term (Falleri et al., 2013). In the preference-alignment exposition based on Robust Preference Optimization, "RPHarmony" denotes a harmonizing, robust preference-alignment layer that sits atop standard preference-optimization methods (Cao et al., 29 Sep 2025). In the ChARM exposition, the expression "RPHarmony-style system" is used for a role-playing agent aligned to user and character-specific preferences for harmonious interactions (Fang et al., 29 May 2025). By contrast, the theoretical paper on RPH studies the real polynomial hierarchy, not harmonization (Hamm et al., 13 Feb 2025).

2. Random Poisson Blending and dataset construction

RPB is defined as a region-to-region transformation. Rather than globally adjusting foreground color statistics, it transfers the color and illumination characteristics from a randomly selected suitable region in a random reference image into the foreground region of a target real image using Poisson image editing (Zhang et al., 13 Aug 2025). The pipeline is given step by step for a real target image $I_t$ and its foreground mask $M$.

Foreground extraction is defined by

$F_t = I_t \circ M,$

where $M \in \{0,1\}^{H \times W}$.

A different reference image $I_r$ is then selected, and a region $R$ in $I_r$ is randomly sampled to serve as guidance for blending. The foreground $F_t$ is blended onto $I_r$ at region $R$ using Poisson blending, yielding $M$0. A new foreground $M$1 is re-extracted from $M$2 at the paste location, and blending strength is controlled by

$M$3

Finally, $M$4 is pasted back into its original position in $M$5 using the original mask $M$6 to produce the synthetic composite $M$7.

The underlying Poisson image editing formulation is stated as a variational problem over a masked region $M$8:

$M$9

where $F_t = I_t \circ M,$0 is the unknown harmonized image inside $F_t = I_t \circ M,$1, $F_t = I_t \circ M,$2 is the source or foreground to paste, and $F_t = I_t \circ M,$3 is the background image used for boundary conditions. In discrete form, this becomes a least-squares system equivalent to solving a Poisson equation with a guidance field $F_t = I_t \circ M,$4:

$F_t = I_t \circ M,$5

The randomness in RPB arises from random selection of the reference image $F_t = I_t \circ M,$6, random sampling of a suitable region $F_t = I_t \circ M,$7 in $F_t = I_t \circ M,$8, random blending strength $F_t = I_t \circ M,$9, the foreground mask $M \in \{0,1\}^{H \times W}$0 that defines $M \in \{0,1\}^{H \times W}$1, and the position of paste-in and region-scale variability. Additional checks and filtering are applied to ensure the Poisson blending is valid and the synthesized image quality is high; the stated mechanism is an aesthetics scoring model plus manual filtering, while the precise details are deferred to the Supplementary (Zhang et al., 13 Aug 2025).

3. Dataset composition, supervision, and evaluation protocol

RPHarmony is synthesized from high-quality masks taken from DUTS, with 15,572 images, and ADE20K, with 27,574 images, and is divided into two sub-datasets, R-DUTS and R-ADE (Zhang et al., 13 Aug 2025). Each sample includes a composite image $M \in \{0,1\}^{H \times W}$2 as input, the original real image $M \in \{0,1\}^{H \times W}$3 as target, and the foreground mask $M \in \{0,1\}^{H \times W}$4 as annotation, enabling supervised training and evaluation with pixel-level mask guidance.

The experimental protocol is fixed. Training images are resized to $M \in \{0,1\}^{H \times W}$5. Testing is performed at $M \in \{0,1\}^{H \times W}$6, and results are resized to $M \in \{0,1\}^{H \times W}$7 for evaluation, following the same protocol used for iHarmony4. Models are first pre-trained on iHarmony4 and then fine-tuned on RPHarmony. Reported metrics are PSNR, MSE, fMSE, and SSIM, with standard definitions used in practice (Zhang et al., 13 Aug 2025).

Model	PSNR / MSE	fMSE / SSIM
Composite baseline	25.91 / 366.32	2362.08 / 0.9580
PCT-Net	33.26 / 60.39	332.61 / 0.9796
AICT	33.28 / 60.38	333.15 / 0.9547
HDNet	34.46 / 47.52	252.54 / 0.9810
DiffHarmony++	36.03 / 42.16	203.45 / 0.9861
R2R (Ours)	36.32 / 40.25	192.66 / 0.9872

The reported interpretation is that RPHarmony’s richer local variations challenge models: several methods that perform comparably on iHarmony4 drop notably on RPHarmony. The proposed R2R model maintains superior performance, which the paper presents as evidence of improved robustness to localized disharmony (Zhang et al., 13 Aug 2025).

Generalization is also evaluated on RealHM using DeQA-Score:

Model	DeQA-Score
HDNet	3.824
HDNet* (fine-tuned on RPHarmony)	3.877
Ours	4.039
Ours* (fine-tuned on RPHarmony)	4.082

These numbers are reported as showing that fine-tuning on RPHarmony improves realism on real composites, suggesting improved generalization beyond synthetic settings (Zhang et al., 13 Aug 2025). A plausible implication is that the dataset is useful not only as a benchmark but also as a regularizer against the synthetic bias induced by global color-transfer pipelines.

4. Role within the R2R architecture

RPHarmony is tightly coupled to the R2R model described in the same work (Zhang et al., 13 Aug 2025). The stated relationship has three parts: the dataset stresses models with region-localized color and lighting differences, it strengthens the Harmony Controller and Mask-aware Adaptive Channel Attention (MACA) during fine-tuning, and its RPB composites serve as negative samples in Clear-VAE’s contrastive regularization.

Clear-VAE is motivated by the claim that a standard VAE encoder in an LDM may lose high-frequency details, while naive skip connections can reintroduce disharmonious elements. The Adaptive Filter extracts high-frequency information from skip features during encoding of composite images and merges them into the decoder with zero-initialized $M \in \{0,1\}^{H \times W}$8 convolutions. The losses are

$M \in \{0,1\}^{H \times W}$9

$I_r$0

with $I_r$1 the $I_r$2 distance, $I_r$3, and

$I_r$4

The paper states that using RPB composites as hard negatives strengthens the model’s ability to preserve details while discarding disharmonious cues.

MACA is defined on a feature map $I_r$5 and mask $I_r$6 with $I_r$7. Foreground and background feature maps are

$I_r$8

Global channel descriptors $I_r$9 and $R$0 are produced via convolution and GAP, then concatenated and passed through an MLP to regress channelwise scale $R$1 and shift $R$2. Channel modulation is

$R$3

and the final mask-aware output is

$R$4

The stated function is to adapt low-level style, color, and brightness channels in the foreground under guidance from the background.

The latent diffusion objective is a foreground-weighted MSE:

$R$5

Here $R$6 is resized to latent resolution, $R$7, and $R$8 and $R$9 are the standard diffusion noise target and prediction. Training uses Clear-VAE for 30 epochs with AdamW, $I_r$0, $I_r$1, learning rate $I_r$2, batch size 4, and gradient accumulation 8; the controller is then fine-tuned for 5 epochs with AdamW at learning rate $I_r$3. Inference uses an Euler ancestral discrete scheduler with 10 steps, and the implementation is in PyTorch on $I_r$4 GPUs (Zhang et al., 13 Aug 2025).

5. RPHarmony as a preference-harmonization layer

In a separate usage, the Robust Preference Optimization exposition defines RPHarmony as a harmonizing, robust preference-alignment layer that sits atop standard PO methods (Cao et al., 29 Sep 2025). Here the object is not image harmonization but harmonization of heterogeneous, noisy human feedback. The core problem setting observes preference labels that may be noisy and may come from heterogeneous annotators. For pairwise data $I_r$5, a latent correctness variable $I_r$6 indicates whether the observed orientation agrees with the underlying collective preference.

The model introduces annotator-specific reliability $I_r$7 and computes posterior correctness weights

$I_r$8

These weights drive a reweighted preference objective,

$I_r$9

or a one-sided variant

$F_t$0

Within that exposition, RPHarmony names the harmonization of pluralistic and noisy preference signals into a stable collective preference.

The same summary extends the idea to pluralistic settings via annotator-specific reliabilities and latent preference clusters. The distinction drawn there is between random noise, which is downweighted by low $F_t$1, and systematic differences, which manifest as persistent disagreements captured by per-annotator or per-cluster reliabilities rather than being treated only as outliers (Cao et al., 29 Sep 2025). This suggests a generalized notion of harmonization in which "harmony" refers to consensus inference under noise, not to foreground–background consistency in images.

6. Related and potentially confusing uses

A further adjacent usage appears in the ChARM exposition, which connects ChARM to building an "RPHarmony-style system": a role-playing agent aligned to user and character-specific preferences for harmonious interactions (Fang et al., 29 May 2025). In that setting, the relevant mechanisms are the act-adaptive margin,

$F_t$2

the margin-enhanced Bradley–Terry objective,

$F_t$3

and the combined reward-model loss

$F_t$4

with $F_t$5 in the implementation summary. The associated datasets are RoleplayPref and RoleplayEval rather than RPHarmony proper, so the expression functions as a design analogy rather than as the name of the image harmonization dataset (Fang et al., 29 May 2025).

Two additional confusions are explicitly addressed in the supplied literature. First, "RPHarmony" does not appear in "The Harmony Platform"; the paper consistently uses "Harmony" or "Harmony platform", an OSGi-based framework for Mining Software Repositories with a unified abstract model over Source, Event, Author, Action, and Item (Falleri et al., 2013). Second, "RPH" in "Oracle Separations for RPH" denotes the real polynomial hierarchy, defined via real Turing machines and alternating quantification over reals, with results such as $F_t$6 and oracle separations including $F_t$7 (Hamm et al., 13 Feb 2025). Those uses are terminologically proximate but conceptually unrelated.

Taken narrowly, RPHarmony is therefore a dataset for synthetic image harmonization generated by Random Poisson Blending and used to improve robustness to localized disharmony (Zhang et al., 13 Aug 2025). Taken broadly across nearby literature summaries, the string also functions as a label for harmonization in preference-alignment and role-playing settings. The common motif is the normalization of heterogeneous signals—local appearance cues in images, heterogeneous annotator feedback in preference optimization, or multi-act role-playing preferences—but only the first of these is the formal dataset name RPHarmony in the cited arXiv corpus.