Exemplar-Based Portrait Retouching

• Exemplar-based portrait photo retouching is a technique that uses reference portraits to explicitly guide localized edits while preserving unique facial identity.
• It integrates methods like correspondence matching, GANs, and diffusion models to transfer subtle global tone and local contrast adjustments.
• The approach emphasizes human-region priority and group-level consistency, enabling robust, semantically-aware enhancements across portrait sets.

Searching arXiv for recent and foundational papers on exemplar-based portrait retouching, portrait editing, and portrait retouching datasets. Exemplar-based portrait photo retouching is a portrait editing paradigm in which a reference portrait, a set of reference portraits, or an exemplar pair encodes the desired retouching operation for a new query portrait. Across the literature, the exemplar may provide global tone and color, local contrast and lighting, semantic color palettes and masks, or a before/after transformation whose underlying structural edit must be inferred and transferred. The area overlaps with portrait-specific requirements such as human-region priority and group-level consistency, but differs from generic photo style transfer by emphasizing identity preservation, localized facial and body edits, and the naturalness of skin, eyes, and illumination (Liang et al., 2021, Liu et al., 28 Jun 2026).

1. Scope and task formulation

Portrait photo retouching has been formalized as a distinct problem rather than a generic instance of image enhancement. PPR10K defines portrait photo retouching as the enhancement of collections of flat-looking RAW portrait photos under two portrait-specific constraints: Human-Region Priority (HRP), which requires that more attention be paid to human regions, and Group-Level Consistency (GLC), which requires that a group of portrait photos be adjusted to a consistent tone (Liang et al., 2021). In this formulation, the output is not merely an individually improved image; it is a portrait whose human regions are preferentially optimized and whose global tone remains coherent across a set.

Exemplar-based variants make the control signal explicit. In optimization-based systems, the exemplar is a guide portrait with a desirable skin tone; in patch-based and superpixel-based systems, it is one or more reference portraits whose local style statistics are transferred; in semantic GAN systems, it is converted into palettes, color distributions, or light and shadow masks; and in recent diffusion systems, it is an exemplar pair $(X_s, X_t)$ whose difference defines an implicit retouching operation to be applied to a new query image $X_q$ (Shirai et al., 2021, Liu et al., 2022, Liu et al., 28 Jun 2026). MirrorPPR makes this formulation explicit: the model receives an exemplar pair and must infer and apply the same retouching operations to a new query image, producing $\hat{Y}_q$ from $X_q$ (Liu et al., 28 Jun 2026).

A central motivation for exemplar guidance is the inadequacy of text for delicate structural portrait editing. MirrorPPR argues that textual descriptions struggle to convey fine-grained changes to facial features and body proportions, such as inner or outer eye distance, nasal alae width, lip thickness, jawline sharpness, chin length, temple filling, shoulder width, arm thickness, leg slimming, or waist slimming (Liu et al., 28 Jun 2026). This limitation is specific to portrait retouching because many desired edits are highly localized, small in magnitude, and identity-sensitive.

2. Correspondence-based and optimization-based foundations

A major early line of work formulates exemplar retouching through explicit correspondences and local statistical transfer. SuperBIG constructs dense correspondences between an input image and a reference image, extracts hierarchical features, partitions pixels into coupled superpixels with a two-step bipartite graph procedure, matches superpixels with the Hungarian algorithm, and performs local color transformation in the $l\alpha\beta$ decorrelated color space, followed by guided image filtering to smooth superpixel boundaries (Liu et al., 2016). Although proposed for general photographic style transfer, this formulation is directly relevant to portrait retouching because it treats style as photographic rendering—color, tone, and contrast—rather than painterly texture.

A related portrait-specific formulation transfers style through local energy rather than only color statistics. “Stylizing Face Images via Multiple Exemplars” uses multiple exemplar headshots in the same style, establishes patch correspondences with a Markov Random Field over overlapping $40 \times 40$ patches, and transfers local energy in a Laplacian stack. Different exemplar patches can be selected for different facial regions, and an edge-preserving guided filter removes artifacts caused by inconsistent boundaries between patches from different exemplars (Song et al., 2017). This multi-exemplar design addresses a core problem of portrait retouching: a single exemplar rarely matches every local component of a new face, whereas a set of exemplars can supply better local analogues for forehead, nose, cheeks, hair, or chin.

Exemplar guidance can also be cast as constrained optimization over a guide image. “Guided Facial Skin Color Correction” detects the face with Viola–Jones, extracts a facial skin region by HSV hue clustering with $k=4$ and saturation/value thresholds, performs exemplar-based color grading in RGB with Pitié et al.’s distribution matching, and then solves a hybrid guided image filtering problem in which the corrected image $\vec{x}$ is constrained to stay close to the graded guide image $\vec{g}$ on skin pixels and close to the original image $\vec{y}$ on background pixels (Shirai et al., 2021). The method adopts a local linear model

$X_q$0

and concludes with a luminance–chroma recombination step that uses the original intensity and the filtered chroma to preserve natural shading (Shirai et al., 2021). In portrait retouching terms, this is a principled exemplar-driven skin color normalization method that preserves background appearance and local facial detail.

Lighting and shadow are another foundational axis. “Portrait Shadow Manipulation” is not an exemplar method at inference time, but it parameterizes portrait lighting edits in a way that maps naturally to exemplar-based use. It decomposes the problem into foreign shadow removal and facial shadow softening plus virtual fill light, and exposes control parameters for light size $X_q$1 and fill intensity $X_q$2, implemented by two affine-field GridNet models (Zhang et al., 2020). The paper explicitly notes that these parameters define a lighting style space that can be fitted or inferred from exemplars, which is directly aligned with portrait retouching practice.

3. Learned portrait retouching with semantic and disentangled controls

Neural portrait editing introduced exemplar control through semantic abstractions rather than dense correspondences. “Flexible Portrait Image Editing with Fine-Grained Control” proposes a single asymmetric conditional GAN for geometry, color, light, and shadow editing, where the generator takes user-editable controls—noisy edge maps $X_q$3, color palettes $X_q$4, light masks $X_q$5, and shadow masks $X_q$6—while the discriminators receive cleaner and more spatially explicit conditions, including the clean edge map and a color map $X_q$7 (Liu et al., 2022). Exemplar-based retouching is realized by extracting a palette or a color-distribution palette from a reference portrait and combining it with the target’s geometry and lighting. The same framework also supports exemplar-based relighting by transferring light and shadow masks from a reference image (Liu et al., 2022). A further portrait-specific refinement is the region-weighted discriminator, which focuses on face and eye regions.

Disentanglement of content and style became more explicit in StyleGAN-based systems. “Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer” introduces DualStyleGAN, which separates portrait content and style into an intrinsic style path and an extrinsic style path. The intrinsic path reuses the pretrained StyleGAN face prior, while the extrinsic path modulates structure in coarse layers with ModRes blocks and color in fine layers with color transform blocks $X_q$8 (Yang et al., 2022). The design supports layer-wise control: coarse layers govern structure, fine layers govern color and texture. Although demonstrated on artistic portraits, the mechanism maps directly onto portrait retouching because the same hierarchy can encode subtle geometric refinement in coarse or mid-level layers and tone or skin rendering in fine layers (Yang et al., 2022).

A more explicit separation of geometry and appearance appears in “Exemplar-Based 3D Portrait Stylization,” which defines a two-stage pipeline. First, an exemplar-conditioned landmark translation network maps content and style landmarks to a stylized landmark configuration, and a landmark-guided Laplacian deformation transfers the geometry to a 3D face model. Second, a differentiable renderer and a multi-view STROTSS-style texture loss optimize a canonical texture $X_q$9 to match the exemplar’s appearance while preserving content semantics (Han et al., 2021). This 3D-aware decomposition is pertinent to retouching because it isolates geometry style from texture style and makes explicit the long-standing distinction between structural beautification and photometric enhancement.

These models collectively changed the unit of exemplar transfer. Instead of transferring only low-level color statistics, learned systems transfer semantic palettes, region-aware structure, or disentangled style codes, allowing exemplar-driven operations to be more localized, more identity-preserving, and more controllable.

4. Datasets, metrics, and evaluation protocols

The empirical study of portrait retouching depends heavily on portrait-specific datasets and metrics. PPR10K is the first large-scale portrait photo retouching benchmark of its kind, with 1,681 groups and 11,161 high-quality RAW portrait photos, each retouched by three experts and accompanied by high-resolution human-region masks (Liang et al., 2021). The dataset is explicitly organized around HRP and GLC, and provides a training/test split of 1,356 groups and 8,875 images for training and 325 groups and 2,286 images for testing (Liang et al., 2021). For exemplar-based systems, the group structure is especially useful because one retouched image can act as a style exemplar for the remaining images in the same group.

PPR10K also defined evaluation measures that have become central to portrait retouching. Beyond PSNR and $\hat{Y}_q$0, it introduced human-centered variants that weight human-region errors more heavily, and a group-level consistency measure based on color statistics in Lab space. The preferred formulation uses the $\hat{Y}_q$1 and $\hat{Y}_q$2 channels: $\hat{Y}_q$3 with lower values indicating more consistent global color tone within a group (Liang et al., 2021). This measure is especially natural for exemplar-based retouching, where multiple images are expected to align to the same reference style.

Recent exemplar-specific work required much larger paired resources. MirrorPPR introduces MirrorPPR47M, a dataset with over 47 million retouched pairs, divided into a simulated subset and a professional subset to support progressive curriculum learning (Liu et al., 28 Jun 2026). The simulated subset is built with Landmark-Guided Local Warping on FFHQ faces and covers 16 operations derived from eight base types with two directions, while the professional subset is produced with a commercial retouching API and covers 27 operations spanning facial features, face shape, and body proportions (Liu et al., 28 Jun 2026). The same work also introduces SimFace-100 and ProPortrait-500 as cross-identity benchmarks for facial and professional portrait retouching, respectively (Liu et al., 28 Jun 2026).

PortraitBench addresses a closely related evaluation problem from the generation side. It contains 1,000 real human images and is used to score portrait realism with OmniReward, UnifiedReward, and PickScore (Li et al., 25 Jun 2026). Although designed for photorealistic portrait generation rather than direct retouching, its portrait-centric criteria—content correctness, clarity, lighting and color, composition, coherence, and style—reflect the same quality axes emphasized in retouching practice (Li et al., 25 Jun 2026).

5. Diffusion-era exemplar transfer and operation learning

Diffusion-based portrait systems shifted exemplar retouching from explicit correspondence and latent mixing toward learned conditioning mechanisms. MagiCapture frames portrait customization as a multi-concept personalization problem in which a few subject exemplars and a few style exemplars are encoded as learned tokens on top of Stable Diffusion v1.5 (Hyung et al., 2023). It uses a two-phase schedule—first token embedding optimization, then LoRA fine-tuning—and a set of spatially aware losses: masked reconstruction for subject and style regions, a face-recognition identity prior based on ArcFace and RetinaFace, and an Attention Refocusing loss that constrains special tokens to attend only to their intended regions (Hyung et al., 2023). The pipeline ends with Real-ESRGAN and CodeFormer for high-resolution post-processing. This architecture is exemplar-based in the strict sense that the subject and the desired portrait style are both provided by example sets rather than textual specification alone.

MirrorPPR addresses an even narrower target: subtle structural portrait retouching. Its Retouching Operation Extractor uses a frozen MAE encoder and an R-Former with learnable query tokens to distill the operation embodied by an exemplar pair $\hat{Y}_q$4, and a connector maps the resulting representation into the conditioning space of a pretrained Diffusion Transformer (Liu et al., 28 Jun 2026). The final model fine-tunes only the connector, the extractor, and LoRA modules in the DiT under a flow-matching objective: $\hat{Y}_q$5 A key training device is data self-augmentation: instead of pairing different identities with partially misaligned operations, the method applies the same random spatial augmentation to both members of a retouched pair, producing $\hat{Y}_q$6 and preserving edit alignment while avoiding pixel-level shortcut learning (Liu et al., 28 Jun 2026). On ProPortrait-500, MirrorPPR-Pro reports PSNR $\hat{Y}_q$7, SSIM $\hat{Y}_q$8, LPIPS $\hat{Y}_q$9, and Face Similarity $X_q$0, substantially above the reported baselines (Liu et al., 28 Jun 2026).

A related but broader frontier is reward-based portrait post-training. PortraitGen introduces Exemplar-Driven GRPO, in which a real portrait is inserted into each GRPO sampling group via image inversion, and a dual-reward mechanism combines OmniReward with AI-Portrait to suppress AI artifacts such as oily skin and anatomical implausibilities (Li et al., 25 Jun 2026). This work targets photorealistic portrait generation rather than explicit retouching, but it shows that real exemplars and portrait-specific rewards can be used to teach a generative model what a retouched, artifact-free portrait should look like (Li et al., 25 Jun 2026). This suggests a convergence between exemplar conditioning and portrait-specific reward modeling.

6. Limitations, misconceptions, and future directions

A common misconception is that exemplar-based portrait retouching is synonymous with global color transfer. The literature is broader: it includes guide-image skin color correction, superpixel-wise local color statistics, multi-exemplar Laplacian energy transfer, semantic palette and mask conditioning, disentangled structure and color paths, and diffusion models that infer edit operators from before/after exemplar pairs (Shirai et al., 2021, Liu et al., 2016, Song et al., 2017, Liu et al., 2022, Liu et al., 28 Jun 2026). Another misconception is that text guidance can replace exemplars for subtle structural edits; MirrorPPR explicitly argues the opposite for fine-grained portrait reshaping (Liu et al., 28 Jun 2026).

Method-specific limitations remain substantial. In the conditional GAN framework of “Flexible Portrait Image Editing,” freckles, moles, and fine skin texture are often smoothed out, complex backgrounds are not preserved or recolored correctly, out-of-distribution accessories are not reproduced, frame-by-frame application to video leads to flicker, and extreme lighting or poses can produce artifacts (Liu et al., 2022). In guide-image skin color correction, HSV-based skin extraction can fail when hair and skin colors are similar, Viola–Jones face detection can fail under extreme pose or occlusion, and some residual artifacts may remain under severe shadow or flash conditions even after luminance recombination (Shirai et al., 2021). Multiple-exemplar patch methods still depend on reliable alignment and can leave seams or residual artifacts when neighboring patches from different exemplars disagree (Song et al., 2017). MagiCapture notes that abnormal body parts, demographic biases, and the general proximity of high-fidelity personalization to deepfake-style manipulation remain unresolved concerns (Hyung et al., 2023).

The direction of current research is therefore not only toward stronger generators, but toward better edit representations, stronger portrait-specific supervision, and better alignment between exemplar intent and target content. The surveyed work repeatedly points to explicit detail modeling, foreground–background separation, higher-resolution and multi-scale refinement, temporal coherence for video, better exemplar alignment through 3D models or landmarks, more interpretable sliders, and richer forms of conditioning beyond text alone (Liu et al., 2022, Han et al., 2021, Zeng et al., 2023). The recent emergence of paired-operation datasets and operation-centric diffusion conditioning further suggests that future exemplar-based portrait retouching will be defined less by style transfer in the narrow sense and more by the direct transfer of precise, identity-preserving retouching operators (Liu et al., 28 Jun 2026).