Instance-Level Style Transfer Overview

• Instance-Level Style Transfer is defined as a method that conditions stylization on specific instances, such as exemplars, regions, or object masks, rather than a single global style.
• Methods range from exemplar-conditioned statistical alignment like AdaIN to spatially adaptive approaches using attention and mask-based control, illustrating a spectrum of instance specificity.
• These techniques enable targeted applications such as facial attribute transfer, sim2real adaptation, and object-specific stylization, improving visual fidelity and performance in diverse scenarios.

Instance-level style transfer is a family of style-transfer formulations in which stylization is conditioned by a specific instance rather than only by a fixed style category or a globally shared transformation. In the literature, the phrase is used in several non-equivalent ways. It can denote arbitrary exemplar-conditioned transfer, where a previously unseen style image determines the stylization of a content image at test time, as in Adaptive Instance Normalization (Huang et al., 2017). It can also denote spatially adaptive or region-aware transfer, where different positions in the content image receive different style cues through attention or local correspondence, as in SAFIN (Singh et al., 2021) and Z-STAR+ (Deng et al., 2024). In a stricter vision sense, it can denote object- or mask-level stylization of a selected image instance, as in targeted style transfer with instance-aware semantic segmentation (Castillo et al., 2017). Taken together, these works suggest that instance-level style transfer is best understood as a spectrum from exemplar-conditioned global modulation to explicit per-instance control.

1. Scope and terminology

The literature uses the term “instance-level” to describe several related but distinct operations. In one usage, the instance is the style exemplar itself: a single style image provided at inference time, with no retraining for that style. In another, the instance is a spatial region or semantic part whose stylization differs from the rest of the image. In a third, the instance is an explicit object mask or object identity selected by a user or recovered by a segmentation pipeline. In still other settings, the instance is an object identity in a structured domain, such as a facial attribute exemplar, a rendered object for sim2real adaptation, or a grouped 3D scene element (Yin et al., 2018).

Usage of “instance-level”	Defining mechanism	Representative works
Exemplar-conditioned arbitrary transfer	Style parameters are computed from a specific unseen style image	AdaIN (Huang et al., 2017), DIN (Jing et al., 2019)
Region-aware or spatially adaptive transfer	Different spatial positions receive different style cues	SAFIN (Singh et al., 2021), Z-STAR+ (Deng et al., 2024)
Explicit object-instance stylization	Masks or instance segmentation constrain where style is applied	Targeted style transfer (Castillo et al., 2017), Soulstyler (Chen et al., 2023)
Structured-domain instance transfer	Transfer operates on object, attribute, or 3D instance identities	facial attributes (Yin et al., 2018), sim2real pose (Ikeda et al., 2022), 3D scenes (Xu et al., 4 Sep 2025)

This suggests that the central question is not only what style is transferred, but also what unit is being conditioned or edited. Some papers seek arbitrary transfer from a single exemplar. Others seek control over where style appears. Others still seek faithful transfer of a particular object- or attribute-level appearance instance.

2. Exemplar-conditioned arbitrary style transfer

A foundational meaning of instance-level style transfer appears in arbitrary style transfer: the model is conditioned on a specific style image instance at inference time. AdaIN formalized this by aligning the channel-wise mean and variance of content features to those of the style features,

$$\mathrm{AdaIN}(x, y) = \sigma(y)\left(\frac{x-\mu(x)}{\sigma(x)}\right)+\mu(y),$$

within an encoder $\rightarrow$ AdaIN $\rightarrow$ decoder architecture using a fixed VGG-19 encoder up to $relu4\_1$ and a trainable decoder (Huang et al., 2017). The key consequence is that no style-specific retraining is required; the current style exemplar directly supplies the affine parameters. The same formulation also supports content-style trade-off, style interpolation, color preservation, and spatial control by applying AdaIN to different masked regions.

Later normalization-based methods broadened this exemplar-conditioned view. Dynamic Instance Normalization replaces fixed affine modulation with a style-conditioned dynamic convolution,

$$\mathrm{DIN}(\mathcal{F}_c,\mathcal{F}_s)= f[\mathcal{F}_s,\mathrm{IN}(\mathcal{F}_c)],$$

so that the current style image generates convolution weights and biases rather than only channel-wise statistics (Jing et al., 2019). This removes the need for a heavy shared encoder for both content and style and permits a lightweight MobileNetV1-based content encoder. The paper reports a reduction factor of more than twenty in computational cost relative to existing approaches, while preserving arbitrary-style behavior.

Graph Instance Normalization keeps the arbitrary-style setting but changes what “instance” means. It constructs a graph over style instances in a mini-batch, smooths style mean vectors with graph convolution during training, and then removes the graph layers at test time so that inference remains single-image and AdaIN-like (Jung et al., 2020). Here, the instance is not an object region inside an image but a batch element. The method is therefore instance-level in the sense of per-style-sample conditioning, not object-level control.

Thumbnail Instance Normalization extends exemplar-conditioned normalization to ultra-resolution transfer by computing normalization statistics from a thumbnail of the current content image and reusing them for all full-resolution patches (Chen et al., 2021). The paper explicitly frames this as a per-image adaptive mechanism rather than an object-instance method. This distinction is important: in some works, “instance-level” refers to conditioning on a particular image instance as a whole, not to segmenting and stylizing separate scene instances.

3. Spatially adaptive and region-aware modulation

A second major line of work pushes arbitrary style transfer beyond global channel statistics toward spatially varying, content-conditioned modulation. SAFIN is a central example. It begins from Factorized Instance Normalization,

$$\text{FIN}(F_c) = \gamma_s \odot \big(\gamma_{ind} \odot \overline{F_c} + \beta_{ind} \big) + \beta_s,$$

where $\gamma_{ind}, \beta_{ind}\in\mathbb{R}^{C}$ are style-independent channel parameters and $\gamma_s, \beta_s\in\mathbb{R}^{H * W * C}$ are style-dependent spatially varying parameters inferred by attention between content and style features (Singh et al., 2021). These parameters are generated as

$$\gamma_s = \operatorname{ReLU}(W_{\gamma} \otimes \text{SA}(\overline{F_c}, \overline{F_s})), \qquad \beta_s = \operatorname{ReLU}(W_{\beta} \otimes \text{SA}(\overline{F_c}, \overline{F_s})).$$

The paper’s core claim is that attention should generate normalization parameters rather than directly synthesize or replace stylized features. This preserves the benefits of semantic correspondence while reducing SANet-style artifacts such as “eye-like artifacts.” The resulting mechanism is explicitly described as weaker than semantic segmentation or dense instance matching, but stronger than global style transfer: different spatial positions receive different affine parameters inferred from content-style correspondence.

Z-STAR+ reaches a similar region-aware goal in a zero-shot diffusion setting. It uses dual denoising paths, a Cross-attention Reweighting module in which content tokens query style tokens, and a scaled AdaIN module for global color and statistic alignment (Deng et al., 2024). The paper is careful about terminology: it does not claim explicit instance-level control in the strict sense of masks or segmented objects. Rather, it describes the method as patch-aware, token-aware, and weakly region-aware. Spatial locality arises from latent feature correspondence, not from explicit instance supervision.

A related 2024 formulation combines local cross-attention with Transformer-derived global style modulation and CLIP-based contrastive supervision. It introduces Style Consistency Instance Normalization,

$$SCIN(F_c,F_s)=\gamma^s \times \operatorname{IN}(\mathcal{F}_c)+\beta^s,$$

where $\rightarrow$0 are generated from Transformer-encoded style tokens, and an Instance-based Contrastive Learning objective that groups stylized outputs by shared style instance or shared content instance (Zhang et al., 2024). This remains global channel-wise modulation in its normalization formula, but it enriches exemplar conditioning beyond mean-variance alignment and explicitly learns relations among individual style instances.

4. Explicit object- and mask-level stylization

In the strictest sense, instance-level style transfer means that only a selected object instance should be stylized while the rest of the scene remains unchanged. Early targeted style transfer followed a modular pipeline: apply Gatys-style transfer to the whole image, select an object instance using instance-aware semantic segmentation, extract the stylized object region, and composite it back into the original image (Castillo et al., 2017). The distinctive step is boundary refinement with a binary MRF over an ambiguous narrow band near the mask boundary. Its energy

$\rightarrow$1

uses a unary term that keeps the seam near the original segmentation boundary and a pairwise term that favors transitions where stylized and original appearances are already similar. This formulation is explicitly targeted at one user-selected object instance.

Another route is to stylize the instance domain directly rather than stylizing the full image and cutting afterward. “A Method for Arbitrary Instance Style Transfer” proposes Forward Stretching and Backward Stretching: an irregular masked instance is mapped row by row into a rectangular tensor, stylized with WCT, then restored to its original support and composited back into the image (Yu et al., 2019). The method is topologically motivated but implemented as a discrete raster-space deformation. Its central limitation is equally explicit: the mapping is only one-dimensional per row, so highly complex or hole-containing shapes can be distorted.

More recent object-based methods tighten the integration between localization and stylization. A YOLOv8x-seg-based approach reuses the same network family for instance segmentation and style/content feature extraction, then applies mask-based blending to stylize single or multiple objects, including a three-vase example in which different styles are assigned to different instances (Kulkarni et al., 2024). Soulstyler replaces the reference style image with a natural-language instruction. A LLM splits the prompt into “Stylized Content” and “Stylized Objects,” CRIS generates the target mask from the object phrase, and a CLIPstyler-based model adds a mask-aware loss so that only the referred object region is stylized (Chen et al., 2023). In both cases, instance-level control depends on an explicit mask rather than on implicit attention alone.

5. Structured-domain and application-specific instance transfer

Outside generic artistic transfer, the term acquires even sharper meanings. In facial attribute transfer, instance-level means transferring the exact mustache, goatee, or eyeglasses from a source face to a target face rather than generating a generic semantic attribute. Geometry-aware flow models this with a landmark-guided dense warp, an attention mask for warp-and-blend composition, and a refinement network that predicts an appearance residual (Yin et al., 2018). The transfer module $\rightarrow$2 and the removal module $\rightarrow$3 are trained in a cycle-consistency framework on unpaired data. Here, the instance is a concrete attribute exemplar whose geometry and fine appearance should be preserved.

In sim2real adaptation for 6D pose estimation, instance-level style transfer means translating each rendered object instance independently, with one style transfer network per object class, then compositing the translated object back into the synthetic scene (Ikeda et al., 2022). The pipeline uses weakly paired synthetic/real crops, PatchNCE and GAN-based training for TransNet, and mask-based overlay at inference. The paper reports that its method is the only one that improves pose-estimation performance over non-adaptation for all six tested objects, including transparent and shiny household objects, under ADD pass rate at 2 cm and AUC.

In 3D scene stylization, SSGaussian defines instance-level transfer through Gaussian Grouping identities and cross-view consistency rather than through 2D masks alone (Xu et al., 4 Sep 2025). Stylized key views are first generated with cross-view attention in the last upsampling block of a diffusion UNet, then transferred back to the 3D Gaussian representation with a group-matched nearest-neighbor style loss,

$\rightarrow$4

The aim is not exact pixel-level consistency but instance-level consistency: the same object should retain stylistic coherence across viewpoints even when individual renders differ.

6. Representations, evaluation practices, and recurrent tensions

A persistent tension in the field concerns what constitutes the style of an instance. AdaIN and related methods treat channel-wise means and variances as sufficient style carriers (Huang et al., 2017). SAFIN and SCIN retain normalization as the carrier but make the modulation spatially adaptive or Transformer-conditioned (Singh et al., 2021). Diffusion-based methods such as Z-STAR+ and StyleStudio add cross-attention, scaled AdaIN, selective guidance, or teacher-based stabilization to separate local correspondence, global calibration, and prompt fidelity (Deng et al., 2024). By contrast, “Style is a Distribution of Features” argues that low-order statistics are fundamentally incomplete and replaces conventional style losses with an adversarial approximation to the Wasserstein distance between empirical feature distributions (Huang et al., 2020). That paper’s central claim is that richer distribution matching improves style fidelity, but it also identifies a new failure mode: stronger style matching can leak semantic content from the style image into the result.

The literature also repeatedly warns against terminological overreach. SAFIN explicitly does not perform semantic segmentation, object-instance matching, or dense semantic correspondence supervision; it is better characterized as region-aware or spatially adaptive feature modulation (Singh et al., 2021). Z-STAR+ likewise does not use masks or detection boxes and is better described as patch-aware or token-aware than as strict instance-level control (Deng et al., 2024). The central-object-aware extension of Gatys-style transfer modifies the content loss with a spatial importance mask derived from classifier perturbations, but it does not support independent stylization of multiple objects or per-instance style assignment (Schekalev et al., 2019). Even GrIN’s “instance” is the style sample in a mini-batch, not an object in a scene (Jung et al., 2020). This suggests that “instance-level” is best read in context, not as a uniform task label.

Evaluation practice reflects the same diversity. Some object-targeted methods are mainly qualitative, emphasizing seam quality, boundary preservation, or background protection (Castillo et al., 2017). Others rely on user studies, deception tests, or CLIP-based alignment scores (Zhang et al., 2024). Structured applications often use task-specific metrics: ADD and AUC for pose estimation (Ikeda et al., 2022), or multi-view LPIPS and RMSE for 3D scene consistency (Xu et al., 4 Sep 2025). A plausible implication is that the field has not converged on a single benchmark definition of instance-level success. Depending on the formulation, success may mean faithful transfer from one style exemplar, semantically coherent placement of textures, precise stylization of a user-selected object, preservation of object identity across viewpoints, or selective suppression of unwanted style elements.

Across these formulations, instance-level style transfer remains defined by a common ambition: stylization should depend on a particular instance rather than only on a global style label. The technical realization of that ambition, however, ranges from channel-statistic alignment and dynamic filtering to spatial attention, mask-constrained compositing, geometry-aware warping, and identity-consistent 3D transfer.