Controllable Style Brush Techniques

• Style Brush is a family of techniques enabling controllable style transfer by disentangling stylistic appearance from structural content across various media.
• Methods range from latent diffusion and explicit brushstroke optimization to density-aware NeRF, with evaluations using metrics like FID and ArtFID.
• Interactive interfaces and robotic systems demonstrate practical user control despite challenges in computational cost and multi-modal consistency.

Searching arXiv for papers on “Style Brush” and closely related stylization frameworks. “Style Brush” is not a single canonical technique in the arXiv literature. It denotes a family of methods that treat style transfer as a controllable operation over images, text, 3D scenes, meshes, or physical brush trajectories, with the common goal of separating stylistic appearance from structural content while preserving user control. In some works, the term refers to single-reference image stylization built on latent diffusion; in others, it refers to interactive local transfer, explicit brushstroke-domain optimization, 3D-guided texture synthesis, reusable sculpting brushes, or robotic reproduction of human stroke style (Feng et al., 2024). A related critical usage appears in “The Silent Brush,” where the phrase names unintended stylistic resurfacing in generative models (Joshi et al., 17 May 2026).

1. Scope and historical usage

Early uses of the “brush” metaphor were largely exemplar-based and photographic rather than painterly. “Photo Stylistic Brush” formulated style transfer as robust correspondence between an input photo and a reference photo, emphasizing color, tone, and contrast rather than brush marks, and implemented this through SuperBIG, a two-step superpixel-based bipartite graph pipeline in $l\alpha\beta$-CIE space (Liu et al., 2016). This usage established a durable interpretation of a style brush as an operation that picks up appearance statistics from an exemplar and applies them locally with structural awareness.

Later work expanded the term in several distinct directions.

Domain	Representative work	Distinctive notion of “Style Brush”
Photographic exemplar transfer	“Photo Stylistic Brush” (Liu et al., 2016)	Local correspondence-guided photo look transfer
Interactive 2D stylization	“Interactive Style Transfer: All is Your Palette” (Lin et al., 2022)	Dip-and-paint local style application with fluid diffusion
Single-image reference stylization	“StyleBrush: Style Extraction and Transfer from a Single Image” (Feng et al., 2024)	Dual-branch diffusion for style extraction and structure preservation
Brushstroke-domain NST	“Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes” (Kotovenko et al., 2021)	Optimization over parameterized strokes instead of pixels
3D texture or scene stylization	“Style Brush: Guided Style Transfer for 3D Objects” (Kovács et al., 3 Oct 2025)	Directional, multi-style mesh texturing with guide textures

This diversity matters because the phrase can otherwise be misunderstood as a literal paintbrush simulation. In practice, the literature spans at least four conceptually different objects: a local editing interface, a style-conditioned generator, an explicit stroke representation, and a reusable geometric brush asset. A related industrial variant is the “Smart Brush” for AAA map editing, which regenerates masked tile-mask regions using contextual map representations rather than an external style exemplar (Gnatyuk et al., 25 Mar 2025).

2. Style–structure disentanglement as the central design problem

The dominant modern formulation treats Style Brush as a disentanglement problem: style should be extracted from a reference, but structure should remain tied to the target content. “StyleBrush: Style Extraction and Transfer from a Single Image” implements this directly in latent diffusion with two branches on top of Stable Diffusion v1.5: ReferenceNet extracts style from a single reference image, while Structure Guider extracts structural features from a grayscale-and-blurred version of the content image (Feng et al., 2024). The grayscale-plus-blur preprocessing is explicitly motivated as a way to remove color style and suppress texture style while retaining layout and semantics. The method further uses CLIP features and cross-attention for semantic alignment, and reports state-of-the-art quantitative performance with ArtFID $14.1$, FID $8.3$, LPIPS $0.51$, and user score $8.6$ on 500 randomly selected image pairs (Feng et al., 2024).

A closely related decomposition appears in text-image stylization. “TextStyleBrush: Transfer of Text Aesthetics from a Single Example” extracts a fixed 512D style code $e_s = F_s(\mathbb{I}_{s,c})$ from a localized source word image and a spatial content representation $e_c \in \mathbb{R}^{512 \times 4 \times W}$ from a rendered target string, then injects style through layer-specific mappings $w_{s,i} = M_i(e_s)$ in a StyleGAN2-inspired generator (Krishnan et al., 2021). Here the style code is deliberately non-parametric: typography, handwriting identity, color, background interaction, noise, and deformation are not factorized into interpretable attributes. The method is self-supervised on real data, uses a pre-trained typeface classifier and OCR recognizer, and achieves OCR accuracies of $97.2$ on ICDAR13, $97.6$ on ICDAR15, and $14.1$0 on TextVQA (Krishnan et al., 2021).

Handwritten line generation extends the same problem to line-level regularities. “DiffBrush: Beyond Isolated Words” argues that handwriting style includes not only intra-word cues such as slant, ligatures, and stroke width, but also inter-word properties such as baseline consistency, vertical alignment, and horizontal spacing (Dai et al., 5 Aug 2025). Its content-decoupled style learning uses column-wise masking for vertical style and row-wise masking for horizontal style, combined with Proxy-NCA-style objectives. Multi-scale content learning then adds a line discriminator and a word discriminator. On IAM, DiffBrush reports HWD $14.1$1, $14.1$2 $14.1$3, $14.1$4 $14.1$5, FID $14.1$6, and IS $14.1$7; on CVL, it reports HWD $14.1$8, $14.1$9 $8.3$0, $8.3$1 $8.3$2, and FID $8.3$3 (Dai et al., 5 Aug 2025).

A structurally similar but non-diffusion approach appears in brush handwriting font generation. “SE-GAN” treats brush handwriting font generation as requiring preservation of a character skeleton together with target-style stroke deformation, using an image encoder, a skeleton encoder, self-attentive refined attention, and a skeleton discriminator (Yuan et al., 2022). The skeleton is both conditional input and supervisory target. Across six styles, SE-GAN reports the best or nearly best Top-1 content accuracy and the lowest FID for almost all styles, including Acc/FID pairs such as $8.3$4 for Style 3 and $8.3$5 for Style 5 (Yuan et al., 2022).

3. Brushstroke-domain methods and interactive control

A second major lineage interprets Style Brush literally: style transfer should operate in the domain of strokes rather than pixels. “Rethinking Style Transfer: From Pixels to Parameterized Brushstrokes” replaces pixel optimization with optimization over $8.3$6 parameterized strokes, rendered by a differentiable renderer $8.3$7, where each stroke is centered on a quadratic Bézier curve,

$8.3$8

and has explicit color and width parameters (Kotovenko et al., 2021). The style objective remains Gatys-style perceptual loss, but the output space is constrained to painterly primitives. The method reports a deception score of $8.3$9 and human deception rate of $0.51$0, compared with $0.51$1 for Gatys and $0.51$2 for AST (Kotovenko et al., 2021). The later “Parameterized Brushstroke Style Transfer” restates the same idea as a brushstroke-domain alternative to RGB-domain style transfer, with $0.51$3 stroke parameters and a hybrid pipeline combining stroke-parameter optimization and a final pixel-level refinement step (Meleti et al., 8 Mar 2026).

Interactive systems translate the brush metaphor into a user interface. “Interactive Style Transfer: All is Your Palette” defines a literal dip-and-paint workflow: style is sampled from local regions of one or more style images, then applied to selected content regions through similarity-guided fluid diffusion (Lin et al., 2022). The method computes similarity maps with VGG-19 features, derives a penetration map $0.51$4 by solving

$0.51$5

and uses local weighted AdaIN statistics to stylize only the painted region. It reports $0.51$6 FPS for fluid simulation alone on $0.51$7 and $0.51$8 FPS including style transfer (Lin et al., 2022). This is one of the clearest realizations of a style brush as a local, brush-like operator rather than a global filter.

“StyleTune” pushes this same idea toward mobile editing by making stroke size, intensity, and orientation directly controllable (Reimann et al., 2021). Stroke size is controlled through a low-resolution stylization branch parameterized by $0.51$9, intensity through conditional instance normalization driven by $8.6$0, and orientation through rotation $8.6$1 of the content input. Local edits are implemented by blending feature tensors from several stroke-size levels under a painted mask $8.6$2, using image-space blending for fast preview and feature-space blending for the final decode (Reimann et al., 2021). On an iPad Pro 3rd gen, it reports real-time global style transfer and image-space blending on $8.6$3, about $8.6$4 s for pre-generation of $8.6$5 stroke sizes, and about $8.6$6 s for decoder-based model-space blending (Reimann et al., 2021).

4. 3D scenes, meshes, and geometric brushes

In 3D settings, Style Brush diverges into appearance stylization of scene representations and generation of reusable geometric brush assets. “Multi-style Neural Radiance Field with AdaIN” addresses stylized novel view synthesis by combining a frozen vanilla NeRF backbone with style features derived from VGG statistics and AdaIN supervision (Pao et al., 2024). Its distinctive contribution for brush-heavy styles is a density-aware variant in which style conditions not only color but also density prediction. The paper explicitly motivates this by arguing that strong brush strokes depend on more than recoloring; they also depend on opacity, spatial accumulation, and style-dependent modulation of where matter appears along a ray (Pao et al., 2024). The same model supports multi-style inference, style interpolation, and intensity control by interpolating style feature statistics.

“Style Brush: Guided Style Transfer for 3D Objects” transfers the brush metaphor to textured meshes (Kovács et al., 3 Oct 2025). The user paints a guiding texture $8.6$7 in UV space, optionally a style mask texture $8.6$8, and the system computes Edge Tangent Flow on both style images and rendered guides. The central loss is a directional nearest-neighbor feature matching objective that restricts each rendered feature to style features from the same orientation bin: $8.6$9 This yields directional stylization, support for multiple styles or masked portions thereof, and smooth transitions over the surface (Kovács et al., 3 Oct 2025). The method uses $e_s = F_s(\mathbb{I}_{s,c})$0 viewpoints on a Fibonacci sphere, $e_s = F_s(\mathbb{I}_{s,c})$1 textures, Adam with learning rate $e_s = F_s(\mathbb{I}_{s,c})$2, $e_s = F_s(\mathbb{I}_{s,c})$3 iterations per multiscale step, and approximately $e_s = F_s(\mathbb{I}_{s,c})$4 minutes total runtime on an RTX 4080 SUPER with 16GB VRAM for the default one-style setting (Kovács et al., 3 Oct 2025).

A different 3D interpretation appears in “Text2VDM,” where a style brush is a reusable vector displacement map for sculpting rather than an appearance-transfer operator (Meng et al., 27 Feb 2025). The system optimizes a dense planar mesh under score distillation sampling, then bakes the final displacement into a $e_s = F_s(\mathbb{I}_{s,c})$5 three-channel VDM compatible with Blender and ZBrush. Its main technical issue is “semantic coupling,” addressed by token-wise CFG-weighted blending $e_s = F_s(\mathbb{I}_{s,c})$6, with semantic enhancement weight $e_s = F_s(\mathbb{I}_{s,c})$7, SDS CFG guidance scale $e_s = F_s(\mathbb{I}_{s,c})$8, and Laplacian deformation parameter $e_s = F_s(\mathbb{I}_{s,c})$9 (Meng et al., 27 Feb 2025). In user studies, it receives $e_c \in \mathbb{R}^{512 \times 4 \times W}$0 preference for geometry quality and $e_c \in \mathbb{R}^{512 \times 4 \times W}$1 for consistency with text (Meng et al., 27 Feb 2025).

5. Text, handwriting, and robotic painting

The text and robotics literature broadens Style Brush from visual transfer to procedural style reproduction. In text generation, the key issue is that style is often holistic: it includes character shape, spacing, slant, texture, and background interaction. TextStyleBrush solves this at the word-image level with one-shot transfer from a single localized example, while DiffBrush extends it to full lines with explicit handling of intra-word and inter-word style (Krishnan et al., 2021). SE-GAN, by contrast, addresses Chinese brush handwriting font generation by making skeleton structure an explicit prior and style transfer an image-to-image translation problem with skeleton-space supervision (Yuan et al., 2022).

Robotic painting makes the term more literal still. “Artistic Style in Robotic Painting” argues that style lives not only in final image statistics but also in manual painting technique and brushstroke execution (Bidgoli et al., 2020). The system collects over $e_c \in \mathbb{R}^{512 \times 4 \times W}$2 human brushstrokes and corresponding 6-DoF brush motions at $e_c \in \mathbb{R}^{512 \times 4 \times W}$3 frames per second, stores motion as $e_c \in \mathbb{R}^{512 \times 4 \times W}$4 arrays and stroke images as $e_c \in \mathbb{R}^{512 \times 4 \times W}$5, and trains a convolutional VAE with latent dimension $e_c \in \mathbb{R}^{512 \times 4 \times W}$6, Adam, learning rate $e_c \in \mathbb{R}^{512 \times 4 \times W}$7, batch size $e_c \in \mathbb{R}^{512 \times 4 \times W}$8, and $e_c \in \mathbb{R}^{512 \times 4 \times W}$9 epochs (Bidgoli et al., 2020). In a perceptual evaluation, over $w_{s,i} = M_i(e_s)$0 of participants rated reconstructed strokes at $w_{s,i} = M_i(e_s)$1 or higher on a $w_{s,i} = M_i(e_s)$2–$w_{s,i} = M_i(e_s)$3 resemblance scale, and $w_{s,i} = M_i(e_s)$4 of participants did not identify the robotic “Portrait of Misun Lean” as robot-made in a five-image selection task (Bidgoli et al., 2020).

“Spline-FRIDA” sharpens this procedural interpretation by modeling stroke style as a learned trajectory manifold rather than a low-DOF Bézier primitive (Chen et al., 2024). It records human trajectories with motion capture, normalizes them to canonical form, resamples them to $w_{s,i} = M_i(e_s)$5 points, and learns a TrajVAE with latent dimension $w_{s,i} = M_i(e_s)$6. A separate Traj2Stroke renderer uses only $w_{s,i} = M_i(e_s)$7 learnable parameters $w_{s,i} = M_i(e_s)$8 and maps trajectory geometry to a differentiable distance-field stroke image (Chen et al., 2024). Fine-tuning on fewer than $w_{s,i} = M_i(e_s)$9 trajectories converges in less than a minute, and a survey of $97.2$0 participants preferred Spline-FRIDA over FRIDA by $97.2$1 to $97.2$2 on human-likeness, $97.2$3 to $97.2$4 on better overall, and $97.2$5 to $97.2$6 on artisticness (Chen et al., 2024).

6. Evaluation, limitations, and controversy

Evaluation across Style Brush systems is highly modality-specific. Image stylization papers typically report ArtFID, FID, LPIPS, user preference, or perceptual realism; text systems add OCR-based metrics, HWD, WIER, or Geometric Score; 3D geometric brushes use CLIP consistency, self-intersection, or human preference; robotic systems rely heavily on human studies (Feng et al., 2024). This diversity makes direct comparison across subfields inappropriate. It also means that the same phrase can hide radically different optimization targets: perceptual style similarity in one paper, semantic legibility in another, or physical stroke plausibility in a third.

Several recurring limitations appear across the literature. StyleBrush reports strong results but does not provide a conventional component-by-component ablation for ReferenceNet, Structure Guider, or cropping; its video extension is qualitative and uses AnimateDiff without extra training (Feng et al., 2024). Multi-style NeRF with AdaIN provides architectural rationale for density-aware brush handling but no strong quantitative validation for brush-stroke improvements (Pao et al., 2024). Parameterized brushstroke methods remain computationally expensive and can lose fine facial details (Meleti et al., 8 Mar 2026). Style Brush for textured meshes is slowed by $97.2$7 nearest-neighbor search and substantial memory cost due to rotated style dictionaries (Kovács et al., 3 Oct 2025). Text2VDM remains constrained by SDS-style multi-view inconsistency (Meng et al., 27 Feb 2025). DiffBrush still fails on low-frequency characters such as punctuation or Greek letters (Dai et al., 5 Aug 2025). Robotic systems capture only part of style, often emphasizing intra-stroke geometry while leaving color strategy, inter-stroke structure, or physical paint dynamics only partially modeled (Bidgoli et al., 2020).

A distinct controversy concerns the phrase itself. “The Silent Brush” uses it not to name a transfer system but to diagnose a failure mode in text-to-image generation: stylistic resurfacing without explicit prompt mention of an artwork or artist (Joshi et al., 17 May 2026). Its Art Arena protocol measures explicit imitation, pairwise artwork interaction, and proactive blending across Stable Diffusion v1.5, SDXL, and SANA-1.5. The paper argues that style leakage is uneven, interaction-dependent, and asymmetric rather than reducible to simple memorization (Joshi et al., 17 May 2026). This broadens the meaning of Style Brush from a creation tool to a governance problem: not merely how style is applied, but when it reappears without being asked for.

Taken together, the literature suggests that “Style Brush” is best understood as an umbrella concept for controllable stylization systems in which style is treated as a manipulable resource rather than a fixed global filter. The dominant technical strategies are branch-wise disentanglement of style and structure, explicit brushstroke or trajectory parameterization, direction-aware or region-aware conditioning, and multiscale or multi-view constraints. The dominant open problem is not basic stylization quality, but controllable, semantically faithful, and medium-appropriate transfer under realistic computational and governance constraints.