FantasyStyle: Diffusion-Driven 3D Style Transfer

• FantasyStyle is a 3D style transfer method relying on Gaussian splatting and diffusion distillation to preserve scene structure while applying artistic styles.
• It employs Multi-View Frequency Consistency to suppress low-frequency inconsistencies and Controllable Stylized Distillation to mitigate content leakage.
• The framework outperforms traditional VGG-based methods by enhancing view consistency and texture detail, impacting 3D graphics, avatar stylization, and more.

FantasyStyle denotes, in its strictest published sense, a 3D Gaussian Splatting (3DGS) style transfer framework for transferring the artistic style of a reference image to an entire 3D scene while preserving scene structure and maintaining consistent appearance across views. The framework was introduced to address two recurrent failures of prior 3D style transfer systems: multi-view inconsistency, which produces style conflicts, smoothing, and distortion, and content leakage from style images, which causes unintended semantics to appear in the target scene. More broadly, contemporaneous work suggests a wider research program in which “fantasy-style” behavior is treated as a controllable latent property across 3D graphics, avatar stylization, text-to-image personalization, literary pastiche, dialogue generation, fashion semantics, and user perception (Yang et al., 11 Aug 2025, Liu et al., 2023, Nguyen-Phuoc et al., 2023, Park et al., 2024, Han et al., 21 Feb 2025, Gao et al., 2019, Banerjee et al., 2020).

1. Core formulation in 3D Gaussian Splatting

FantasyStyle is formulated on top of standard 3DGS reconstruction. The reconstruction objective is written as

$$\min_\Theta \frac{1}{N}\sum_{i=1}^N \mathcal{L}_{L1+SSIM}(\mathcal{R}(C_i;\Theta), V_i^{gt}),$$

where $C_i$ is the $i$-th camera, $V_i^{gt}$ is the ground-truth image for that view, $\mathcal{R}(\cdot)$ is differentiable rasterization, and $\Theta$ denotes the 3D Gaussian scene parameters. For style transfer, geometry is kept fixed and only the color-related parameters are optimized:

$$\min_{\Theta_c} \frac{1}{N}\sum_{i=1}^N \mathcal{L}(\mathcal{R}(C_i;\Theta_c); P, I_{style}),$$

with $P$ the text prompt describing the original scene and $I_{style}$ the style reference image. The operative principle is therefore preserve geometry, modify appearance (Yang et al., 11 Aug 2025).

The method is explicitly positioned against VGG-feature-based 3DGS stylization methods such as StyleGaussian and SGSST. The criticism is twofold. First, VGG features are described as weak at disentangling style from content in a style image, which encourages content leakage. Second, such methods can overemphasize low-level texture statistics, leading to over-stylization and weak content preservation. FantasyStyle responds by becoming the first 3DGS style transfer method to rely entirely on diffusion-model distillation rather than VGG-feature matching (Yang et al., 11 Aug 2025).

A related but architecturally distinct line appears in zero-shot NeRF stylization. StyleRF represents scenes with an explicit 3D feature grid, performs style transformation in radiance-field feature space, and uses sampling-invariant content transformation together with deferred style transformation of 2D feature maps to maintain multi-view consistency while generalizing to arbitrary new styles (Liu et al., 2023). This establishes a useful contrast: FantasyStyle works in 3DGS with diffusion distillation, whereas StyleRF works in NeRF-like radiance fields with feature-space style transfer.

2. Multi-View Frequency Consistency

The first core module of FantasyStyle is Multi-View Frequency Consistency (MVFC), introduced to reduce style conflicts across rendered views. The paper’s frequency analysis of multi-view stylized priors reports that low-frequency components mainly encode view-dependent local structure and are often inconsistent across views, whereas high-frequency components better capture stable texture cues and are more consistent. On this basis, MVFC suppresses low-frequency components selectively while preserving high-frequency detail (Yang et al., 11 Aug 2025).

Starting from $N$ rendered views, DDIM noise is added to the latent:

$C_i$0

The multi-view latent is then decomposed by a 3D Fourier transform over spatial and batch dimensions:

$C_i$1

where $C_i$2 is a high-pass filter and $C_i$3 is elementwise multiplication. The ablation described in the paper indicates that reducing low-frequency components slightly weakens local detail but greatly improves cross-view consistency, whereas reducing high-frequency components destroys brushstroke texture and yields blur (Yang et al., 11 Aug 2025).

The final MVFC construction replaces part of the low-frequency band with shared low-frequency noise:

$C_i$4

$C_i$5

The significance of this design is not merely denoising. It acts as a cross-view regularizer in frequency space: the inconsistent part of stylized priors is treated as predominantly low-frequency, and replacing part of that band with shared noise encourages different views to agree while retaining high-frequency texture richness (Yang et al., 11 Aug 2025).

3. Controllable Stylized Distillation

The second core module is Controllable Stylized Distillation (CSD), which replaces VGG-style supervision with a diffusion-distilled objective. Style injection uses SDXL as the base diffusion model, IP-Adapter for style injection, and ControlNet for structural guidance. The distinctive step is to replace the unconditional branch in classifier-free guidance with a negative prompt derived from the content information of the style image, thereby suppressing content leakage (Yang et al., 11 Aug 2025).

Using IP-Adapter-Instruct, the method extracts $C_i$6 for style features and $C_i$7 for content features from the same reference image. The target stylization signal is defined as

$C_i$8

while the source reference signal is

$C_i$9

The final gradient becomes

$i$0

Here controllability comes from the guidance scale $i$1, the negative content embedding, and the fact that only color-related 3DGS parameters are updated (Yang et al., 11 Aug 2025).

A central theoretical claim of FantasyStyle is that the reconstruction term embedded in SDS- and DDS-style formulations is a bottleneck for 3D style transfer. Because the task does not seek faithful reconstruction of the original source image and only fine-tunes style-related color parameters, the paper argues that reconstruction acts as an overly strong constraint, producing overly smooth outputs, losing brushstroke detail, and slowing optimization. CSD therefore removes the reconstruction term entirely (Yang et al., 11 Aug 2025).

This design choice parallels a broader shift in style guidance away from fixed perceptual losses. In AlteredAvatar, for example, supervision also comes from CLIP rather than VGG, and the objective combines a CLIP style term with a self-similarity-based identity term to preserve person identity while adapting a dynamic 3D avatar to text prompts, reference images, or both (Nguyen-Phuoc et al., 2023). The common pattern is the replacement of classical texture-statistic matching with semantically richer guidance signals.

4. Related stylization paradigms

The technical landscape surrounding FantasyStyle includes several distinct stylization paradigms, each defined by a different representation, adaptation mechanism, and notion of style input.

Work	Representation	Key mechanism
FantasyStyle (Yang et al., 11 Aug 2025)	3D Gaussian Splatting	MVFC and CSD with diffusion-model distillation
StyleRF (Liu et al., 2023)	NeRF-like scene representation	sampling-invariant content transformation; deferred style transformation
AlteredAvatar (Nguyen-Phuoc et al., 2023)	Instant Avatar dynamic 3D head	Reptile-style meta-learning; CLIP-guided fast adaptation
StyleForge (Park et al., 2024)	Stable Diffusion v1.5 personalization	dual binding with StyleRef and Aux images

AlteredAvatar addresses a different but adjacent problem: fast stylization of dynamic 3D avatars. It fine-tunes only the bias mappers inside the Identity Encoder while freezing the rest of the avatar network, and it learns a meta-initialization that adapts to a novel style in a small number of update steps. After meta-training, the main experiments use 200 adaptation steps at test time; the reported runtime is under two minutes for adaptation on a V100 GPU, with meta-training taking about 16 hours per avatar. The system accepts text, image, or mixed style guidance and is reported to preserve identity, expression, and eye gaze better than compared methods (Nguyen-Phuoc et al., 2023).

StyleForge targets personalized text-to-image synthesis for arbitrary artistic styles. Single-StyleForge uses approximately 15 to 20 StyleRef images and, in implementation, 20 StyleRef images plus 20 Aux images to bind a unique token to a target style. Multi-StyleForge extends this by binding multiple tokens to partial style attributes, such as people-related and background-related style components. The reported trade-off is that Single-StyleForge achieves the best FID and KID, whereas Multi-StyleForge achieves the best CLIP alignment across the six tested artistic styles (Park et al., 2024).

Taken together, these systems suggest that FantasyStyle belongs to a family of methods that reject one-shot style matching in favor of representation-aware control: frequency-space control in 3DGS, feature-space control in radiance fields, meta-learned adaptation in avatars, and token binding in diffusion personalization.

5. Style as latent control in text and fashion

Outside 3D graphics, fantasy-style modeling appears as a latent control problem in long-form text generation. WriterAgent introduces Pastiche Novel Generation, a task in which generated novels must imitate a target author’s recognizable style while remaining narratively coherent. Its curriculum proceeds through language style learning, world-building or character modeling, plot structure learning, and stylish writing, and it is implemented with WriterLoRA, which combines a shared stylistic foundation with hierarchical task-specific modules. Evaluated on Harry Potter and Dream of the Red Chamber, the system is reported to outperform baselines on ROUGE, style metrics, plot metrics, and human preference; on Dream of the Red Chamber, two PhD annotators preferred WriterAgent outputs in 73.1% of cases with $i$2 (Han et al., 21 Feb 2025).

StyleFusion studies style-controlled response generation without parallel style-dialogue pairs. It aligns a sequence-to-sequence dialogue model and a style autoencoder in a shared structured latent space, then samples near the conversational latent prediction. The normalized sampling radius $i$3 acts as a continuous control over style intensity: smaller $i$4 remains close to the plain conversational response, while larger $i$5 produces outputs that are more style-like. The method is demonstrated with Reddit dialogue and two target styles, arXiv prose and Sherlock Holmes novels, and is reported to achieve the best appropriateness-style trade-off among non-human systems in the paper’s human evaluation (Gao et al., 2019).

In fashion, the same high-level problem is posed as transfer from observable low-level structure to harder-to-annotate stylistic semantics. Attr2Style trains an attention-based ResNet101-LSTM captioning system first on 20,000 apparel images with attribute-based captions and then fine-tunes it on 2,500 images with style-based captions, evaluating on a 430-image test set. Style is inferred from generated captions via keyword matching, and the reported performance is BLEU $i$6 versus $i$7 for the baseline and accuracy $i$8 versus $i$9 (Banerjee et al., 2020).

SATCOGen extends style conditioning from captioning to outfit generation. Its Variational Style Encoder Network (VSEN) learns a smooth latent style space for outfits, and its Style-Compatibility-Attention Network (SCA Net) conditions compatibility on item image, category, target category, and style vector. Generation takes an anchor item, a style or theme, and an outfit template, then uses beam search to produce compatible outfits. On Zalando, the reported results substantially exceed Theme Matters on FITB accuracy and compatibility AUC. The paper does not define a literal FantasyStyle label, but it states that the framework is general enough to support arbitrary style or theme conditions if such labels are represented in the data or latent space (Banerjee et al., 2022).

6. Perception, taxonomy, and limits of fantasy style

Fantasy style is not only a generation target; it is also a perceptual and taxonomic variable. In a single-blind repeated-measures within-subject survey using Magic: The Gathering cards, visual style was operationalized as “cute and harmless” versus “heroic and mighty.” The study found no statistically significant overall difference in perceived card strength across style conditions: mean perceived strength was 4.10 for heroic, 4.07 for no image, and 3.92 for cute; Cohen’s $V_i^{gt}$0; the Mann–Whitney $V_i^{gt}$1 test yielded $V_i^{gt}$2. Yet card-level and participant-level analyses were more nuanced. About half the cards showed small style effects, one card showed significant differences with the no-image version rated stronger than both illustrated versions, and participant bias was normally distributed around a neutral center, indicating that some participants tended to rate cute cards as stronger while others favored heroic cards (Kallabis et al., 23 Jun 2025).

A different kind of style taxonomy emerges from network analysis of Goodreads reader behavior. Using reader and enjoyment networks, community detection, and principal component analysis, the study concludes that variance in fiction communities is best explained by two broad dimensions: maturity or childishness and realism or fantastical nature. Fantasy becomes more legible in the enjoyment network than in the reader network, and the authors propose the maturity-realism plane as a coarse classification tool for stories. In this formulation, fantasy is not a single fixed category but a region in preference space, interacting with both fantasticality and maturity (Sakal et al., 2023).

These findings suggest that fantasy-style labels are neither universally interpreted nor purely formal. Their meaning depends on the relation between representation, audience expectations, and task definition. This is consistent with methodological limits elsewhere in the literature. AlteredAvatar currently performs global stylization rather than local edits, cannot reliably model accessories such as hats, horns, or eye patches, may over-exaggerate shading because shading and albedo are not fully disentangled, and can slightly dampen expressions (Nguyen-Phuoc et al., 2023). In related fashion work, style guidance is only as expressive as the annotation scheme or latent pooling strategy that defines the style label in the first place (Banerjee et al., 2022).

FantasyStyle, understood across these literatures, is therefore best treated as a family of controllable style representations rather than a single aesthetic recipe. In 3DGS it is a diffusion-distilled optimization framework; in avatars it is fast adaptation; in image synthesis it is token binding; in narrative systems it is curriculum-based pastiche; in fashion it is latent style conditioning; and in perception studies it is an empirical variable whose effects are often context-dependent rather than universal.