CraftGraffiti: Identity-Preserving Graffiti

• CraftGraffiti is an end-to-end text-guided graffiti generation framework that preserves facial identity amid extreme stylistic transformations.
• It employs a LoRA-fine-tuned diffusion transformer with a face-consistent self-attention mechanism to maintain local facial details during pose customization.
• The modular pipeline integrates style injection, CLIP-guided prompt extension, and VAE decoding to produce high-quality, identity-respectful graffiti portraits.

CraftGraffiti is an end-to-end text-guided graffiti generation framework designed around a specific technical objective: preserving facial identity under extreme stylistic transformation in graffiti, a high-contrast and abstract medium in which small distortions to the eyes, nose, or mouth can erase recognizability and compromise personal and cultural authenticity. The framework takes an input image together with a style and pose descriptive prompt, applies graffiti style transfer through a LoRA-fine-tuned pretrained diffusion transformer, and then enforces identity fidelity through a face-consistent self-attention mechanism with explicit identity embeddings; pose customization is performed without keypoints through CLIP-guided prompt extension. The system is organized around a “style-first, identity-after” paradigm that is both formally justified and empirically validated to reduce attribute drift relative to reverse ordering (Banerjee et al., 28 Aug 2025).

1. Definition and problem setting

CraftGraffiti addresses a composite generation problem that conventional text-guided stylization, pose editing, and unified editing methods do not resolve simultaneously: the system must alter pose, impose a strong graffiti style, and retain facial identity despite the abstraction induced by graffiti rendering. In the formulation reported for the framework, the central failure mode of prior approaches is that local facial details are lost because the self-attention mechanisms used in diffusion systems do not adequately stabilize identity-critical structure under large style and pose changes (Banerjee et al., 28 Aug 2025).

The framework therefore treats facial preservation not as a secondary aesthetic preference but as a primary design constraint. Its inputs are an image, a style-and-pose prompt, and a reference face from which an identity embedding is derived. Its outputs are high-quality graffiti portraits in which local facial attributes remain coherent even when the global image is re-posed and heavily stylized. The paper situates this as a problem of identity-respectful AI-assisted artistry rather than generic image stylization, with recognizability serving as a technical and cultural criterion rather than merely a perceptual convenience (Banerjee et al., 28 Aug 2025).

A common misconception is that facial identity preservation in stylization can be handled adequately by global similarity alone. The evaluation discussed for CraftGraffiti suggests otherwise: global facial embedding metrics may rank highly stylized images poorly even when local attributes remain recognizable, because strong style changes alter global features. This suggests that local attribute preservation, rather than only whole-face embedding proximity, is central to graffiti-specific portrait generation (Banerjee et al., 28 Aug 2025).

2. End-to-end pipeline and operational workflow

The CraftGraffiti pipeline is explicitly modular. First, graffiti style is injected with a LoRA-fine-tuned pretrained diffusion transformer based on FLUX.1 dev, described as a diffusion transformer or DiT-style backbone. The paper cites a dedicated graffiti style LoRA, “graffiti-style-fluxFaces.” Second, identity preservation is enforced through a face-consistent self-attention mechanism that augments attention layers with an explicit identity embedding. Third, pose customization is achieved without keypoints: T5 extends the input prompt into richer pose-descriptive text, CLIP encodes the extended prompt, and the resulting cross-attention maps act as a guided signal for pose. Finally, a VAE decoder reconstructs the graffiti portrait from denoised latent features (Banerjee et al., 28 Aug 2025).

The recommended workflow is correspondingly concrete. One prepares an input face image, ideally front-facing or in 3/4 view, and good lighting is reported to help. A style and pose prompt is then crafted, including context such as “as a DJ,” “playing a guitar,” or “singing on stage.” The system extends the prompt via T5 and encodes it with CLIP. Style injection is then applied first using the LoRA-fine-tuned FLUX.1 dev style fusion model, after which pose customization is carried out through CLIP-guided prompt extension and LoRA-updated cross-attention and self-attention. Face-consistent self-attention remains active during this stage to preserve local facial structure across pose changes. Denoising is run for 100 steps, with latent composition in the first 25 steps, followed by VAE decoding (Banerjee et al., 28 Aug 2025).

The paper also reports practical parameter settings. Guidance scale is 7.5. LoRA rank is 64, the regularization or scaling factor $\alpha$ is 128, the subject guidance factor $\lambda$ is 0.95, and the style intensity factor is 0.7. LoRA updates are applied to the attention matrices $W_q$, $W_k$, and $W_v$ in both cross-attention and self-attention. Resolution, optimizer, batch size, and diffusion schedule details are not specified, and the paper explicitly notes that identity embedding dimensionality, normalization, and exact face-region extraction or masking are not specified either (Banerjee et al., 28 Aug 2025).

The avoidance of keypoints is a distinctive design decision. Rather than relying on skeletal or landmark control, CraftGraffiti uses prompt extension and text-conditioned cross-attention to drive dynamic re-posing. A plausible implication is that the framework is intended to preserve compatibility with open-ended textual authoring, allowing pose control to remain embedded in the same natural-language interface that specifies style (Banerjee et al., 28 Aug 2025).

3. Architecture, attention mechanisms, and formal justification

The backbone is a FLUX.1 dev (12B) diffusion transformer with multi-scale latent feature processing and VAE decoding. A CLIP-based text encoder provides semantic guidance, and T5 enriches pose prompts. The principal architectural novelty is the face-consistent self-attention module, which augments self-attention by concatenating an identity embedding to the spatial tokens so that the resulting queries and keys become identity-aware (Banerjee et al., 28 Aug 2025).

The paper gives the identity-augmented formulation explicitly. For spatial token $x$ and identity embedding $id$,

$Q' = W_Q [x; id], \qquad K' = W_K [x; id]$

and attention is then computed as

$$A = \mathrm{softmax}\left(\frac{Q'K'^T}{\sqrt{d_k}}\right)V.$$

The standard self-attention baseline is likewise stated as

$$A = \mathrm{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V.$$

The identity vector is shared across spatial tokens to bias attention globally toward identity-consistent features. The paper conceptually discusses appending the identity vector as an extra token or channel, but does not specify additional identity-specific biases, multiplicative gates, or facial-token-only constraints (Banerjee et al., 28 Aug 2025).

The framework’s theoretical claim is organized around the “style-first, identity-after” paradigm. Let $\lambda$0 denote the style operator and $\lambda$1 the identity-restoring operator, with attribute encoder $\lambda$2 and attribute discrepancy

$\lambda$3

The paper argues that if $\lambda$4 for all $\lambda$5, then

$\lambda$6

whereas

$\lambda$7

in general. It follows that

$\lambda$8

Within the paper’s logic, this formalizes why applying style blending first and then identity-preserving pose customization yields lower attribute drift than the reverse sequence (Banerjee et al., 28 Aug 2025).

The diffusion preliminaries are also stated. The forward process is

$\lambda$9

and the reverse process is

$W_q$0

The paper notes minimizing the error between predicted and true noise or $W_q$1 depending on parameterization, but does not specify whether $W_q$2-prediction or $W_q$3-prediction is used (Banerjee et al., 28 Aug 2025).

For style fine-tuning, the paper reports style and content losses in feature space:

$W_q$4

$W_q$5

$W_q$6

LoRA itself is parameterized as

$W_q$7

with $W_q$8 and $W_q$9, where $W_k$0. These equations define the reported mechanism, but the paper does not specify optimizer settings for this stage, batch size, or sampler details for FLUX.1 dev in this setup (Banerjee et al., 28 Aug 2025).

4. Datasets, evaluation, and real-world deployment

Style fine-tuning uses the 17K-Graffiti dataset, while validation and pose customization use consenting images from the Computer Vision Center (CVC) Barcelona. Experiments are reported on a single NVIDIA A6000 48GB GPU. The evaluation protocol uses three principal metrics: Facial feature consistency (FFC), computed as cosine similarity of face embeddings using FaceNet (InceptionResnetV1 pretrained on VGGFace2); Aesthetic score (Aes), using the LAION aesthetic classifier; and Human Preference Score (HPS). FID and LPIPS are not reported (Banerjee et al., 28 Aug 2025).

The quantitative results distinguish identity preservation, aesthetic quality, preference, and runtime rather than reducing performance to one scalar. CraftGraffiti is reported to achieve state-of-the-art style blending and aesthetics against image editing baselines, while ranking below VLMs and IP-Adapter in FFC because global facial embedding metrics become difficult under strong style changes. The paper’s interpretation is therefore not that CraftGraffiti maximizes every metric simultaneously, but that it obtains a specific balance between local facial coherence and graffiti-specific stylization quality (Banerjee et al., 28 Aug 2025).

Model	FFC / Aes / HPS	Time
FLUX + IP-Adapter	0.8324 / 3.2414 / 0.3012	8.2 s
GPT-4o	0.8761 / 4.5193 / 0.3412	13.4 s
Ours (Baseline FLUX.1 dev)	0.7618 / 3.6913 / 0.2911	2.9 s
Ours (+ Style Fusion)	0.6814 / 4.7195 / 0.3001	5.3 s
Ours (+ Face-Consistent Self-Attention)	0.7713 / 5.1376 / 0.3176	8.7 s
CraftGraffiti (full)	0.7713 / 5.2271 / 0.3536	10.1 s

The ablation logic is central. Baseline FLUX.1 dev is described as yielding neither consistent faces nor proper graffiti style. Face-consistent self-attention before style preserves attributes during pose customization but style fails, producing cartoonish wall drawing. Style after pose customization preserves neither facial attributes nor proper graffiti style. The reported “correct pipeline,” namely style-first plus identity-aware self-attention during pose customization, produces realistic graffiti posters with preserved facial attributes (Banerjee et al., 28 Aug 2025).

The framework was also deployed at the Cruïlla Festival in Barcelona from July 9–12, 2025. The festival attendance is reported as approximately 80,000 over 4 days; the installation attracted more than 1,100 visitors to the booth; 586 posters were generated; and anonymized feedback was collected. The workflow involved on-site capture or user-provided images, graffiti-style portrait generation with pose customization, informed consent, and operator guidance. Feedback indicated the installation was fun and engaging, but also surfaced concerns about data storage, which were addressed by not storing data beyond quality control and only with permission. The deployment also revealed biases, including heavy makeup on women, occasional misgendering, younger-looking outputs, and stylized body shapes (Banerjee et al., 28 Aug 2025).

5. Position within the broader CraftGraffiti research landscape

Within the supplied literature, “CraftGraffiti” can denote more than a single software artifact. In the narrower sense, it is the facial-preserving diffusion framework just described. In a broader research sense, it suggests a family of graffiti-oriented computational systems spanning portrait generation, embodied AR authoring, artistic typography synthesis, and graffiti recognition. This broader interpretation is not presented as a unified formal program, but it is strongly suggested by the juxtaposition of the cited works (Banerjee et al., 28 Aug 2025).

At the level of embodied creation, GestoBrush reframes the smartphone as a virtual spray can and lets graffiti artists create in situ using whole-body gestures, with a workflow that begins by scanning and registering a wall as a 2D canvas and then extends strokes into surrounding 3D space. A co-design workshop with five graffiti artists emphasized embodiment and communication with space, and a field study with six artists in Tianjin reported increased intuitiveness, immersion, and expressiveness, alongside technical limits such as drift outdoors in bright light or low-texture scenes (Chen et al., 6 Sep 2025). This suggests that a broader CraftGraffiti ecology includes not only generative portrait synthesis but also artist-centered embodied interfaces that preserve graffiti’s relation to place.

At the level of typography and localized visual control, WordCraft introduces an interactive artistic typography system based on FLUX.1, a training-free regional attention mechanism, and noise blending for continuous refinement. It supports precise multi-region generation, iterative editing, multi-character composition, and LLM-based prompt parsing into structured global and regional directives. The mechanism is organized around regional masks $W_k$1 and masked multi-modal attention, enabling localized effects such as drips, splatters, glows, shadows, and region-specific stylistic variation without additional training (Wang et al., 13 Jul 2025). A plausible implication is that WordCraft provides a technical substrate for graffiti word design complementary to CraftGraffiti’s face-preserving portrait generation.

A further adjacent line concerns recognition rather than synthesis. The CGCL study assembled an open dataset of more than 4000 images for 34 types of historical carved letters from the walls of St. Sophia Cathedral in Kyiv and showed that carved letters are difficult to separate by dimensionality reduction such as t-SNE because of erosion, shallow relief, and irregular representation. Multinomial logistic regression obtained area-under-curve values not lower than about 0.60 on CGCL, while a compact 2D CNN with high lossy data augmentation achieved AUC values close to 0.99 on the A/H task and, in the paper’s broader wording, “close to 0.99” for both CGCL and notMNIST under strong augmentation (Gordienko et al., 2018). This suggests that the CraftGraffiti domain also includes the machine reading of graffiti-like marks and historical inscriptions, especially where glyph degradation and environmental variability are central.

Taken together, these works delimit a technical field with at least four axes: identity-preserving portrait stylization, embodied AR graffiti authoring, region-aware graffiti typography generation, and graffiti or inscription recognition. The overlap is methodological rather than merely thematic: diffusion transformers, attention control, AR tracking and anchoring, and augmentation-heavy visual learning all appear as operative components across the landscape (Banerjee et al., 28 Aug 2025).

6. Limitations, ethical questions, and future directions

The CraftGraffiti paper identifies several unresolved technical limitations. Identity embedding specifics are not fully specified: the face encoder used to generate the identity embedding during training or inference is not named, the embedding dimensionality and normalization are omitted, and the layer-wise injection points and face-region restrictions are not described. Training objective details such as optimizer, batch size, exact diffusion parameterization, schedules, and possible additional pose or perceptual losses are likewise unspecified. These omissions matter for strict reproducibility even though the conceptual pipeline is clear (Banerjee et al., 28 Aug 2025).

Failure modes are also documented. Over-stylization can alter global features, making outputs appear less consistent under global embedding metrics. Some results may appear similar, reducing perceived uniqueness. Real-time generation at public events is resource-intensive, and environmental conditions such as lighting and noise affect results. The festival deployment exposed bias phenomena, including heavy makeup tendencies on women, occasional misgendering, a tendency to make people look younger, and stylized body shapes. The paper frames mitigation in terms of curated data, fairness-aware training, and engagement with affected communities (Banerjee et al., 28 Aug 2025).

The ethical dimension extends beyond bias. The work emphasizes that graffiti is tied to marginalized voices and that cultural authenticity requires respect for cultural motifs and avoidance of homogenizing aesthetics. At Cruïlla, informed consent and transparency on data handling were part of the deployment protocol, and no storage of user data beyond permitted quality control was reported. This suggests that identity-preserving graffiti generation is not only an image synthesis problem but also a problem of consent, representation, and cultural stewardship (Banerjee et al., 28 Aug 2025).

Related systems reveal parallel constraints. GestoBrush reports that limitless virtual space can dilute contextual relevance because meaning is often site-specific, and it notes that legal and ethical considerations persist around public authoring in AR (Chen et al., 6 Sep 2025). WordCraft identifies failure cases in extremely dense wildstyle or very fine-grained directives, where the backbone’s semantic granularity can be exceeded, producing inconsistencies; it recommends coarse-to-fine iterations and limiting concurrent regions (Wang et al., 13 Jul 2025). The CGCL work, by contrast, emphasizes small datasets, modest and uneven per-class counts, weathering, lighting, and weak class neighborhoods in raw appearance space as ongoing barriers for graffiti recognition (Gordienko et al., 2018).

The forward-looking agenda stated or implied across the papers includes extension beyond graffiti to other cultural art forms, real-time bias detection and mitigation, broader ecological evaluation, cross-cultural studies on identity preservation and style adaptation, multi-user and persistent AR graffiti, and richer technical grounding for typographic and glyph-level graffiti systems. A plausible implication is that “CraftGraffiti” names not only a single framework but an emerging research domain in which identity, embodiment, place, and stylization are treated as coupled variables rather than separate design targets (Banerjee et al., 28 Aug 2025).