ExpressEdit: Fast Editing of Stylized Facial Expressions with Diffusion Models in Photoshop

Abstract: Facial expressions of characters are a vital component of visual storytelling. While current AI image editing models hold promise for assisting artists in the task of stylized expression editing, these models introduce global noise and pixel drift into the edited image, preventing the integration of these models into professional image editing software and workflows. To bridge this gap, we introduce ExpressEdit, a fully open-source Photoshop plugin that is free from common artifacts of proprietary image editing models and robustly synergizes with native Photoshop operations such as Liquify. ExpressEdit seamlessly edits an expression within 3 seconds on a single consumer-grade GPU, significantly faster than popular proprietary models. Moreover, to support the generation of diverse expressions according to different narrative needs, we compile a comprehensive expression database of 135 expression tags enriched with example stories and images designed for retrieval-augmented generation. We open source the code and dataset to facilitate future research and artistic exploration.

• The paper presents an innovative Photoshop plugin that delivers rapid, artifact-free facial expression editing using diffusion models.
• It integrates a retrieval-augmented prompt generator with native Photoshop controls to effectively eliminate global artifacts.
• The system achieves sub-3-second edit latencies and supports high-resolution, multi-layered workflows for professional art pipelines.

ExpressEdit: Diffusion-Based, Artifact-Free Stylized Facial Expression Editing in Professional Workflows

Introduction

ExpressEdit introduces an open-source Photoshop plugin for artifact-free, rapid editing of stylized facial expressions using diffusion models. Addressing persistent challenges in the artistic and animation production pipeline, the system focuses on eliminating global noise, pixel drift, and workflow incompatibilities that are pervasive in proprietary tools. ExpressEdit further offers a retrieval-augmented prompt interface and a curated expression tag database, facilitating both novice and professional users in detailed artistic control without incurring high latency or requiring advanced technical knowledge.

System Design: Retrieval-Augmented Prompting and Tight Photoshop Integration

The ExpressEdit workflow is fundamentally divided between a retrieval-augmented prompt generator and a targeted expression editor. The prompt generator translates user intent—often spanning casual stories or nuanced instructions—into terse, semantically precise expression tags via visual-LLM (VLM) retrieval, subsequently forming hybrid prompts through templating.

Figure 1: The retrieval-augmented prompt generator in ExpressEdit; only the expression tag is VLM-generated, while templates sandwich the generated tag for robust, user-friendly prompting.

The underlying knowledge base is constructed from 135 curated expression tags, with extensive multi-lingual support, example images, stories, and fine-grained editability flags. These are designed to comprehensively encompass both standard and idiosyncratic expressions relevant to visual storytelling. Tags are mapped from Danbooru and Pixiv standards, and transformation requirements (e.g., those not accomplishable by prompt alone) are explicitly flagged to guide workflow.

Photoshop integration leverages native operations (e.g., Liquify, selection tools, scaling), allowing for fine spatial control, seamless layer management, and the direct application of diffusion-based edits to specified image regions. This addresses core issues in current AI editing workflows, where lack of tight integration results in undesirable global artifacts or disruptive compositional changes.

Clean, Non-Destructive Regional Editing

A central claim—demonstrated quantitatively and visually—is ExpressEdit's ability to restrict changes strictly to the intended, user-selected region, eliminating the propagation of destructive noise and preventing watermark injection that is endemic to recent proprietary image editing tools. Baseline comparisons with methods such as Nano Banana Pro and GPT-based editors reveal that these methods introduce global artifacts, edge mismatches, and watermark patterns even when restricted selections are provided, with cascading degradation in multi-step or high-resolution workflows.

Figure 2: ExpressEdit\ introduces strictly no noise outside the edited region; the magenta mask indicates the user-specified facial region.

Figure 3: Competing Photoshop integrations (e.g., Nano Banana Pro) and naive inpainting induce artifacts that are not confineable via selection, in contrast to ExpressEdit, which maintains artifact-free borders.

Figure 4: ExpressEdit\ supports high-resolution, detail-preserved editing on demanding inputs, outperforming baseline models both in contour sharpness and color fidelity.

This design decision, linking native Photoshop selection mechanisms directly to the spatially controllable denoising of the diffusion backend, results in remarkable containment of edit locality and immunity to noise accumulation. In scenarios with iterative edits or overlapping selections, remaining negligible boundary artifacts are easily repaired in a single pass, underscoring robustness for professional multi-step workflows.

Precision and Responsiveness in Expression Editing

ExpressEdit differentiates itself by enabling highly responsive, granulable edits, including those requiring explicit geometric or colorimetric transformations. Unlike prompt-only systems, which fail with precise quantitative instructions (e.g., “reduce iris diameter by 50%”), ExpressEdit allows users to physically manipulate relevant regions (selection, scale, liquify), conveying intent via direct pixel transformations. The diffusion backend, informed by the RGB-space modification, reconstructs semantically coherent, photorealistic results conforming to these precise cues in a single inference step.

Figure 5: ExpressEdit\ accurately executes a geometric instruction—reducing iris size by 50%—using minimal transformations as hints; all baseline models fail to follow this directive.

This procedural hybridization sidesteps critical limitations in compositional and spatial understanding inherent to current text-to-image diffusion models. Moreover, it obviates the reliance on prompt engineering or semantic sliders, and is robust to a broad spectrum of expression editing and artifact-removal tasks.

Synergistic Integration with Manual and Automated Editing Tools

ExpressEdit is natively engineered to synergize with complex manual edits—most notably Photoshop’s Liquify tool—enabling users to quickly rough in structural changes and then rely on the backend to correct deformation artifacts naturally. This supports a workflow where even substantial distortions or artifacts from manual edits (such as warped bows, irises, or facial features) are iteratively repaired to natural results without manual retouching.

Figure 6: Liquify-induced deformations are rapidly repaired by ExpressEdit, restoring natural geometry in seconds and eliminating the need for painstaking manual corrections.

The high adaptability is further evidenced by ExpressEdit’s ability to address non-expression-centric corrections, such as accessory restoration or color pattern fixes introduced by generative models or complex narrative requirements.

Figure 7: ExpressEdit\ rectifies complex design inconsistencies in character details, such as miscolored or missing accessories, guided solely by simple pixel-level sketches.

Computational Efficiency: Speed-Up LoRAs and Deployment

ExpressEdit offers competitive computational efficiency, achieving $<$3 second edit latencies on consumer GPUs (RTX 4090, 24GB VRAM), and supports further reduction to $\sim$2.2 seconds via speed-optimized LoRA modules with minimal loss of render quality. This is in stark contrast with proprietary cloud-based APIs or open-source models that can require $40$--$50$ seconds per edit. The system retains full support for high-resolution (2K+) and complex multi-layered projects typical in professional animation pipelines.

Figure 8: Lightning LoRA integration reduces inference latency by 46% with only minor detail trade-offs—demonstrating ExpressEdit’s suitability for rapid, iterative workflows.

Theoretical and Practical Implications, Future Prospects

The practical implication of ExpressEdit is the democratization of high-fidelity, expression-centric image editing in professional art pipelines without relying on proprietary black-box models or surrendering fine-grained control. Theoretically, the work illustrates that hybrid workflows—in which generative diffusion models and direct, interface-native user input jointly inform the editing process—surpass purely generative or prompt-only approaches in both user experience and artifact containment.

This paradigm opens potential research directions in tight co-design of creative tool interfaces and generative model pipelines, such as further improving feedback loops between feature-level manipulations and diffusion-based synthesis, or developing even more expressive prompt-to-edit mappings supported by larger, richer expression lexicons and retrieval architectures.

Conclusion

ExpressEdit delivers artifact-free, rapid stylized facial expression editing tightly integrated with professional Photoshop workflows, leveraging a curated tag database and advanced RAG-driven prompting. By directly addressing spatial control, artifact containment, and computational latency, it establishes a robust new standard for AI-powered image editing tools in creative production, with immediate impacts on digital art, animation, and broader generative visual storytelling (2604.03448).