--- title: 'AnimalBooth: Personalized Animal Image Generation' url: https://www.emergentmind.com/topics/animalbooth type: topic --- # AnimalBooth: Personalized Animal Image Generation Searching arXiv for the cited AnimalBooth and related animal-avatar papers to ground the article in current preprints. arXiv search query: `AnimalBooth multimodal feature enhancement for animal subject personalization` AnimalBooth is a framework for personalized animal image generation that addresses rich appearance cues, large morphological variability, and identity drift in encoder-based, tuning-free methods. It strengthens identity preservation with an Animal Net and an adaptive attention module, and it introduces a frequency controlled feature integration module that applies Discrete Cosine Transform filtering in the latent space to guide the diffusion process from coarse structure to detailed texture. The framework is paired with AnimalBench, a high-resolution dataset for animal personalization, and is evaluated against BLIP-Diffusion, Omnigen, and IP-Adapter on LPIPS, DINO, CLIP-T, and CLIP-I metrics [2509.16702]. ## 1. Problem setting and scope AnimalBooth is situated in the problem of animal subject personalization from a single reference image. The paper characterizes this setting as more challenging than personalization for rigid objects or human faces for three reasons: animals exhibit subtle fur, feather or scale textures and fine-grained patterns; even within a single species, pose, lighting and viewpoint can dramatically change appearance; and existing encoder-based, tuning-free methods suffer from cross-domain feature misalignment, causing the generated subject to forget distinctive identity cues and drift toward the generic diffusion prior [2509.16702]. Within this formulation, identity preservation is treated as the central failure mode. The framework therefore focuses on extracting and reinjecting subject-specific information without discarding the generative prior of the base diffusion model. The architectural consequence is a separation between a trainable identity branch and a largely frozen image-generation backbone. This design is consistent with the paper’s emphasis on mitigating cross-domain alignment errors while maintaining perceptual quality [2509.16702]. ## 2. Animal-Net and multimodal identity representation AnimalBooth introduces a small, trainable “Animal-Net” branch whose sole purpose is to extract multi-modal identity features from a single reference image. The input modalities are the reference image $\mathcal{X}_a\in\mathbb{R}^{3\times H\times W}$ and the implicit text prompt $\mathcal{C}_t$ fed into the frozen Photography-Net. The reference image is first encoded by the stable-diffusion VAE encoder into a latent map $L_a\in\mathbb{R}^{4\times(H/8)\times(W/8)}$, and a frozen CLIP image encoder produces $K$ image-token embeddings. A small trainable Q-Former, adopted from BLIP-2, maps CLIP tokens into a fixed set of query vectors, after which several transformer blocks fuse $L_a$ as key/value with the Q-Former queries as query [2509.16702]. The output of this branch is the feature set $\mathfrak{F}_a$, later injected into the main Photography-Net via adaptive attention. The paper states that these fused features encode both coarse structure, such as silhouette and limb positions, and high-frequency detail, such as fur stripes, and remain fixed during diffusion inference; Animal-Net is never noised or finetuned at test time [2509.16702]. This representation strategy is important because it explicitly combines latent spatial structure with CLIP-derived semantic content. A plausible implication is that Animal-Net is intended to preserve both global morphology and appearance-specific cues under prompt-conditioned synthesis, rather than relying on a single feature space to carry all identity information. ## 3. Adaptive attention and identity injection To inject $\mathfrak{F}_a$ into the frozen UNet, referred to as “Photography-Net,” each self-attention block is replaced by an adaptive attention block that combines a frozen self-attention sub-layer and a trainable cross-attention sub-layer. Let $Z_n\in\mathbb{R}^{m\times d}$ denote the features from the UNet at a given layer and $\mathfrak{F}_a\in\mathbb{R}^{k\times d}$ the identity tokens from Animal-Net. The queries, keys, and values are formed as $$ Q=Z_nW_q,\qquad K=Z_nW_k,\qquad V=Z_nW_v, $$ with identity-conditioned terms $$ K_{ID}=\mathfrak{F}_aW_{k_{ID}},\qquad V_{ID}=\mathfrak{F}_aW_{v_{ID}}. $$ The adaptive attention output is $$ \mathbf{O} = \underbrace{\mathrm{Softmax}\!\left(\tfrac{QK^T}{\sqrt{d}}\right)V}_{\text{Self-Attention (frozen)}} + \lambda\, \underbrace{\mathrm{Softmax}\!\left(\tfrac{QK_{ID}^T}{\sqrt{d}}\right)V_{ID}}_{\text{Cross-Attention (trainable)}}. $$ Here $\lambda\in[0,1]$ balances identity injection versus the generic prior; the reported best setting is $\lambda=0.4$ [2509.16702]. The mechanism is architecturally conservative. The frozen self-attention preserves the generative prior of the base model, whereas the trainable cross-attention injects subject identity. This suggests that AnimalBooth is designed to reduce identity drift without fully re-optimizing the generative backbone. In the paper’s terminology, the adaptive attention module is the point at which multi-modal identity features are made operational inside the denoising network [2509.16702]. ## 4. Frequency-controlled feature integration AnimalBooth further introduces a frequency controlled feature integration module that applies Discrete Cosine Transform filtering in latent space and feeds back the filtered signal during diffusion. For the $n$-th channel of the clean VAE latent, $\mathbf{L}_0^{(n)}\in\mathbb{R}^{h\times w}$, the 2D DCT is defined as $$ F_{\mathrm{DCT},u,v}^{(n)} = \frac{2}{\sqrt{h\,w}\,m(u)\,m(v)} \sum_{i=0}^{h-1}\sum_{j=0}^{w-1} \bigl(\mathbf{L}_0^{(n)}\bigr)_{i,j} \cos\!\Bigl(\tfrac{(2i+1)u\pi}{2h}\Bigr) \cos\!\Bigl(\tfrac{(2j+1)v\pi}{2w}\Bigr), $$ where $m(0)=1/\sqrt{2}$ and $m(\gamma)=1$ for $\gamma>0$ [2509.16702]. The framework defines four binary masks in the $(u,v)$ plane. The mini mask is active when $u+v\le 10$, the low mask when $u+v\le 20$, the mid mask when $20