Trait Swapping in Phylo-Diffusion Models

• Trait swapping is a generative modeling technique that edits organism traits by swapping hierarchical embedding segments linked to phylogenetic branching events.
• It leverages the Phylo-Diffusion framework to condition a diffusion model on modified latent vectors, producing interpretable hypothetical phenotypes.
• Empirical validation shows biologically plausible trait modifications with strong quantitative metrics (FID, IS, F1) on fish and bird image datasets.

Trait swapping is a generative modeling technique that enables targeted editing of organismal traits in image synthesis, leveraging hierarchical embeddings structured according to phylogenetic trees. Developed in the context of the Phylo-Diffusion framework, trait swapping provides an interpretable generative-analogue of gene-editing and gene-swapping experiments in evolutionary biology. The methodology structurally injects the latent "trait vector" associated with evolutionary branching events from a donor species into a source species, conditioning a diffusion model to visualize the resulting hypothetical phenotype. This approach yields empirically interpretable modifications in generated images, facilitating fine-grained investigation of evolutionary innovations and trait origination across clades (Khurana et al., 2024).

1. Phylogenetic Hierarchical Embeddings (HIER-Embed)

In Phylo-Diffusion, each taxon and ancestral node in a discretized phylogenetic tree is associated with a unique, learnable embedding. For a given species $S_i$, the embedding at phylogenetic level $\ell$ is denoted $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$, where $S_i^\ell$ is the ancestor of $S_i$ at level $\ell$. The full hierarchical embedding $$E_i = \tau(S_i) = [E_i^1 \| E_i^2 \| E_i^3 \| E_i^4] \in \mathbb{R}^{d}$$ is constructed by concatenation, with $d = 4d'$. This concatenation ensures that any pair of species sharing a common ancestor up to level $k$ will have identical embeddings in the first $k$ segments. These hierarchical embeddings are treated strictly as conditioning vectors in the latent diffusion model; there is no explicit probabilistic prior over $\ell$0, aside from parameter-level priors.

2. Trait Swapping Protocol

Trait swapping targets the manipulation of trait subspaces corresponding to evolutionary features acquired at specific phylogenetic levels. The approach does not require explicit principal direction discovery or supervised trait annotation; rather, each segment $\ell$1 of the hierarchical embedding corresponds implicitly to traits that emerged at branching event $\ell$2. Given a source species $\ell$3 and a donor species $\ell$4 from the same branching level, the trait swap operation at level $\ell$5 is formalized as:

$\ell$6

or, equivalently, $\ell$7, where $\ell$8 is a block-masking operator isolating the $\ell$9-th embedding segment. This replacement substitutes only the level-$$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$0 trait vector from the donor into the source, with all other segments preserved.

3. Diffusion-Based Conditional Generation

Generation proceeds with the swapped embedding using a standard DDIM or DDPM reverse sampling process. At each time step $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$1, the conditioned U-Net denoiser is supplied with the manipulated embedding $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$2 as context:

$$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$3

The next latent is sampled via:

$$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$4

where $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$5, $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$6, $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$7, and $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$8 are the diffusion noise schedules and $$E_i^\ell = \mathrm{Embed}(S_i^\ell) \in \mathbb{R}^{d'}$$9. The training objective remains

$S_i^\ell$0

with hierarchical information supplied exclusively by the structure of $S_i^\ell$1, without added regularization.

4. Experimental Setup and Validation

Empirical evaluation of trait swapping was conducted on two biological datasets:

Dataset	Species (N)	Preprocessing
Great Lakes Fish	5,434	38 spp., 256×256 images, OpenTree-based phylogeny (4 levels)
Birds (CUB-200-2011 subset)	190	Background removed

No explicit manual annotation of individual traits was required; trait-level diversity emerges through the modeling. Plausibility of generated images was assessed by a ResNet-18 classifier trained to assign images to species (85% accuracy on real fish images; 76% on birds). Quantitative metrics include FID, Inception Score (IS), feature-space recall and precision, classification $S_i^\ell$2 scores, and alignment between embedding-space cosine distances and ground-truth phylogenetic distances.

5. Empirical Findings and Biological Interpretation

Trait swapping generates targeted, biologically plausible modifications in synthesized phenotypes:

• In fish, swapping level-2 embedding segments from Noturus exilis to the Notropis subtree removes barbels and introduces a forked caudal fin.
• Substituting level-2 traits from Gambusia affinis with those of Esox americanus results in a more pointed head and slender body, while retaining belly pigmentation of G. affinis.
• Replacing level-3 traits of Lepomis gulosus with those from the Morone genus instates a horizontal stripe pattern and dorsal-fin splitting.

These results suggest that individual trait vectors in hierarchical embeddings map coherently to phenotypic changes localized to identifiable evolutionary branching events. Trait masking further confirms that ablating level-$S_i^\ell$3 segments reduces classifier confidence more among species sharing that subtree than in those outside. In benchmarking, Phylo-Diffusion achieves image quality metrics comparable to or favorable against class-conditional baselines (FID 11.38 vs. 11.46; IS 2.53 vs. 2.47; recall 0.367 vs. 0.359) and high classification fidelity for generated images (F1 82.2%, nearly matching real-image performance at 85%). A direct comparison to Phylo-NN shows that Phylo-Diffusion yields sharper, more interpretable trait-altered images, whereas Phylo-NN perturbations remain blurry and lack substantial trait shifts (Khurana et al., 2024).

6. Current Limitations and Prospective Directions

Methodological constraints include the use of fixed, discretized phylogenetic levels, which does not fully capture real-world variability in tree depth and branching. Trait convergence arising independently in separate clades is not explicitly modeled. Enhancements could comprise continuous embeddings per tree edge or node, disentanglement losses, or multi-node trait swapping to probe convergent evolution. Ancestral state uncertainty remains unaddressed; Bayesian phylogenetic embeddings could facilitate credible-interval visualizations. Currently, phenotypic transformations are interpreted qualitatively—a plausible implication is that integrating known genetic or morphometric databases could yield automated trait annotation and enable statistical validation. Extending the approach beyond images, Phylo-Diffusion could condition on tree-of-life knowledge to model additional phenotype modalities such as 3D scans or CT volumes.

7. Significance in Evolutionary Phenotype Modeling

Trait swapping via hierarchical conditioning of diffusion models, as instantiated in Phylo-Diffusion, presents a conceptually transparent protocol for hypothesis generation regarding evolutionary innovation. It enables controlled, interpretable manipulations in the generative embedding space, uncovering putative trait origination events directly from image data. The technique offers a methodological bridge between computational evolutionary biology and modern generative models, with demonstrated capacity for producing biologically plausible trait transformations and supporting fine-grained phylogenetic analysis at scale (Khurana et al., 2024).