WarpedGANSpace: Nonlinear Latent Editing

• WarpedGANSpace is an unsupervised framework that discovers non-linear, semantically meaningful trajectories in GAN latent space using RBF warping functions.
• It employs smooth, spatially adaptive scalar fields whose gradients enable expressive and disentangled semantic edits.
• Empirical results on various GAN architectures show superior semantic editing and reduced attribute entanglement compared to linear latent traversal methods.

WarpedGANSpace is an unsupervised, model-agnostic framework for discovering families of non-linear, semantically meaningful trajectories in the latent space of pretrained generative adversarial networks (GANs) via learnable radial basis function (RBF) warpings. Unlike prior approaches restricted to global, linear directions, WarpedGANSpace equips the latent space with smooth, spatially adaptive scalar fields whose gradients define locally non-linear vector fields, enabling more expressive, disentangled, and interpretable edits. The method is trained so that distinct latent traversals yield maximally distinguishable transformations in the output space, and demonstrates superior semantic editing capabilities compared to state-of-the-art linear methods (Tzelepis et al., 2021).

1. Non-linear Warping Functions in Latent Space

The latent space $\mathcal{Z} \subseteq \mathbb{R}^d$ is endowed with $K$ smooth scalar “warping” functions $f^k : \mathbb{R}^d \rightarrow \mathbb{R}$, $k=1, \ldots, K$, each parameterized as a weighted sum of $N$ Gaussian RBFs: $$f^k(z) = \sum_{i=1}^N \alpha^k_i\, \exp\left(-\gamma^k_i \|z - s^k_i\|^2\right),$$ where $s^k_i \in \mathbb{R}^d$ are centers, $\alpha^k_i \in \mathbb{R}$ are weights, and $\gamma^k_i > 0$ control width. The gradient

$$\nabla f^k(z) = -2\sum_{i=1}^N \alpha^k_i\,\gamma^k_i\,\exp\left(-\gamma^k_i \|z - s^k_i\|^2\right) (z - s^k_i)$$

specifies a non-linear vector field over $\mathcal{Z}$ for each warping.

Sampling a trajectory consists of initializing $z_0 \sim \mathcal{N}(0, I_d)$ and iteratively performing

$$z_{t+1} = z_t + \epsilon_k \frac{\nabla f^k(z_t)}{\|\nabla f^k(z_t)\|},$$

with step size $\epsilon_k$, yielding a smooth curve whose image sequence $G(z_t)$ under the pretrained GAN generator $G$ typically controls one or two high-level semantics in the output space (e.g., facial expression, pose, zoom).

2. Optimization and Training Regime

WarpedGANSpace jointly optimizes the parameters $\{s^k_i, \alpha^k_i, \gamma^k_i\}$ of each RBF warping and the parameters $\theta_R$ of a reconstructor network $R$. For each training iteration, a latent code $z$, a warp index $k$, and a signed step magnitude $\epsilon_k$ are sampled; images $x_1 = G(z)$ and $x_2 = G(z + \delta z)$ are generated, with $$\delta z = \epsilon_k\, \nabla f^k(z)/\|\nabla f^k(z)\|$$. The reconstructor $R$ predicts both the warp index and step magnitude from the image pair. The objective is the sum of cross-entropy (classification) loss for $k$ and $\ell_1$ (regression) loss for $\epsilon_k$: $$\min \; \mathbb{E}_{z, k, \epsilon_k} \left[ \mathcal{L}_\text{cls}(k,\tilde k) + \lambda\, \mathcal{L}_\text{reg}(\epsilon_k, \tilde\epsilon) \right],$$ where $\lambda$ balances classification and regression losses.

The training paradigm incentivizes the RBF fields to induce non-linear latent curves that control distinct, easily recognizable semantic factors, as distinguished by $R$.

3. Relation to Linear Latent Space Traversals

Linear latent manipulation models such as Voynov & Babenko's method and PCA-based GANSpace consider fixed, global directions in $\mathcal{Z}$. WarpedGANSpace recovers linear paths as a special case: with exactly two RBFs per warping (centers $s$ and $-s$, weights $+\alpha$ and $-\alpha$, widths $\gamma \rightarrow 0$), the gradient reduces to a constant direction,

$\nabla f(z) \approx 4\alpha\gamma s,$

thus replicating global-linear traversal. Therefore, linear latent editing constitutes a degenerate instance of the WarpedGANSpace framework.

4. Experimental Analysis and Empirical Outcomes

Experiments on multiple GAN architectures (SN-GAN, BigGAN, ProgGAN, StyleGAN2) demonstrate that non-linear paths discovered by WarpedGANSpace yield more pronounced, disentangled, and interpretable image transformations compared to linear baselines. Specific findings include:

• Reconstructor Accuracy: Classification accuracy for the warp index reaches $92$–$99\%$ (non-linear), compared to $85$–$90\%$ (linear) across tested GANs.
• Path Steepness: Non-linear paths achieve up to $2\times$ larger cumulative semantic attribute changes (e.g., in pose, smile, identity, age) before image distortion.
• Disentanglement: Diagonal self-correlation entries in $\ell_1$-normalized attribute–path matrices are $0.52$–$0.61$ (non-linear) vs. $0.47$–$0.57$ (linear); off-diagonal correlations systematically lower, indicating reduced attribute entanglement.
• Non-linearity Coefficient: Defined as $$\varphi = \frac{\text{path length}}{\|\text{endpoint}-\text{start}\|}$$, with $\varphi \approx 1.1$–$1.5$ for non-linear paths ($\varphi=1$ is linear).
• Qualitative Edits: WarpedGANSpace on StyleGAN2’s $\mathcal{W}$-space uncovers unsupervised edits including “increase age”, “change skin tone”, “increase femaleness”, and “yaw rotation”, with improved orthogonality over baseline methods.

5. Properties, Limitations, and Prospective Directions

WarpedGANSpace operates in a fully unsupervised regime and thus does not guarantee alignment of each discovered path with human-perceivable semantic concepts; post hoc annotation via attribute classifiers is required for semantic labeling. The method currently synthesizes only first-order curve shapes (based on gradients); employing higher-order shape control (e.g., second-order tensors) could enable richer transformations. Fixed-step Euler integration may diverge from the imposed prior (e.g., Gaussian), implying a possible benefit from adaptive integration or geodesic latent traversal techniques.

The primary demonstrations are on face-generation models; extension to other domains or conditional GANs is presented as straightforward but remains to be empirically validated. Potential future enhancements include conditional or composite latent traversals (e.g., text-guided edits, fused warpings) and application to spatiotemporal GANs for video manipulation.

6. Summary and Significance

WarpedGANSpace generalizes latent space editing for GANs by learning non-linear, RBF-based scalar fields whose normalized gradients induce locally adaptive traversals through $\mathcal{Z}$. These traversals result in steeper, less entangled, and more semantically meaningful transformations in generated image space, consistently outperforming linear editing methods in both quantitative and qualitative assessments. The framework thus constitutes a unifying extension that encompasses linear latent path methodologies as a limiting case (Tzelepis et al., 2021).