---
title: 'RACEpSp: Race-Preserving Encoder in RA-GAN'
url: https://www.emergentmind.com/topics/racepsp
type: topic
---

# RACEpSp: Race-Preserving Encoder in RA-GAN

RACEpSp is a module within RA-GAN, proposed in “A Race Bias Free Face Aging Model for Reliable Kinship Verification” [2509.15177]. It is a face-aging encoder that injects an explicit “race-preservation” pathway into the latent-code encoder so that race-sensitive features are carried intact through the StyleGAN generator. In the RA-GAN pipeline, RACEpSp freezes two pre-trained subnetworks—a “RaceNet” trained on FairFace and the multi-scale “PyramidNet” from pSp—and fuses their intermediate features via light “RaceMixer” blocks. The resulting encoder $E_{race}(x)$ produces an $18\times 512$ style code $S_{face}$, which is then co-processed with the age code $S_{age}$ by the feature mixer so that the output face will retain its original racial characteristics while undergoing age transformation [2509.15177].

## 1. Design motivations and problem setting

RACEpSp is motivated by the age-gap problem in kinship verification, which addresses the time difference between the photos of the parent and the child. Their same-age photos are often unavailable, and face aging models are racially biased, which impacts the likeness of photos. Standard face-aging GANs such as SAM-GAN and CUSP-GAN, when trained on race-imbalanced datasets, often “drift” the synthesized face’s perceived race when producing older or younger variants. In a kinship verification pipeline, this break in racial consistency harms downstream matching [2509.15177].

The objective of RACEpSp is threefold: to extract a robust, pose-invariant representation of the subject’s race, to fuse these race features with the pSp age-style codes, and to yield a final $\mathcal{H}^{+}$ style code that guarantees the output face will retain its original racial characteristics. Within RA-GAN, RACEpSp is therefore not a stand-alone generator; it is the encoder branch responsible for preserving racial characteristics before the age and face codes are combined by the feature mixer and passed to the pre-trained StyleGAN-v2 generator. A common misconception is to equate RACEpSp with the entire RA-GAN system, whereas the paper defines RA-GAN as consisting of two new modules, RACEpSp and a feature mixer [2509.15177].

## 2. Mathematical formulation

Let $x\in\mathbb{R}^{3\times H\times W}$ be a normalized input face, and let $G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$ denote the pre-trained StyleGAN-v2 generator. RACEpSp implements an encoder
$$
S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.
$$

The full RA-GAN forward pass is
$$
x' = G\bigl(F\bigl[\;S_{age}\;,\;S_{face}\bigr]\bigr),
$$
where
$$
S_{age} = E_{age}(x\Vert \alpha_t),
$$
$F(\cdot)$ is the feature mixer, and $\alpha_t$ is the target age scalar [2509.15177].

The race-consistency loss that drives RACEpSp to preserve race is
$$
L_{race}(x,x')= \bigl\|\,F_{RaceNet}(x)\;-\;F_{RaceNet}(x')\bigr\|_2,
$$
where $F_{RaceNet}(\cdot)$ is the penultimate convolutional feature from the frozen RaceNet. The total RA-GAN training loss is
$$
L_{total} = \lambda_{l2}\,\|x-x'\|_2
+\lambda_{id}\,\|\phi_{Arc}(x)-\phi_{Arc}(x')\|_2
+\lambda_{age}\,\|\phi_{Age}(x)-\phi_{Age}(x')\|_2
+\lambda_{race}\,L_{race}(x,x')
+\lambda_{w}\,L_{w\_norm}(F_{mix}),
$$
where $\phi_{Arc}$ is the ArcFace ResNet-50 embedding, $\phi_{Age}$ is the VGG-DEX age feature, and $L_{w\_norm}$ penalizes large style-code norms. This formulation places race preservation alongside reconstruction, identity preservation, age consistency, and latent regularization in a single joint objective [2509.15177].

## 3. Encoder architecture and RaceMixer design

The overall layout feeds the input $x$ to two parallel frozen encoders. The first is RaceNet, a ResNet-34 trained on FairFace. The model takes the activations after its 7th, 13th, and 15th residual blocks, denoted $R_i\in\mathbb{R}^{C_{r_i}\times h_{r_i}\times w_{r_i}}$, $i=1,2,3$. The second is PyramidNet, namely the three “pyramid” feature maps $P_i\in\mathbb{R}^{C_{p_i}\times h_{p_i}\times w_{p_i}}$ from the pSp encoder levels 1, 2, and 3; these are also frozen [2509.15177].

For each level $i=1\ldots 3$, a small RaceMixer block fuses $P_i$ and $R_i$, together with an upsampled $H_{i-1}$ residual from the previous mixer, into a mixed tensor $H_i\in\mathbb{R}^{C_0\times h_0\times w_0}$, where $C_0=512/\text{higher levels}$. Each RaceMixer consists of a transpose-convolution that brings $P_i$ and $R_i$ to a common spatial resolution $h_0\times w_0$; a two-layer convolutional autoencoder, written as conv$\to$ReLU$\to$conv$\to$ReLU with batch-norm in between, which “blends” $R_i$ into $P_i$ space; and a parallel “scalar” $1\times 1$ convolution on $P_i$ alone that learns a gating weight $\alpha_i\in\mathbb{R}$. The final output is
$$
H_i = AE(P_i\oplus R_i) + \alpha_i\cdot Upsample(H_{i-1}).
$$

Each $H_i$ is then mapped to six style codes, each a 512-dimensional vector, by six successive “Map2Style” FC layers as in pSp. Stacking the $3\times 6=18$ codes yields $S_{face}\in\mathbb{R}^{18\times 512}$. All RaceMixer convolutions use ReLU, batch-norm is applied on each convolution output, upsampling is nearest-neighbor $\times 2$, and there is no attention or gating beyond the scalar $\alpha_i$ [2509.15177].

## 4. Integration within RA-GAN

The RA-GAN data flow is specified in five stages. First, $x\to E_{race}=\text{RACEpSp}\to S_{face}$, producing an $18\times 512$ code. Second, $x\Vert \alpha_t\to E_{age}$, the pSp age-encoder, yielding $S_{age}\in\mathbb{R}^{18\times 512}$. Third, the model concatenates along the style-code dimension:
$$
C = stack(S_{age},S_{face}) \in\mathbb{R}^{36\times 512}.
$$
Fourth, the feature mixer $F$ learns a soft-attention over these 36 codes to select or weight the most relevant ones and outputs $F_{mix}\in\mathbb{R}^{18\times 512}$. Fifth, $F_{mix}\to G\to x'$, producing the synthesized face [2509.15177].

During backward-propagation, only $E_{age}$, $E_{race}$, including all RaceMixer sub-modules, and $F$ are trainable. The generator $G$, RaceNet, and PyramidNet are frozen. This division of labor is central to the module’s definition: RACEpSp contributes a race-preserving latent representation, but the final age-transformed image is produced only after joint processing with the age encoder and feature mixer. A plausible implication is that the module is designed to constrain the transformation path rather than to relearn a full generative backbone [2509.15177].

## 5. Training protocol and optimization regime

The reported training protocol uses batch size 4 with $256\times 256$ crops. Two optimizers are used. Optimizer #1, described as the forward pass, is Adam with learning rate $1\times 10^{-7}$, $\beta_1=0.9$, $\beta_2=0.99$, and weight decay $0$. Optimizer #2, described as the reconstruction pass, is Adam with learning rate $5\times 10^{-5}$, $\beta_1=0.5$, $\beta_2=0.99$, and weight decay $1\times 10^{-5}$ [2509.15177].

The loss weights are
$$
\lambda_{l2}=0.25,\qquad
\lambda_{id}=0.10,\qquad
\lambda_{age}=5.0,\qquad
\lambda_{race}=3.0,\qquad
\lambda_{w}=0.005.
$$
Training is conducted for approximately 25 epochs on a race-balanced UTKFace split of approximately 4800 train and approximately 1140 test samples. All inputs are zero-mean, unit-variance normalized per-channel, and the fourth age channel satisfies $\alpha_t\sim Uniform(10,80)$. These settings locate RACEpSp within a tightly constrained training regime in which the race-preservation term is explicitly weighted rather than left to emerge implicitly from reconstruction or identity objectives [2509.15177].

## 6. Experimental behavior, ablations, and downstream kinship verification

Experimental validation is reported on a held-out UTKFace-derived test set with approximately 1100 samples per age. For race preservation, a frozen ResNet-34 race-classifier obtains overall accuracies of 36.6%–67.1% for SAM-GAN, with catastrophic drops in the 70–80 group; 53.7%–78.9% for CUSP-GAN up to age 60, with no 70/80 results; and 49.3%–77.4% for RA-GAN consistently across 20–80 years. RA-GAN beats SAM-GAN by an average of $+13.14$ percentage points across all ages and outperforms CUSP-GAN by $+9.1$ percentage points at age 60. Class-wise $F_1$ improves on average over SAM-GAN by $+7.4$ for Asian, $+10.6$ for White, $+17.3$ for Black, and $+23.3$ for Indian [2509.15177].

Identity preservation is evaluated through ArcFace-embedding cosine similarity between $x$ and $x'$. RA-GAN reports 0.40–0.49, versus 0.42–0.61 for SAM-GAN and 0.35–0.67 for CUSP-GAN. The paper states that RA-GAN matches SAM-GAN on older faces and outperforms on 70–80. For age fidelity, Face++ estimated age MAE is 4.7–9.8 years for RA-GAN, compared with 4.7–10.1 years for SAM-GAN and 3.7–7.2 years for CUSP-GAN; RA-GAN is within $\pm 0.3$ years of SAM-GAN on ages 20–80, and within $+2$ years of CUSP-GAN up to age 60. This indicates that RACEpSp’s contribution is not simply lower age error. A plausible implication is that the module trades toward racial consistency while keeping age fidelity within a narrow range relative to the baselines [2509.15177].

The ablation without RACEpSp removes the RaceMixer path and drops race accuracy by 11 percentage points at age 50–80, while lowering kinship-verification gains by 3–5 percentage points in controlled experiments. In downstream kinship verification, transforming parent and child images from the KinFaceW-I and KinFaceW-II datasets to the same age can enhance the verification accuracy across all age groups. On KinFaceW-I, the accuracy increases with RA-GAN for father-son, father-daughter, mother-son, and mother-daughter by 5.22, 5.12, 1.63, and 0.41, respectively. On KinFaceW-II, the corresponding increases are 2.9 for father-daughter, 0.39 for father-son, and 1.6 for mother-son. The paper therefore situates RACEpSp as the component that brings a targeted, lightweight “race continuity” branch into the pSp + StyleGAN pipeline, yielding a double-digit improvement in racial consistency that is crucial for downstream kinship verification [2509.15177].

Source: https://www.emergentmind.com/topics/racepsp