RACEpSp: Race-Preserving Encoder in RA-GAN

• The paper introduces RACEpSp, a module that leverages frozen RaceNet and PyramidNet with RaceMixer blocks to extract and fuse race features.
• RACEpSp integrates a dual-path encoder and a feature mixer to ensure that original racial characteristics are retained during age transformation.
• Extensive experiments show that RA-GAN, with RACEpSp, outperforms competing models by up to +13.14 percentage points in race accuracy for kinship verification.

RACEpSp is a module within RA-GAN, proposed in “A Race Bias Free Face Aging Model for Reliable Kinship Verification” (Nazari et al., 18 Sep 2025). It is a face-aging encoder that injects an explicit “race-preservation” pathway into the latent-code encoder so that race-sensitive features are carried intact through the StyleGAN generator. In the RA-GAN pipeline, RACEpSp freezes two pre-trained subnetworks—a “RaceNet” trained on FairFace and the multi-scale “PyramidNet” from pSp—and fuses their intermediate features via light “RaceMixer” blocks. The resulting encoder $E_{race}(x)$ produces an $18\times 512$ style code $S_{face}$, which is then co-processed with the age code $S_{age}$ by the feature mixer so that the output face will retain its original racial characteristics while undergoing age transformation (Nazari et al., 18 Sep 2025).

1. Design motivations and problem setting

RACEpSp is motivated by the age-gap problem in kinship verification, which addresses the time difference between the photos of the parent and the child. Their same-age photos are often unavailable, and face aging models are racially biased, which impacts the likeness of photos. Standard face-aging GANs such as SAM-GAN and CUSP-GAN, when trained on race-imbalanced datasets, often “drift” the synthesized face’s perceived race when producing older or younger variants. In a kinship verification pipeline, this break in racial consistency harms downstream matching (Nazari et al., 18 Sep 2025).

The objective of RACEpSp is threefold: to extract a robust, pose-invariant representation of the subject’s race, to fuse these race features with the pSp age-style codes, and to yield a final $\mathcal{H}^{+}$ style code that guarantees the output face will retain its original racial characteristics. Within RA-GAN, RACEpSp is therefore not a stand-alone generator; it is the encoder branch responsible for preserving racial characteristics before the age and face codes are combined by the feature mixer and passed to the pre-trained StyleGAN-v2 generator. A common misconception is to equate RACEpSp with the entire RA-GAN system, whereas the paper defines RA-GAN as consisting of two new modules, RACEpSp and a feature mixer (Nazari et al., 18 Sep 2025).

2. Mathematical formulation

Let $x\in\mathbb{R}^{3\times H\times W}$ be a normalized input face, and let $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$ denote the pre-trained StyleGAN-v2 generator. RACEpSp implements an encoder

$$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$

The full RA-GAN forward pass is

$$x' = G\bigl(F\bigl[\;S_{age}\;,\;S_{face}\bigr]\bigr),$$

where

$S_{age} = E_{age}(x\Vert \alpha_t),$

$18\times 512$0 is the feature mixer, and $18\times 512$1 is the target age scalar (Nazari et al., 18 Sep 2025).

The race-consistency loss that drives RACEpSp to preserve race is

$18\times 512$2

where $18\times 512$3 is the penultimate convolutional feature from the frozen RaceNet. The total RA-GAN training loss is

$18\times 512$4

where $18\times 512$5 is the ArcFace ResNet-50 embedding, $18\times 512$6 is the VGG-DEX age feature, and $18\times 512$7 penalizes large style-code norms. This formulation places race preservation alongside reconstruction, identity preservation, age consistency, and latent regularization in a single joint objective (Nazari et al., 18 Sep 2025).

3. Encoder architecture and RaceMixer design

The overall layout feeds the input $18\times 512$8 to two parallel frozen encoders. The first is RaceNet, a ResNet-34 trained on FairFace. The model takes the activations after its 7th, 13th, and 15th residual blocks, denoted $18\times 512$9, $S_{face}$0. The second is PyramidNet, namely the three “pyramid” feature maps $S_{face}$1 from the pSp encoder levels 1, 2, and 3; these are also frozen (Nazari et al., 18 Sep 2025).

For each level $S_{face}$2, a small RaceMixer block fuses $S_{face}$3 and $S_{face}$4, together with an upsampled $S_{face}$5 residual from the previous mixer, into a mixed tensor $S_{face}$6, where $S_{face}$7. Each RaceMixer consists of a transpose-convolution that brings $S_{face}$8 and $S_{face}$9 to a common spatial resolution $S_{age}$0; a two-layer convolutional autoencoder, written as conv$S_{age}$1ReLU$S_{age}$2conv$S_{age}$3ReLU with batch-norm in between, which “blends” $S_{age}$4 into $S_{age}$5 space; and a parallel “scalar” $S_{age}$6 convolution on $S_{age}$7 alone that learns a gating weight $S_{age}$8. The final output is

$S_{age}$9

Each $\mathcal{H}^{+}$0 is then mapped to six style codes, each a 512-dimensional vector, by six successive “Map2Style” FC layers as in pSp. Stacking the $\mathcal{H}^{+}$1 codes yields $\mathcal{H}^{+}$2. All RaceMixer convolutions use ReLU, batch-norm is applied on each convolution output, upsampling is nearest-neighbor $\mathcal{H}^{+}$3, and there is no attention or gating beyond the scalar $\mathcal{H}^{+}$4 (Nazari et al., 18 Sep 2025).

4. Integration within RA-GAN

The RA-GAN data flow is specified in five stages. First, $\mathcal{H}^{+}$5, producing an $\mathcal{H}^{+}$6 code. Second, $\mathcal{H}^{+}$7, the pSp age-encoder, yielding $\mathcal{H}^{+}$8. Third, the model concatenates along the style-code dimension:

$\mathcal{H}^{+}$9

Fourth, the feature mixer $x\in\mathbb{R}^{3\times H\times W}$0 learns a soft-attention over these 36 codes to select or weight the most relevant ones and outputs $x\in\mathbb{R}^{3\times H\times W}$1. Fifth, $x\in\mathbb{R}^{3\times H\times W}$2, producing the synthesized face (Nazari et al., 18 Sep 2025).

During backward-propagation, only $x\in\mathbb{R}^{3\times H\times W}$3, $x\in\mathbb{R}^{3\times H\times W}$4, including all RaceMixer sub-modules, and $x\in\mathbb{R}^{3\times H\times W}$5 are trainable. The generator $x\in\mathbb{R}^{3\times H\times W}$6, RaceNet, and PyramidNet are frozen. This division of labor is central to the module’s definition: RACEpSp contributes a race-preserving latent representation, but the final age-transformed image is produced only after joint processing with the age encoder and feature mixer. A plausible implication is that the module is designed to constrain the transformation path rather than to relearn a full generative backbone (Nazari et al., 18 Sep 2025).

5. Training protocol and optimization regime

The reported training protocol uses batch size 4 with $x\in\mathbb{R}^{3\times H\times W}$7 crops. Two optimizers are used. Optimizer #1, described as the forward pass, is Adam with learning rate $x\in\mathbb{R}^{3\times H\times W}$8, $x\in\mathbb{R}^{3\times H\times W}$9, $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$0, and weight decay $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$1. Optimizer #2, described as the reconstruction pass, is Adam with learning rate $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$2, $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$3, $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$4, and weight decay $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$5 (Nazari et al., 18 Sep 2025).

The loss weights are

$$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$6

Training is conducted for approximately 25 epochs on a race-balanced UTKFace split of approximately 4800 train and approximately 1140 test samples. All inputs are zero-mean, unit-variance normalized per-channel, and the fourth age channel satisfies $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$7. These settings locate RACEpSp within a tightly constrained training regime in which the race-preservation term is explicitly weighted rather than left to emerge implicitly from reconstruction or identity objectives (Nazari et al., 18 Sep 2025).

6. Experimental behavior, ablations, and downstream kinship verification

Experimental validation is reported on a held-out UTKFace-derived test set with approximately 1100 samples per age. For race preservation, a frozen ResNet-34 race-classifier obtains overall accuracies of 36.6%–67.1% for SAM-GAN, with catastrophic drops in the 70–80 group; 53.7%–78.9% for CUSP-GAN up to age 60, with no 70/80 results; and 49.3%–77.4% for RA-GAN consistently across 20–80 years. RA-GAN beats SAM-GAN by an average of $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$8 percentage points across all ages and outperforms CUSP-GAN by $$G:\mathbb{R}^{18\times 512}\to\mathbb{R}^{3\times 256\times 256}$$9 percentage points at age 60. Class-wise $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$0 improves on average over SAM-GAN by $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$1 for Asian, $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$2 for White, $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$3 for Black, and $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$4 for Indian (Nazari et al., 18 Sep 2025).

Identity preservation is evaluated through ArcFace-embedding cosine similarity between $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$5 and $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$6. RA-GAN reports 0.40–0.49, versus 0.42–0.61 for SAM-GAN and 0.35–0.67 for CUSP-GAN. The paper states that RA-GAN matches SAM-GAN on older faces and outperforms on 70–80. For age fidelity, Face++ estimated age MAE is 4.7–9.8 years for RA-GAN, compared with 4.7–10.1 years for SAM-GAN and 3.7–7.2 years for CUSP-GAN; RA-GAN is within $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$7 years of SAM-GAN on ages 20–80, and within $$S_{face} = E_{race}(x)\in\mathbb{R}^{18\times 512}.$$8 years of CUSP-GAN up to age 60. This indicates that RACEpSp’s contribution is not simply lower age error. A plausible implication is that the module trades toward racial consistency while keeping age fidelity within a narrow range relative to the baselines (Nazari et al., 18 Sep 2025).

The ablation without RACEpSp removes the RaceMixer path and drops race accuracy by 11 percentage points at age 50–80, while lowering kinship-verification gains by 3–5 percentage points in controlled experiments. In downstream kinship verification, transforming parent and child images from the KinFaceW-I and KinFaceW-II datasets to the same age can enhance the verification accuracy across all age groups. On KinFaceW-I, the accuracy increases with RA-GAN for father-son, father-daughter, mother-son, and mother-daughter by 5.22, 5.12, 1.63, and 0.41, respectively. On KinFaceW-II, the corresponding increases are 2.9 for father-daughter, 0.39 for father-son, and 1.6 for mother-son. The paper therefore situates RACEpSp as the component that brings a targeted, lightweight “race continuity” branch into the pSp + StyleGAN pipeline, yielding a double-digit improvement in racial consistency that is crucial for downstream kinship verification (Nazari et al., 18 Sep 2025).