Population-Based Multi-Objective Training of Discriminators for Semi-Supervised GANs

Abstract: Semi-supervised generative adversarial networks (SSL-GANs) can exploit large unlabeled datasets while retaining a classifier in the discriminator, but their training is often unstable. This paper proposes a population-based evolutionary training strategy in which discriminator learning is formulated as a multi-objective optimization problem. Instead of aggregating the supervised and unsupervised components of the SSL objective into a single scalar loss, the method maintains a population of discriminators ranked by Pareto dominance, enabling the exploration of different trade-offs between classification accuracy and real/fake discrimination. This formulation aims to improve both roles of SSL-GANs: learning accurate classifiers and training generators capable of producing realistic samples. We analyze several variants, including an elitist strategy and a mono-objective ablation, to assess the role of multi-objective selection. Experiments on MNIST with limited labels show improved training robustness compared to SSL-GAN and CE-SSL-GAN state-of-the-art baselines, while the elitist variant consistently achieves the highest classification accuracy.

• The paper introduces COMOD-SSLGAN, a multi-objective framework that decouples supervised classification and unsupervised discrimination to improve training stability.
• It employs population-based Pareto optimization to evolve discriminators, yielding up to 20% improvement in classification accuracy and enhanced sample quality.
• Empirical results on MNIST demonstrate that evolutionary selection and elitist strategies mitigate training collapse while promoting model diversity.

Population-Based Multi-Objective Discriminator Training for Semi-Supervised GANs

Introduction and Motivation

Population-based evolutionary approaches have gained traction for addressing instability and performance bottlenecks in adversarial training paradigms. Conventional Semi-Supervised GANs (SSL-GANs) collapse the dual role of the discriminator (supervised classification and unsupervised real/fake discrimination) into a single aggregated loss, often obscuring underlying trade-offs and exacerbating training instability. This work introduces COMOD-SSLGAN, a paradigm shift wherein discriminator learning becomes a bi-objective optimization problem managed via population-based Pareto optimization, preserving a spectrum of classifier-discriminator trade-offs and thus enhancing both robustness and downstream performance.

The COMOD-SSLGAN Architecture

COMOD-SSLGAN deploys twin populations of generators and discriminators. The discriminator population is evolved under explicit bi-objective selection pressure: one objective tracks supervised classification loss on labeled samples, the other tracks unsupervised discrimination loss on unlabeled and generated data. Selection is managed through nondominated sorting (as per NSGA-II), encouraging maintenance of a Pareto front of discriminators that jointly optimize both objectives. Generators, conversely, are selected monolithically by minimizing their usual adversarial loss.

Discriminator architecture follows the "K+1" output paradigm: $K$ class probabilities plus a "fake" class, supporting both classification and sample discrimination Figure 1.

Figure 1: SSL-GAN architecture—Generator maps latent noise to samples; Discriminator predicts both class and "fake."

The generator converts latent vectors into image-space samples. Training alternates between generator and discriminator updates, but, unlike conventional approaches, in COMOD-SSLGAN, SGD operates as the variation operator in the coevolutionary search.

Multi-Objective Formulation and Evolutionary Dynamics

Each discriminator $D_v$ is evaluated on two objectives: (1) average supervised loss over labeled data and (2) average unsupervised loss evaluated with respect to interactions with all generators in the current population. Generators are similarly cross-evaluated against all discriminators.

Survival selection for discriminators is performed via Pareto dominance and crowding distance to maximize diversity along the trade-off front. An elitist option is incorporated, allowing retention of the best individuals across generations to promote stability and preserve high-quality solutions.

Empirical evaluation adopts all-vs-all matchups between the two populations per iteration, ensuring comprehensive competitive pressure.

Experimental Setup

All experiments are conducted using MNIST, adopting a challenging SSL regime with only 100 labeled examples per class Figure 2.

Figure 2: Representative samples from the MNIST testbed used for SSL evaluation.

ConvNet architectures are used for both generator and discriminator. The generator includes a fully connected layer (latent to feature map) and transposed convolutions; the discriminator is a four-block ConvNet with sigmoid for final binary classification (real/fake).

Comparisons are made against standard SSL-GAN (scalarized objective) and CE-SSL-GAN (mono-objective population baseline), as well as within COMOD-SSLGAN variants: base, elitist, and mono-objective.

Empirical Results: Classification and Sample Quality

Discriminator Classification Accuracy

Significant improvements are reported across all population-based evolutionary variants compared to standard SSL-GAN. For $\mu=1$, COMOD-SSLGAN achieves $0.82$ median accuracy (vs $0.75$ for SSL-GAN, a $9.33\%$ relative improvement). Larger populations elicit even greater gains; the elitist variant achieves $0.90$ median accuracy at $\mu=5,7$, corresponding to $20\%$ improvement over baseline. Stability (IQR) is also markedly improved—population strategies reduce variance across runs and promote consistent convergence.

For small populations ($\mu=1$), performance remains robust—indicating that the benefit stems predominantly from the bi-objective selection rather than just population diversity.

Figure 3: Discriminator accuracy evolution for $D_v$0 and $D_v$1, illustrating both rapid improvement and enhanced final accuracy under population-based protocols.

Further, preserving elite individuals via elitist selection demonstrably mitigates performance collapse, maintaining high accuracy despite adversarial learning dynamics.

Generator Quality (SSIM)

Sample quality, as measured by SSIM, improves similarly. For the base variant with $D_v$2, SSIM median increases from $D_v$3 (SSL-GAN) to $D_v$4—a $D_v$5 improvement. Population-based variants yield SSIM up to $D_v$6 in CE-SSL-GAN with $D_v$7, reflecting more realistic and structurally similar outputs. Despite these quantitative differences, visual sample inspection reveals comparable high perceptual quality across all coevolutionary variants.

Figure 4: Early-stage samples ($D_v$8, epoch 10) from the base COMOD-SSLGAN, before significant generative refinement occurs.

Figure 5: Representative generated samples post-training, exemplifying visually indistinguishable output quality among coevolutionary methods.

Improvements in generator fidelity intensify the challenge for discriminators, triggering the observed accuracy drop late in training—a known adversarial "arms race" effect.

Theoretical and Practical Implications

COMOD-SSLGAN demonstrates that explicitly separating supervised and unsupervised objectives for discriminators not only facilitates exploration of meaningful trade-offs but also yields classifiers with superior accuracy and sample discrimination capability. Population-based optimization—especially with Pareto-based selection—establishes a robust framework immune to many adversarial training pathologies.

On the theoretical side, the results indicate that scalarization of multitask adversarial objectives is frequently suboptimal, as it masks conflicting gradients and stymies progress along the Pareto frontier. Practically, these findings suggest that SSL-GAN deployments in resource-constrained, label-scarce environments can leverage multi-objective evolutionary selection for more robust and accurate classifiers, with sample quality improvements as a secondary benefit.

The combination of multi-objective optimization and competitive coevolution, particularly with elitism, stabilizes the notoriously brittle adversarial training process and systematically outperforms both standard and single-objective population SSL-GANs on both classification and generation axes.

Future Directions

Several important avenues emerge:

• Scaling to complex datasets: The method's empirical success with MNIST suggests substantial promise, but evaluation on natural image datasets like CIFAR-10 or SVHN is necessary for more definitive conclusions.
• Multi-objective generator training: Current efforts focus on discriminator objectives; a natural extension is to adapt multi-objective frameworks for generators, balancing fidelity, diversity, and other generative metrics.
• Selection algorithms: Alternative multi-objective and diversity-preserving mechanisms (e.g., crowding, $D_v$9-dominance) may further enhance exploration and convergence.
• Distributed coevolution: To scale populations and enable training efficiency on distributed hardware, parallel coevolutionary schemes could be deployed, aligning with known benefits in adversarial settings.

Conclusion

COMOD-SSLGAN introduces a population-based, Pareto-dominance-driven approach to discriminator training in SSL-GANs. Treating classification and fake discrimination as independent objectives unlocks both improved classifier accuracy and superior sample generation. Elitist strategies further stabilize outcomes and maximize performance, highlighting the efficacy of evolutionary, multi-objective learning in adversarial semi-supervision.

These findings not only validate multi-objective coevolutionary schemes for SSL-GANs on canonical SSL benchmarks, but also point toward general principles of multitask adversarial control in neural systems. Prospective research is poised to generalize the framework to more challenging tasks, extend multi-objective design to the generator side, and elucidate the underlying adaptation mechanisms in broader adversarial learning environments.