From Consistency to Collaborative Discovery: MFEA-CoD for Multitask Novelty Search

Abstract: Evolutionary multitasking (EMT) has shown strong capability in solving multiple optimization problems simultaneously by exploiting latent inter-task consistency, such as similarities in promising solutions or search directions. However, most existing EMT studies remain focused on objective-driven optimization, where such consistency is mainly used to accelerate convergence toward predefined optima. In this paper, we move EMT from consistency to collaborative discovery and propose a multifactorial evolutionary algorithm with collaborative discovery (MFEA-CoD) for multitask novelty search. Unlike conventional EMT, MFEA-CoD coordinates multiple novelty search tasks to collaboratively discover behaviorally novel solutions rather than merely transferring consistent search information for faster convergence. Specifically, a multitask repulsion operator encourages different tasks to explore distinct regions of the unified search space, thereby reducing redundant behavioral discoveries. Meanwhile, an adaptive inter-task transfer mechanism exploits shared discovery opportunities in overlapping novelty-improving regions by adjusting the transfer probability according to the online contribution of transferred information. Furthermore, MFEA-CoD is extended to multitask novelty-augmented optimization, where behavioral novelty is jointly considered with objective information to alleviate premature convergence caused by deceptive objectives. Experiments on synthetic basin-type problems, deceptive maze navigation problems, MuJoCo policy optimization problems, and generative novelty search problems demonstrate that MFEA-CoD improves the efficiency of discovering diverse novel solutions and shows clear advantages in deceptive objective landscapes.

• The paper introduces MFEA-CoD, a multitask algorithm that employs a repulsion operator and adaptive transfer to enhance novelty search efficiency.
• It demonstrates superior behavioral coverage and faster convergence across synthetic benchmarks, maze navigation, and MuJoCo control tasks.
• Theoretical proofs and empirical validations confirm that structured collaboration reduces redundant exploration and sample evaluations.

Review of "From Consistency to Collaborative Discovery: MFEA-CoD for Multitask Novelty Search" (2607.00761)

Motivation and Theoretical Foundations

The paper analyzes the limitations of conventional Evolutionary Multitasking (EMT) algorithms when applied in the context of novelty search. Traditional EMT leverages latent inter-task consistency for objective-driven optimization but is fundamentally aligned with convergence-centric scenarios. In contrast, novelty search emphasizes behavioral diversity within the phenotype space, and cross-task consistency can be deleterious by limiting exploration and inducing redundant behavior coverage. The authors formalize collaborative discovery as a multitask paradigm for novelty search, positing that complementary exploration across tasks improves coverage efficiency and reduces redundant discovery of behavioral phenotypes. This is particularly salient under varying overlap scenarios among the genotype sets for different tasks.

The theoretical contributions include a rigorous analysis of multitask novelty search in scenarios of overlapping, disjoint, and partially overlapping genotype sets. The authors demonstrate that, by structuring collaborative discovery around repulsion between task-specific search processes, and by dynamically regulating inter-task transfer, the collective search attains higher behavioral coverage with fewer evaluations than independent single-task searches. The provided theorem quantifies sample efficiency benefits under coupon collector hypotheses, showing provably reduced evaluation budgets in the presence of shared novelty units, conditional on structured transfer and repulsion mechanisms.

Figure 1: MFEA-CoD’s collaborative search under overlapping, disjoint, and partially overlapping genotype sets clarifies the transition from consistency-driven EMT to complementary multitask exploration.

Algorithmic Design: MFEA-CoD Framework

MFEA-CoD (Multifactorial Evolutionary Algorithm with Collaborative Discovery) is introduced as the first multitask evolutionary algorithm specifically targeting multitask novelty search. The framework distinguishes itself along two principal axes:

1. Multitask Repulsion Operator: This module estimates the density of each task’s archive using a diagonal-covariance Gaussian and induces search pressure—via explicit gradient-like repulsive updates—away from archive-dense regions associated with other tasks. This enforces search complementarity and discourages redundant exploration, aligning multitask search trajectories with distinct or weakly-overlapping regions of the genotype space.
2. Adaptive Inter-Task Transfer Probability (A-ITP): Cross-task solution transfer is governed by a parameter matrix dynamically updated to reflect the empirical utility of each transfer channel. The mechanism employs moving-average reward matrices predicated on offspring contributions to archive innovation or objective improvement, thereby optimizing the exploitation of overlap regions while mitigating negative transfer in unrelated or adversarial tasks.

The algorithm is built atop the emitter-based illumination framework (e.g., modular compatibility with PyRibs, use of evolution strategy emitters, ProximityArchive), robustly supporting comparative and cooperative exploration across heterogeneous multitask landscapes.

Extension to Novelty-Augmented Optimization

The MFEA-CoD design is generalized to support novelty-augmented optimization, where composite search criteria balance fitness and behavioral novelty via an adaptive weighting (e.g., NSRA-style adaptation). The repulsive operator interacts with these adaptive weights, enforcing stronger repulsion under exploration-favoring regimes and decaying its influence during exploitation. The A-ITP component is reformulated to utilize improvement in objective value as the reward signal, directly optimizing for exploitative transfer in the presence of deceptive fitness landscapes.

Experimental Validation

Synthetic Basin-Type Multitask Problems

Experiments on synthetic two-dimensional multitask settings systematically explore fully overlapping, partially overlapping, and disjoint genotype set scenarios.

• Convergence Trends: MFEA-CoD exhibits faster archive size convergence and higher final coverage across all scenarios compared to single-task novelty search, multitask novelty search (fixed transfer), and MFEA-CoD without A-ITP.
• Search Distribution: Visualizations confirm that the multitask repulsion operator produces decoupled, complementary exploration patterns, especially in overlap regions, quantitatively supporting the theoretical model.

Figure 2: Visualization of overlap regimes in synthetic benchmarks, clarifying where multitask repulsion yields maximal coverage gains.

Figure 3: Archive size convergence on synthetic problems demonstrates that repulsion and adaptive transfer are both necessary for efficient, redundant-free novelty discovery.

Deceptive Maze Navigation

Evaluations on multitask maze navigation (with different starts, goals, and environments) establish that MFEA-CoD produces larger, more diverse archives and achieves higher success rates and faster solution discovery in strongly deceptive setups.

• Comparisons vs. MAP-Elites/CMA-ME: The approach outperforms or matches quality-diversity algorithms, validating its capacity to escape deceptive attractors by leveraging multitask collaboration.
• Statistical Significance: Wilcoxon rank-sum tests confirm robust improvements in coverage and solution timeliness.

Figure 4: Maze scenarios operationalize multitask novelty search’s advantage where optimal behaviors are structurally divergent between tasks.

Figure 5: MFEA-CoD achieves superior and faster archive expansion versus both novelty-centric and quality-diversity baselines in maze navigation.

MuJoCo Policy Optimization: Novelty-Augmented Setting

On high-dimensional continuous control benchmarks (Hopper and Walker2d under dynamics shift), MFEA-CoD yields improved policy quality and convergence rates:

• Reward Maximization: MFEA-CoD matches or surpasses CMA-ES, NSRA-ES, and multitask NSRA-ES in best cumulative rewards.
• Adaptive Transfer: Empirical ITP evolution confirms that transfer probability reduces with task discordance, validating the adaptive mechanism.

Figure 6: Application to complex MuJoCo domains illustrates the scalability and generality of the approach.

Figure 7: Reward curves on MuJoCo show that multitask novelty-augmented optimization both accelerates RL policy improvement and effectively balances exploration/exploitation.

Generative Novelty Search

On GAN latent space exploration, MFEA-CoD produces larger, more semantically diverse behavioral archives than single-task search, supporting the generality of the collaborative-discovery paradigm in non-control, semantic spaces.

Figure 8: MFEA-CoD generates more behaviorally diverse and varied GAN samples than independent novelty search, emphasizing scalable collaborative coverage.

Implications and Future Directions

The principal contribution of this work is the formal establishment of collaborative discovery as a distinct design axis for multitask divergent search, orthogonal to classic consistency-driven EMT. The multitask repulsion operator and adaptive inter-task transfer provide theoretically justified, empirically validated mechanisms for efficient behavioral coverage in multitask novelty search. These innovations have compelling implications for domains characterized by concurrent exploration of related but non-identical goal spaces (e.g., evolutionary robotics, multi-domain generative design, multitask RL).

Practically, the shift from consistency (fitness or search-direction similarity) to structured collaboration (repulsion and adaptive transfer) enables scalable and sample-efficient discovery of novel solutions, especially in complex, deceptive landscapes. The approach generalizes to novelty-augmented optimization and thus interfaces cleanly with contemporary quality-diversity paradigms.

Future development could further integrate hierarchical multitasking, better exploit archive structure for latent space modeling, and extend adaptive transfer learning to scenarios with task heterogeneity beyond differences in search region structure (e.g., variable-genotype problems).

Conclusion

The paper makes a significant advance in the study and practical implementation of multitask divergent search, introducing MFEA-CoD as a principled algorithm that leverages multitask repulsion and adaptive transfer to optimize behavioral coverage. Both theoretical analysis and extensive empirical evidence demonstrate substantial improvements over both single-task and non-repulsion multitask baselines across synthetic, control, navigation, and generative domains. The results position collaborative discovery as an essential component of future multitask novelty and quality-diversity search frameworks.