Benefits of Low-Cost Bio-Inspiration in the Age of Overparametrization

Abstract: While Central Pattern Generators (CPGs) and Multi-Layer Perceptrons (MLP) are widely used paradigms in robot control, few systematic studies have been performed on the relative merits of large parameter spaces. In contexts where input and output spaces are small and performance is bounded, having more parameters to optimize may actively hinder the learning process instead of empowering it. To empirically measure this, we submit a given robot morphology, with limited proprioceptive capabilities, to controller optimization under two bio-inspired paradigms (CPGs and MLPs) with evolutionary- and reinforcement- trainer protocols. By varying parameter spaces across multiple reward functions, we observe that shallow MLPs and densely connected CPGs result in better performance when compared to deeper MLPs or Actor-Critic architectures. To account for the relationship between said performance and the number of parameters, we introduce a Parameter Impact metric which demonstrates that the additional parameters required by the reinforcement technique do not translate into better performance, thus favouring evolutionary strategies.

• The paper demonstrates that low-cost, bio-inspired controllers can achieve competitive performance by maximizing parameter efficiency.
• The paper reveals that CMA-ES outperforms PPO in synthesizing controllers with enhanced adaptability and cross-task transfer.
• The paper introduces the Parameter Impact metric, emphasizing diminishing returns of overparameterization in low-dimensional sensory/motor settings.

Comparative Analysis of Bio-Inspired and Overparameterized Control Architectures for Modular Robot Locomotion

Introduction

This work systematically investigates the interplay between controller architecture, learning algorithm, and reward function in the context of controller optimization for modular robots with restricted sensory capabilities. Specifically, the study evaluates Central Pattern Generators (CPGs) versus Multi-Layer Perceptrons (MLPs), and the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) versus Proximal Policy Optimization (PPO), across distinct reward landscapes. The contributions include (a) empirical quantification of performance versus parameter count for concise and overparameterized controllers, (b) introduction of the Parameter Impact metric to contextualize performance in terms of parameter efficiency, and (c) an exhaustive analysis of transfer and diversity capabilities across controller/trainer paradigms.

Experimental Setup

The evaluation is performed on a modular "spider" robot with 8 degrees of freedom (DoFs) in both actuation and proprioceptive observation, implemented in the ARIEL simulation framework leveraging MuJoCo. CPG controllers are configured by varying the range of pairwise connections (from isolated oscillators to densely interconnected networks), yielding parameter spaces from 8 up to 36 variables. MLP architectures are also systematically varied in depth (0, 1, 2 hidden layers) and width (number of neurons), with policies trained via both CMA-ES and PPO. The reward functions $R_s$ (forward speed), $R_g$ (Gymnasium/Ant-style, penalizing actuation and ground forces), and $R_k$ (RBF kernel-based, targeting a balanced, non-crawling gait at a desired speed and fixed elevation) are each used independently to drive optimization.

Performance Across Networks and Training Algorithms

Results show that CPGs and shallow MLPs can achieve similar levels of raw performance, but architectural and trainer choices yield nontrivial differences in parameter efficiency and adaptability:

• Parameter Efficiency: The most efficient CPG configuration ($c_4$) and the shallowest effective MLPs ($m^1_2$ or $m^2_8$) outperform deeper/overparameterized alternatives when parameter count is considered. Architectures with high numbers of parameters (e.g., PPO-trained $m^2_{64}$) offer negligible or marginal performance gains but at orders of magnitude greater parameter cost.
• Impact of Overparameterization: Overparameterization trends (increasing size/depth of MLPs) lead to diminishing or negative returns when both input and output spaces are low-dimensional, with CMA-ES particularly sensitive to parameter proliferation. PPO can leverage large networks but does not produce commensurate gains, and in some reward regimes (notably $R_g$), overparameterized MLPs perform worse. These results directly challenge general scaling laws in RL/ML that associate larger models with strictly monotonic performance improvement when task complexity does not warrant capacity (Caballero et al., 2022, Zhang et al., 2016, Wong et al., 2024).
• Trainer Effects: CMA-ES outperforms PPO in parameter efficiency. For CPGs, evolutionary optimization matches or exceeds PPO/MLP performance in $R_g$ and $R_k$. For MLPs, PPO delivers top speed in $R_g$0 but is suboptimal in frugal and balanced-gait regimes. Notably, training CPGs with RL remains an open problem, though recent work on gradient-based CPG tuning suggests future opportunities for hybrid approaches (Bellegarda et al., 2022, Bellegarda et al., 2022, Campanaro et al., 2021).

Parameter Impact and Cross-Performance

The Parameter Impact metric, defined as normalized task performance divided by $R_g$1(parameter count), reveals the inefficacy of overparameterization for this domain. Under this metric, nearly minimal CPGs and the smallest nontrivial MLPs produce top results. No architectural family universally dominates: CPGs are optimal for frugal and transferable controllers; MLPs can win on raw speed and enable greater gait diversity; PPO is optimal only in certain settings for maximizing forward speed, but at poor parameter efficiency.

Cross-performance analysis demonstrates that CPGs trained for balanced or frugal gaits generalize best across reward functions, preserving high rank regardless of evaluation metric—a consequence of architectural rhythmic bias and inherent regularization. MLP-based controllers, while capable of higher diversity and some peak task metrics, generalize less efficiently, particularly under penalizing reward regimes.

Behavioral and Diversity Analysis

Principal Component Analysis of fitted sinusoids to end-effector trajectories quantifies gait diversity. MLP-optimized controllers explore a wider variety of gaits due to their higher representational capacity, whereas CPGs generate more stereotyped, rhythmic solutions—corresponding with higher cross-task performance but reduced behavioral diversity. This trade-off is notable: higher diversity may be desirable when task variation is anticipated, but at the cost of efficiency and transfer.

Theoretical and Practical Implications

These results reinforce several insights relevant to neuroevolution, DRL for robotics, and embodied AI:

• Low-cost architectural priors (i.e., CPGs, shallow MLPs) are sufficient and often superior in parameter efficiency for robots with limited observation/action spaces, challenging the unqualified expansion of network capacity now prevalent in RL benchmarks (Bhattasali et al., 2024, Luo et al., 2023).
• Overparameterization can impede learning and degrade performance when controller expressiveness exceeds the complexity dictated by task or embodiment constraints. This is consistent with emerging evidence on the limits and "breaks" of neural scaling laws outside high-dimensional or data-rich regimes (Caballero et al., 2022, Zhang et al., 2016).
• Evolutionary strategies remain effective for controller synthesis in low-complexity, high-regularity settings and can match or outperform DRL approaches when the controller architecture admits direct policy search.
• Bio-inspired controller paradigms with strong architectural priors (e.g., CPGs, hybrid CPG-MLP hierarchies) offer robust generalization, transfer, and efficient learning—attributes increasingly sought for real-world and resource-constrained robotics (Bellegarda et al., 2022, Bellegarda et al., 2022, Wong et al., 2024).

Future Directions

To further validate these findings, key next steps include: (1) evaluating these controller/trainer regimes on morphologies with higher-dimensional sensors and actuation; (2) exploring hybrid architectures (e.g., hierarchical CPG-MLP networks) to capture both rhythmic priors and task-dependent complexity (Bhattasali et al., 2024, 2221.00458); (3) assessing the impact of introducing sensory feedback or richer reward shaping; (4) integrating gradient-based RL for CPGs as off-the-shelf solvers mature (Bellegarda et al., 2022, Campanaro et al., 2021); and (5) quantifying energy efficiency and sim-to-real transfer robustness as additional axes of practical impact (Rebolledo et al., 2021, Bellegarda et al., 2022, Bellegarda et al., 2022).

Conclusion

This comprehensive comparison demonstrates that for modular robots with small, bounded input/output spaces, low-cost, bio-inspired controller architectures with evolutionary optimization or shallow neural policy search deliver superior parameter efficiency and robust cross-task performance compared to overparameterized MLPs trained with state-of-the-art RL algorithms. The trade-offs between expressivity, diversity, and efficiency remain architecture- and trainer-dependent. The insights herein provide evidence for a more nuanced, task- and embodiment-sensitive approach to controller design in robotics and embodied AI, with direct consequences for efficient hardware deployment, generalization, and the ongoing integration of bio-inspired architectural priors in neural policy synthesis (2604.20365).