Muscle-Actuated Behavioral Replay

• Muscle-actuated behavioral replay is a computational and experimental approach that models and controls muscle activations to emulate natural movement patterns.
• It integrates neural control, biomechanics, and machine learning to simulate sensorimotor behaviors across different species, robotics, and prosthetic systems.
• The paradigm, validated through in-silico and experimental studies, enhances applications in neurorehabilitation, prosthetics, and realistic robotic locomotion.

Muscle-actuated behavioral replay is the computational and experimental recapitulation of animal or human behaviors by explicitly controlling or modeling muscle activations and their biomechanical consequences. This paradigm integrates neural control, muscle dynamics, and anatomical biomechanics to generate lifelike motion patterns, analyze sensorimotor pathways, and optimize artificial agents for naturalistic actuation. Modern approaches span deep learning, reinforcement learning, biophysical modeling, and modular simulation frameworks, and are applicable to a wide range of species and robotic or prosthetic systems.

1. Biomechanical and Muscle Modeling Principles

Muscle-actuated replay relies on explicit models of musculotendon units, encompassing both active and passive force generation mechanisms. Most contemporary frameworks use the Hill-type muscle model, in which muscle force is represented as the sum of contractile and elastic elements:

• Hill-type model and force computation: The force output for each muscle or musculotendon unit is given as $$f_{mtu} = \cos\alpha (a\cdot g_{al}(l_m) g_v(\dot{l}_m) + g_p(l_m))$$, describing activation-dependent contractile force modulated by fiber length, shortening velocity, and a passive component (Kim et al., 28 Jan 2024Özdil et al., 8 Sep 2025).
• Fatigue and adaptation: Advanced frameworks incorporate fatigue dynamics, such as the 3CC-r model, tracking the fractions of muscle in activated ($M_a$), fatigued ($M_F$), and resting ($M_r$) states evolving by differential equations (Feng et al., 2023).
• Control strategies: Actuator commands are mapped to muscles via direct neural stimulation (e.g., in FES), indirect neural population activity, or high-level control policies decoded through deep neural networks. Redundancy—multiple muscles actuating each degree of freedom—is resolved by static optimization, pseudo-inverse methods, or learning optimal recruitment patterns (Casas et al., 2020).

2. Neural Control, Connectomics, and Muscle Mapping

In full nervous-system-to-muscle replay, as implemented in modWorm for C. elegans (Kim et al., 25 Apr 2025), modular integration connects neural activity to muscle forces:

• Neural dynamics: Each neuron’s membrane potential $V$ is governed by biophysical equations incorporating leak, gap junction, and synaptic currents: $$dV/dt = -G_C (V-E_{cell}) - I_{gap}(V) - I_{syn}(V,s) + I_{ext}(t)$$.
• Muscle mapping: Motor neuron outputs are linearly mapped to muscle forces by $F_{body} = M \cdot V$, where $M$ is an empirically derived matrix connecting neuron voltage to muscle group activity.
• Proprioceptive feedback and inverse mapping: Simulated proprioceptive pathways invert muscle forces to reconstruct neural activity, closing the feedback loop and enabling adaptive motor behaviors.
• Low-dimensional decomposition: Singular value decomposition (SVD) is used to project high-dimensional neural activity onto “eigenworm” modes, linking circuit-level computation to stereotypical locomotor patterns.

3. Machine Learning and Reinforcement Learning Approaches

Deep learning and RL are heavily utilized to estimate, optimize, and generalize muscle activation policies:

• Deep autoencoders and DNNs: Architectures such as symmetric seven-layer autoencoders compress high-dimensional muscle activation trajectories (e.g., 300D for six muscles over time) into low-dimensional latent spaces and decode endpoint positions into precise activations, outperforming previous joint-torque networks (Khan et al., 2017).
• Reinforcement learning controllers: Algorithms including Normalized Advantage Function (NAF) (Abdi et al., 2018), PPO (Joos et al., 2020Kim et al., 28 Jan 2024), and Soft Actor-Critic (SAC) (Wannawas et al., 2021Wannawas et al., 2022) are used to learn policies mapping observations (kinematics, muscle states) to incremental or absolute muscle activations. Episode-based hard update and dual buffer experience replay stabilize training in high-dimensional spaces.
• Model-based RL and generative modeling: Latent skill representations are learned by variational autoencoders (MuscleVAE), with training objectives including reconstruction (tracking error), KL-divergence, and bioenergy regularization (Feng et al., 2023). Muscle-space PD control is inverted through muscle models to compute activation, subject to physical and fatigue constraints.
• Data efficiency and robustness: Muscle-actuated systems offer intrinsic stability and lower hyperparameter sensitivity than torque-driven actuators, filtering noisy control signals and buffering against perturbations (Wochner et al., 2022).

4. Replay from Experimental Data and Imitation Learning

Replay of naturalistic, measured behaviors is achieved by integrating anatomical models and kinematic data:

• Musculoskeletal simulation: Drosophila and ostrich models are reconstructed from CT/X-ray scans with detailed muscle-tendon geometries. Muscle forces are derived from static optimization (SO) or forward dynamics (FD) in OpenSim, or via RL/IL in MuJoCo for high-frequency, muscle-actuated behaviors (Özdil et al., 8 Sep 2025Barbera et al., 2021).
• Motion capture and bidirectional representations: Synchronized sEMG and video datasets (e.g., Muscles-in-Action) enable training models that predict muscle activation from pose and reconstruct motion from muscle activity, using transformer-based architectures for bidirectional mapping (Chiquier et al., 2022).
• Imitation learning: Policies are trained to reproduce reference kinematics, with reward functions incorporating joint or Cartesian tracking accuracy, and analysis of passive joint properties (damping/stiffness) demonstrating enhanced learning speed and behavioral compliance (Özdil et al., 8 Sep 2025).
• Muscle synergy and dimensionality reduction: Machine learning (e.g., non-negative matrix factorization) reveals that a small number of primitives account for most variance in muscle activation during behavior, consistent across biological and simulated systems.

5. Applications, Impact, and Validation

Muscle-actuated behavioral replay models have broad applications:

• Robotics and prosthetics: Muscle-driven controllers improve motion realism, energy efficiency, and adaptive robustness in anthropomorphic robots and advanced prostheses by harnessing the nonlinear smoothing of muscle activation dynamics (Casas et al., 2020Wochner et al., 2022).
• Neurorehabilitation and FES: RL-based FES controllers using recurrent networks (GRU, LSTM) for dynamic state representation outperform PID strategies, offering stable control under muscle fatigue and personalized stimulation for rehabilitation (Wannawas et al., 2021Wannawas et al., 2022).
• Biomechanics and computational neuroscience: Modular frameworks (modWorm, OpenSim) allow in-silico ablation, connectome variation, and mechanistic dissection of locomotion circuits, revealing neural pathways controlling sensorimotor responses (Kim et al., 25 Apr 2025).
• Simulation and animation: Realistic replay of gait, running, grooming, and acrobatics is achieved through muscle-based generative models (MuscleVAE), enabling style adaptation as fatigue develops and skill interpolation across latent spaces (Feng et al., 2023).
• Experimental validation: Model predictions of muscle synergy are tested via optogenetic or imaging experiments, matching or informing in-vivo measurements of coordinated activity across behaviors (Özdil et al., 8 Sep 2025Barbera et al., 2021).

6. Limitations, Optimization, and Future Directions

Key limitations and strategies for further development include:

• Exploration of redundant action spaces: Ineffective exploration in repetitive high-dimensional muscle spaces is mitigated by self-organizing control principles (e.g., Differential Extrinsic Plasticity in DEP-RL), which induce time- and actuator-correlated exploration for rapid learning (Schumacher et al., 2022Fischer et al., 2023).
• Integration and modularity: Frameworks like modWorm separate connectomics, cytoplasmic dynamics, muscle force mapping, and feedback, enabling rapid development and empirical testing of model variations.
• Biophysical realism: Simplified muscle models (e.g., inelastic tendons) in computational environments may limit physiological accuracy compared to detailed simulators (OpenSim). Further incorporation of fatigue, elasticity, and metabolic cost terms is ongoing.
• Generalization and person-specific adaptation: Transfer learning, subject-specific fine-tuning, and explicit conditioning mechanisms enable models to adapt across individuals and motion styles, enhancing clinical and interactive applications (Schmidt et al., 2022Chiquier et al., 2022).
• Sim-to-real transfer: Physically plausible muscle models are critical for closing the gap between simulation and embodied agents, with current frameworks analyzed for energy metrics, compliance, and robustness against unseen conditions.
• Open research directions: Principled coupling of exploration and policy learning (DEP-RL), broader multimodal datasets, real-time computer vision applications, and expanding model granularity to capture more nuanced aspects of muscle activity and biomechanics.

Muscle-actuated behavioral replay thus represents an overview of computational neuroscience, biomechanics, machine learning, and robotics for mechanistic understanding and lifelike reproduction of movement, validated across species and platforms, with ongoing research focused on increased realism, efficiency, and adaptive control.