Determine whether proposed mechanisms cause plasticity loss

Determine whether dormant neurons, weight drift, representation collapse, and churn are causal mechanisms of plasticity loss in gradient-based deep reinforcement learning or merely symptoms that co-occur with it.

Background

The paper reviews several phenomena associated with plasticity loss in gradient-based reinforcement learning, including dormant neurons, drift of weights away from initialization, and churn caused by updates changing outputs on inputs other than those used to compute the update. The authors note that these phenomena may either cause plasticity loss or arise as consequences of it.

The causal status of these mechanisms remains unresolved because they co-occur empirically, and no single phenomenon has been found to account for plasticity loss. The paper’s neuroevolution experiments show that these symptoms do not necessarily arise under continual task changes, but they do not settle the broader causal question for gradient-based reinforcement learning.