Generality of the convolutional-network drift ordering in recurrent systems

Determine whether the ordering of representational drift profiles observed across continual-learning methods in convolutional networks generalizes to recurrent neural networks.

Background

The convolutional-network experiments compared naive sequential training, elastic weight consolidation, Learning without Forgetting, and experience replay, whereas the recurrent-network analysis reliably compared only naive training and replay. Elastic weight consolidation and Learning without Forgetting did not provide sufficiently reliable continual-learning performance in the recurrent setting for an interpretable drift comparison.

Consequently, the relative ordering of methods by retention, plasticity, and representational stability established in the convolutional experiments has not been shown to hold for recurrent architectures. Establishing this generalization would require reliable, interpretable comparisons among the corresponding continual-learning methods in recurrent systems.