Effectiveness of E-prop approximation for deep, large networks on challenging tasks

Ascertain the empirical effectiveness of the E-prop approximation when performing credit assignment across both depth and time for large deep recurrent neural networks on challenging tasks.

Background

The authors derive a deep-network extension of E-prop but report no experiments. They note confidence that a full RTRL variant would match BPTT, yet emphasize uncertainty about the E-prop approximation’s performance when scaled to large, deep networks addressing difficult problems.

Single-layer E-prop has shown performance approaching BPTT on simpler tasks, leaving open whether similar behavior holds under the added complexity of both depth and long temporal dependencies.