Dice Question Streamline Icon: https://streamlinehq.com

Open problems in scaling, evaluating, and experimentally validating biologically plausible credit assignment algorithms

Establish effective methodologies to improve, scale, and rigorously test the capabilities of biologically plausible credit assignment algorithms, including predictive coding, contrastive Hebbian learning, and forward-only learning, and develop experimental protocols to empirically verify the specific claims these algorithms imply about cortical processing.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper surveys several biologically plausible credit assignment frameworks—predictive coding, contrastive Hebbian learning (including equilibrium propagation), forward-only learning, and related approaches such as direct feedback alignment, target propagation, and local representation alignment. While these methods address key biological and hardware-related critiques of backpropagation, they have not consistently matched the practical performance and scalability of backpropagation on modern deep learning tasks.

The authors emphasize that artificial neural networks provide a test-bed for assessing the capability of neuromimetic algorithms and for rapidly probing hypotheses about cortical computation. However, they note that many challenges remain open: improving and scaling these methods, establishing rigorous evaluation practices, and experimentally verifying the cortical processing implications that these algorithms suggest. Addressing these open problems is necessary both for practical application and for connecting machine learning models to neuroscientific evidence.

References

In the last decade, a number of significant algorithms, which are reviewed in this work, have been proposed as solutions to the problem of cortical credit assignment. While these can serve as starting points for development, there remain many open problems both in improving, scaling, and testing the capabilities of these algorithms, as well as experimentally checking the claims that they imply about cortical processing.

A Review of Neuroscience-Inspired Machine Learning (2403.18929 - Ororbia et al., 16 Feb 2024) in Section 5, Discussion and Conclusion (Impact in the Neurosciences paragraph)