Deep Learning Models of the Retinal Response to Natural Scenes (1702.01825v1)

Abstract: A central challenge in neuroscience is to understand neural computations and circuit mechanisms that underlie the encoding of ethologically relevant, natural stimuli. In multilayered neural circuits, nonlinear processes such as synaptic transmission and spiking dynamics present a significant obstacle to the creation of accurate computational models of responses to natural stimuli. Here we demonstrate that deep convolutional neural networks (CNNs) capture retinal responses to natural scenes nearly to within the variability of a cell's response, and are markedly more accurate than linear-nonlinear (LN) models and Generalized Linear Models (GLMs). Moreover, we find two additional surprising properties of CNNs: they are less susceptible to overfitting than their LN counterparts when trained on small amounts of data, and generalize better when tested on stimuli drawn from a different distribution (e.g. between natural scenes and white noise). Examination of trained CNNs reveals several properties. First, a richer set of feature maps is necessary for predicting the responses to natural scenes compared to white noise. Second, temporally precise responses to slowly varying inputs originate from feedforward inhibition, similar to known retinal mechanisms. Third, the injection of latent noise sources in intermediate layers enables our model to capture the sub-Poisson spiking variability observed in retinal ganglion cells. Fourth, augmenting our CNNs with recurrent lateral connections enables them to capture contrast adaptation as an emergent property of accurately describing retinal responses to natural scenes. These methods can be readily generalized to other sensory modalities and stimulus ensembles. Overall, this work demonstrates that CNNs not only accurately capture sensory circuit responses to natural scenes, but also yield information about the circuit's internal structure and function.

• The paper demonstrates that deep CNNs predict retinal ganglion cell responses with performance nearing the natural variability of neuronal activity.
• The study reveals that CNNs capture key retinal processing features such as feedforward inhibition and contrast adaptation, enhancing biological realism.
• Methodologically, incorporating RNN layers and latent noise offers a robust framework for modeling long-term neural dynamics under natural stimulation.

Analysis of Deep Learning Models of the Retinal Response to Natural Scenes

This paper presents an in-depth investigation into the application of deep convolutional neural networks (CNNs) for modeling retinal responses to natural stimuli. The authors address a significant challenge in sensory neuroscience: understanding the complex neural computations and circuit mechanisms that encode ethologically relevant stimuli. The primary focus is on the prediction of retinal ganglion cell responses, particularly when exposed to natural image sequences.

Key Findings

The paper demonstrates that CNNs offer predictive power nearly matching the inherent variability of single retinal cell responses. Notably, CNNs significantly outperform linear-nonlinear (LN) models and generalized linear models (GLMs) in capturing both white noise and natural scene stimuli. This superior performance underscores CNNs' ability to generalize across stimulus classes without succumbing to overfitting, even when trained on limited data. The work reveals that training CNNs with white noise but testing on natural scenes still yields better results compared to LN models trained on the same specific task, highlighting the robustness of CNNs in capturing retinal responses across different visual environments.

Mechanistic Insights

Beyond predictive performance, the paper uncovers several mechanistic insights about retinal computations. CNNs trained on natural stimuli adopt internal features resonant with known properties of retinal processing, such as feedforward inhibition and contrast adaptation. Interestingly, the inclusion of latent noise within intermediate layers allows the CNNs to model sub-Poisson variability observed in true retinal responses. This indicates that the CNNs not only model mean responses but also capture the dynamic range of neural variability, an essential aspect of biological realism.

Methodological Contributions

The authors leveraged a substantial dataset derived from recordings of the spiking activity of tiger salamander retinal ganglion cells in response to both natural scenes and high-resolution spatiotemporal white noise. Through rigorous experimentation, including meticulous model architecture search and optimization via ADAM, the paper establishes a robust framework for exploring neural encoding through deep learning. Additionally, the CNNs incorporated recurrent neural network (RNN) layers, which enabled them to model long-term dynamical features like adaptation over multiple seconds, enhancing their descriptive power.

Implications and Future Directions

The findings presented in this paper have substantial implications for the field of sensory neuroscience. The research demonstrates how advanced computational techniques can elucidate both known and novel aspects of sensory processing mechanisms. Functionally, the ability of CNNs to capture complex natural scene encoding suggests potential applications in neural prosthetics, where such models could be instrumental in designing devices that more accurately mimic human sensory processing.

Looking forward, the paper opens new avenues for exploring neural systems through the lens of deep learning. Future work could explore extending these modeling techniques to other sensory modalities or higher-level cognitive processes. Additionally, integrating interpretability methods into these neural network models could provide more direct insights into the underlying circuit motifs and connectivity patterns, thus bridging the gap between machine learning models and biological neuroscience.

In summary, this paper not only advances the understanding of retinal responses to natural stimuli but also highlights how deep learning can be harnessed to uncover intricate neural computations across a broad range of sensory systems.