Retinotopic Mapping Enhances the Robustness of Convolutional Neural Networks (2402.15480v2)

Abstract: Foveated vision, a trait shared by many animals, including humans, has not been fully utilized in machine learning applications, despite its significant contributions to biological visual function. This study investigates whether retinotopic mapping, a critical component of foveated vision, can enhance image categorization and localization performance when integrated into deep convolutional neural networks (CNNs). Retinotopic mapping was integrated into the inputs of standard off-the-shelf convolutional neural networks (CNNs), which were then retrained on the ImageNet task. As expected, the logarithmic-polar mapping improved the network's ability to handle arbitrary image zooms and rotations, particularly for isolated objects. Surprisingly, the retinotopically mapped network achieved comparable performance in classification. Furthermore, the network demonstrated improved classification localization when the foveated center of the transform was shifted. This replicates a crucial ability of the human visual system that is absent in typical convolutional neural networks (CNNs). These findings suggest that retinotopic mapping may be fundamental to significant preattentive visual processes.

• The paper demonstrates that integrating retinotopic mapping into CNNs improves robustness against geometric distortions such as rotations and zooms.
• It leverages a logarithmic-polar transform to simulate foveated vision, converting complex image transformations into simpler translations.
• Empirical tests on Vgg16 and ResNet101 reveal that the approach yields comparable or improved classification and localization performance.

Enhancing Convolutional Neural Networks with Retinotopic Mapping

Introduction to Retinotopic Mapping in CNNs

In an exploration of improving image classification and localization tasks, a novel approach integrating retinotopic mapping into Convolutional Neural Networks (CNNs) is presented. This paper focused on adjusting CNNs' input phase with a retinotopic transform—a feature inspired by the foveated vision common in many animal species. The methodology involved retraining widely recognized CNN architectures with images transformed via logarithmic-polar mapping, aiming to examine the impact on the networks' ability to handle image rotations and scale changes.

Core Findings and Experimental Insights

Robustness to Geometric Perturbations

A striking outcome of the paper is the demonstration that CNNs, when adjusted with retinotopic mapping, exhibit enhanced robustness against geometric transformations of input images, specifically zooms and rotations. This finding contrasts with the common vulnerability of traditional CNNs to adversarial attacks, highlighting the potential of retinotopic mapping in crafting more resilient image recognition systems. These vanilla CNNs were shown to significantly falter under rotation-based attacks, underlining the importance of considering realistic perturbations in model evaluations.

Advantages of Biologically Inspired Mapping

The research underpins the computational benefits of biologically inspired visual processing models. By mimicking the human visual system's retinotopic organization, where a logarithmic scale dictates the distribution of photoreceptors in the retina, the transformed input to CNNs showcases an inherent invariance to scale and rotation. This is attributed to the property of the log-polar transform, which converts zooms and rotations in the visual space into simpler translations in the transformed space, thereby leveraging the convolutional layers' translation invariance in a novel manner.

Empirical Validation and Theoretical Implications

The experimental validation involved retraining the Vgg16 and ResNet101 networks on the ImageNet dataset with the retinotopic transformation engaged. Remarkably, these retinotopically mapped networks achieved comparable, and in some dimensions improved, performance against their traditionally trained counterparts. This result is non-trivial, as it suggests that despite significant distortion and information compression inherent in the retinotopic mapping, networks can still extract and utilize relevant features for accurate classification. Additionally, the paper delved into object localization capabilities, finding improved localization performance when analyzing images with varying focal points. This mirrors a critical aspect of human visual attention not previously encapsulated by standard CNN designs.

Future Directions in AI and Robust Vision Systems

The integration of retinotopic mapping opens up new avenues for research in geometric deep learning and AI vision systems. It suggests a pathway towards developing models that not only match but potentially surpass human visual recognition capabilities under varied and challenging conditions. The paper's insights into the naturally robust preattentive processing mechanisms present an exciting frontier for improving AI systems' efficiency and resilience.

Moreover, this approach could inspire future models that incorporate dynamic foveation mechanisms, mimicking saccadic eye movements to iteratively focus on regions of interest within an image. Such models could vastly improve the computational efficiency and accuracy of tasks like visual search, further bridging the gap between artificial and biological vision systems.

In conclusion, the exploration of retinotopic mapping within CNNs marks a significant step towards harnessing biologically inspired principles to bolster AI robustness and capability. It not only demonstrates the practical advantages of such an approach in current architectures but also sets the stage for profound advancements in how machines perceive and understand the visual world.