Log-Polar Space Convolution

• Log-Polar Space Convolution is a method that applies convolutional operations in log-polar coordinates, mimicking foveated vision with high central resolution and coarse peripheral sampling.
• It integrates a differentiable log-polar transform with recurrent attention mechanisms to efficiently extract and aggregate multi-scale visual features.
• This approach enhances robustness against geometric transformations and adversarial attacks while offering computational efficiency in classification and reinforcement learning tasks.

A Recurrent Attention Model with Log-Polar Mapping (RAM-LPM) is a bio-inspired vision architecture that extracts and processes sequences of log-polar “glimpses” from input images, emulating foveated vision and saccadic attention mechanisms observed in the primate visual system. This design confers inherent robustness to geometric transformations, computational efficiency, and—in some variants—increased resistance to adversarial attacks. RAM-LPM comprises a differentiable glimpse-sensor based on the log-polar transform, a recurrent attention mechanism to select fixation points, and a policy for adapting the focus of attention over time based on task objectives. It has been realized in classification, reinforcement learning, and adversarially robust settings, as exemplified by models in RetinotopicNet (Kurbiel et al., 2020), adversarially robust RAM-LPM (Kiritani et al., 2020), and log-polar gaze control for RL agents (Göransson et al., 2023).

1. Log-Polar Mapping: Formulation and Properties

The log-polar transform maps Cartesian image coordinates $(x, y)$ to log-polar coordinates $(\rho, \varphi)$ relative to a chosen pole $(x_c, y_c)$:

$$r = \sqrt{(x - x_c)^2 + (y - y_c)^2}, \qquad \varphi = \mathrm{atan2}(y - y_c, x - x_c),$$

$\rho = \log r.$

The inverse mapping reconstructs pixel locations for differentiable sampling:

$$x = e^{\rho}\cos\varphi + x_c, \quad y = e^{\rho}\sin\varphi + y_c.$$

The log-polar patch samples the input image with high resolution at the center and logarithmically coarser resolution toward the periphery, mimicking retinal foveation (Kurbiel et al., 2020). Log-polar sampling induces equivariance to global scale and rotation, e.g.,

$$(\rho, \theta) \mapsto (\rho + \log s,\, \theta + \phi)$$

for image scale $s$ and rotation $\phi$ (Kiritani et al., 2020).

Some RAM-LPM variants use stacked rectangular windows with increasing subsampling rates emulating log-polar properties, called “log-polar–like” sampling (Göransson et al., 2023). All methods result in fixed-size tensors encoding both fine central and coarse contextual information.

2. Model Architecture and Variants

A. Core Components

• Glimpse Sensor (Log-Polar Retina): At each timestep, a fixed-size log-polar patch centered at $(x_t, y_t)$ is sampled and regridded.
• Feature Extraction: Glimpse patches are processed by convolutional (CNN) backbones, often with padding choices reflecting the topology of log-polar coordinates—wrap-around for angular and reflect for radial axes.
• Recurrent Aggregation: Features from multiple timesteps are integrated using either vanilla RNNs (Kurbiel et al., 2020) or LSTMs (Kiritani et al., 2020, Göransson et al., 2023).
• Attention/Localization Network: Computes the policy for next fixation. Techniques include deterministic coordinate regression (spatial softmax over CNN maps) (Kurbiel et al., 2020), or stochastic Gaussian policies trained via REINFORCE (Kiritani et al., 2020).
• Classifier Head: Final prediction is made by a lightweight MLP on the aggregated recurrent state.

B. Major Architectural Variants

Model	Sampling	Attention Policy	Aggregation	Task
RetinotopicNet (Kurbiel et al., 2020)	Log-polar	Deterministic (softmax)	Vanilla RNN	Classification
Adversarial RAM-LPM (Kiritani et al., 2020)	Log-polar	Stochastic (REINFORCE)	Two-pathway LSTMs	Robust classification
RL Log-polar (Göransson et al., 2023)	Log-polar–like	A3C-LSTM RL	LSTM	RL + gaze control

RetinotopicNet uses a spatially differentiable, deterministic softmax-based saccade mechanism, while Adversarial RAM-LPM and RL variants leverage policy-gradient-based stochastic attention.

3. Recurrent Attention and Gaze Policies

In RAM-LPM, classification or control arises from sequences of glimpses centered at algorithmically selected locations:

• At each step $(\rho, \varphi)$0, state variables $(\rho, \varphi)$1 designate the next fixation center.
• Feature maps from the glimpse feed into the attention network:
  • RetinotopicNet: score map $(\rho, \varphi)$2 via $(\rho, \varphi)$3 conv layers, spatial softmax to $(\rho, \varphi)$4, next $(\rho, \varphi)$5 mapped back to Cartesian (Kurbiel et al., 2020).
  • Adversarial RAM-LPM/RL: LSTM outputs $(\rho, \varphi)$6 as means of a Gaussian policy, with sampling for exploration and robustness (Kiritani et al., 2020, Göransson et al., 2023).
• The attending process is recurrent: features $(\rho, \varphi)$7 or $(\rho, \varphi)$8 are aggregated (RNN/LSTM); after $(\rho, \varphi)$9 steps, outputs are used for the main task (classification, reinforcement learning, etc.).
• In some RL implementations, separate policy heads govern “natural” and “visual” (gaze) actions, each with its own policy/value losses (Göransson et al., 2023).

4. Training Methodologies

A. Supervised Learning

• Deterministically Differentiable Attention: All components, including the glimpse sampler, are differentiable. End-to-end backpropagation with standard cross-entropy loss suffices for training (Kurbiel et al., 2020).
• Two-stage Pretraining: In RetinotopicNet, the localization net is pretrained for single saccades before full recurrence is unrolled (Kurbiel et al., 2020).

B. Reinforcement/Policy Gradient Learning

• Stochastic Hard Attention: Where glimpse selection is non-differentiable (sampling steps), REINFORCE with learned baselines stabilizes gradient estimates for $(x_c, y_c)$0 decisions (Kiritani et al., 2020). Total loss combines task reward (classification, RL objective) and attention policy loss.
• RL-Gaze Control with A3C: The visual/gaze policy is trained alongside natural action in an actor-critic framework, often using Generalized Advantage Estimation (GAE) for credit assignment (Göransson et al., 2023).

5. Empirical Performance and Invariance

<table> <thead> <tr><th>Dataset/Task</th><th>Performance</th><th>Notable Findings</th></tr> </thead> <tbody> <tr> <td>CIFAR-10 (classification)</td> <td>> 80% accuracy (T=4, 32×32 patch)</td> <td>Accuracy robust to image size and patch size doubling yields marginal improvements (Kurbiel et al., 2020)</td> </tr> <tr> <td>MNIST (classification)</td> <td>> 99% accuracy</td> <td>Multipoint aggregation critical; single glimpse loses ~10% on cluttered scenes (Kurbiel et al., 2020)</td> </tr> <tr> <td>RL Atari (Breakout, etc.)</td> <td>Performance matches full-frame baseline (<em>≈ident.</em>)</td> <td>RAM-LPM reduces pixels seen 5× (6400→1225) without quality loss, accelerates learning with peripheral glimpses (Göransson et al., 2023)</td> </tr> <tr> <td>SIM2MNIST (transformed)</td> <td\>5.00% error</td> <td>Rotation and scale equivariance matches polar-transform SOTA (Kiritani et al., 2020)</td> </tr> </tbody> </table>

• Rotation and scale invariance are empirically confirmed by negligible accuracy loss under these transformations; angular wrap-padding is critical (dropping it degrades rotation equivariance by $(x_c, y_c)$15%) (Kurbiel et al., 2020).
• Diminishing returns are observed beyond $(x_c, y_c)$2 saccades (Kurbiel et al., 2020).
• Peripheral vision and wide low-res context accelerate RL learning, and a large low-res field of view is more impactful than tiny high-res fovea in task performance (Göransson et al., 2023).

6. Robustness and Computational Properties

• Adversarial Robustness: RAM-LPM demonstrates significant resistance to adversarial attacks (e.g., SPSA, PGD) relative to baseline CNNs. Success rates for attacks are much lower on RAM-LPM, especially under non-cropped (full-size) images, where ResNet-50 fails on ~90% of $(x_c, y_c)$3 PGD attacks, but RAM-LPM is markedly more robust (Kiritani et al., 2020).
• Mechanisms of Robustness:
  • Gradient Masking: Non-differentiable/stochastic sampling obscures attack gradients.
  • Limited Visibility: Each adversarial step confronts only the current glimpse.
  • Recurrent Integration: Adversarial perturbations must persistently fool the model across a sequence of different glimpses, raising complexity exponentially.
• Computational Efficiency: Glimpses are fixed-size ($(x_c, y_c)$4); total computation scales with $(x_c, y_c)$5, independent of input size (Kiritani et al., 2020). This enables fixed FLOPs per sample and efficiency gains in high-resolution settings and on embedded hardware.
• Low-latency Pathways: Reflex-like shallow paths can supply rapid, coarse predictions from the initial glimpse, analogous to biological reflex arcs (Kiritani et al., 2020).

7. Biological Motivation, Extensions, and Limitations

The RAM-LPM framework is grounded in neurobiological principles:

• Foveated Sampling: Mirrors the exponential falloff in retinal sampling density (center-to-periphery).
• Saccadic Attention: Models iterative, task-driven gaze shifts.
• Parallel “What” and “Where” Pathways: Functional partitioning as observed in dorsal and ventral streams of primate vision (Kiritani et al., 2020).

Extensions under consideration include:

• Continuous (rather than discrete) parametrizations of gaze (Göransson et al., 2023).
• Application of true log-polar sampling beyond rectangular approximations, enhancing invariance and context aggregation.
• Broadening application space: object detection, robotic vision, and highly resource-constrained embedded systems.

Limitations identified include:

• Higher sample complexity: learning effective attention and control policies is accompanied by slower training and partial observability (Göransson et al., 2023).
• Joint gaze-action state–action spaces increase the burden on RL optimization.
• In some implementations, approximate rather than exact log-polar transforms limit theoretical equivariance (Göransson et al., 2023).

RAM-LPM thus provides a versatile, efficient, and robust framework for attention-based visual reasoning, grounded in computational neuroscience and validated across classification, adversarial, and reinforcement learning settings (Kurbiel et al., 2020, Kiritani et al., 2020, Göransson et al., 2023).