LogMap Layers: Mapping Lie Groups in Deep Learning

• Logarithm Mapping (LogMap) layers are neural modules that convert SO(3) rotation matrices into their Lie algebra (so(3)) via the principal matrix logarithm.
• They linearize manifold-valued features by mapping rotation matrices to Euclidean space, enabling standard deep learning layers for applications like 3D action recognition.
• Robust numerical methods, including safety branches and clamping, ensure stability during both forward and backward passes of the log-mapping process.

A Logarithm Mapping (LogMap) layer is a neural network module designed to map matrix Lie group-valued data, such as tuples of rotation matrices from SO(3), onto their associated Lie algebra, such as so(3), by applying the principal matrix logarithm in closed form. This operation linearizes manifold-valued features into a Euclidean vector space, thereby facilitating the application of conventional deep learning layers for subsequent processing and classification, especially in domains such as skeleton-based action recognition (Huang et al., 2016).

1. Mathematical Basis

Let each input skeleton frame be represented by the tuple

$$R = (R_1,\,\ldots,\,R_{\widehat{M}}) \in SO(3) \times \cdots \times SO(3)$$

where $\widehat{M} = 2 \cdot C_M^2$ encodes the number of ordered joint-pair rotations, and $SO(3)$ denotes the 3D rotation group. The associated Lie algebra is

$so(3) \times \cdots \times so(3)$

with each element a $3 \times 3$ real skew-symmetric matrix.

For any $R \in SO(3)$, the principal matrix logarithm $\log R$ yields the unique $X \in so(3)$ with $\|X\| < \pi$ such that $\exp(X) = R$. Operationally, using the axis–angle representation:

• Compute $\theta(R) = \arccos((\mathrm{tr}R - 1)/2)$.
• If $\theta(R) = 0$, set $\log R = 0$.
• Otherwise, compute

$$\log R = \frac{\theta(R)}{2 \sin\,\theta(R)} (R - R^T)$$

where $R - R^T$ is skew-symmetric and the scalar ensures $\exp(\log R) = R$. Spectral definitions using $R=U \Lambda U^T$ are numerically fragile and not used in practice.

2. Layer Structure and Network Integration

Given mini-batch tensors $$R^{(k-1)} \in \mathbb{R}^{B \times \widehat{M} \times 3 \times 3}$$, where each $R^{(k-1)}[b, m] \in SO(3)$:

• The LogMap layer independently applies the matrix logarithm to each $3 \times 3$ block, yielding $$X^{(k)} \in \mathbb{R}^{B \times \widehat{M} \times 3 \times 3}$$, with $X[b, m]\in so(3)$.
• Each skew-symmetric $X$ has three degrees of freedom; in subsequent layers, these may be compacted to a $B \times (\widehat{M} \times 3)$ array for standard fully-connected layers, or processed further as matrices in specialized matrix–FC layers.

Within the overall architecture, the canonical block arrangement is:

$$[\text{RotMap} \rightarrow \text{RotPooling}]^n \rightarrow \text{LogMap} \rightarrow (\text{ReLU}) \rightarrow \text{FC} \rightarrow \text{Softmax}$$

where LogMap serves as the final "manifold" layer. Its Euclidean outputs are compatible with standard deep network components.

3. Computational Implementation and Gradients

Forward Pass

For each $R \in SO(3)$:

• Compute $\theta = \arccos((\mathrm{tr}R - 1)/2)$.
• If $\theta \approx 0$, set $X = 0$; otherwise, $X = (\theta/(2\sin \theta))(R - R^T)$.
• Values of $\theta$ and $(R - R^T)$ are cached for the backward pass.

Backward Pass

Given upstream gradient $$G_X = \partial L / \partial X \in \mathbb{R}^{3\times 3}$$:

• Compute by the chain rule with respect to the Frobenius inner product, using

$$dX = g'(\theta)\,d\theta\cdot(R-R^T) + g(\theta)(dR-dR^T)$$

where $g(\theta) = \theta/(2\sin\theta)$ and $$g'(\theta) = (\sin\theta - \theta\cos\theta)/(2\sin^2\theta)$$.

• Using

$d\theta = -\frac{\mathrm{tr}(dR)}{2\sin\theta}$

• The resulting gradient with respect to $R$ is:

$$G_R = \frac{\theta}{2\sin\theta} (G_X^T - G_X) - \frac{\sin\theta - \theta\cos\theta}{4 \sin^3\theta} \left[\mathrm{tr}(G_X^T(R-R^T))\right] I$$

An alternative, more general formulation for the gradient leverages the Fréchet derivative of the matrix logarithm, expressed as a matrix integral, though in practice the analytic axis–angle version is preferred.

4. Transition to Euclidean Layers

Mapping the manifold-valued features from $SO(3)^{\widehat{M}}$ to $so(3)^{\widehat{M}}$ linearizes the data by situating it in a vector space of skew-symmetric matrices, thus nullifying the orthogonality and determinant constraints inherent to $SO(3)$. This operation enables subsequent layers—ReLU, fully connected, softmax—to treat the features as vectors, bypassing the geometric restrictions of the original manifold. Standard deep learning optimizations and classification techniques can then be applied directly, significantly enhancing flexibility and speed in model training (Huang et al., 2016).

5. Numerical Stability and Implementation

Special attention is given to numerical stability:

• For small $\theta$, $\sin\theta \approx \theta$ and $\theta/(2\sin\theta) \to 1/2$. A safety branch is implemented: if $\theta < \varepsilon$ (e.g., $10^{-3}$), then $\log R \approx (R - R^T)/2$, paralleling the first term in the Taylor expansion of the logarithm.
• The argument of arccos is explicitly clamped to $[0,\pi]$ to avoid NaNs from floating-point errors.
• Such measures are essential in any robust $SO(3)$ log/exp code path.

6. Significance and Context

The integration of LogMap layers addresses the mismatch between non-Euclidean manifold structures arising from action recognition representations and the flat geometry assumed by standard neural network layers. By projecting manifold features into the tangent space, the network can exploit standard deep learning functionality while preserving structural information from original Lie group data. This capability was demonstrated to outperform previous shallow Lie group feature learning and conventional deep learning methods in 3D human action recognition (Huang et al., 2016).