Circular Latent Encoding

• Circular latent encoding is a representational method that maps periodic and rotational features onto circular or toroidal manifolds for efficient and interpretable modeling.
• Methodologies include angular embeddings, circular spring loss, and wrapped geodesic metrics that enable smooth interpolation and uniform distribution in latent spaces.
• Applications span autoencoders, diffusion models, and Bayesian state-space models, providing disentangled representations for features such as hue, phase, and direction.

Circular latent encoding refers to a family of representational strategies in machine learning, probabilistic modeling, and generative frameworks in which the latent variables parameterizing a model are restricted to, or specifically structured as, circular or toroidal manifolds—for example, angles modulo $2\pi$, the product of $d$ circles (the $d$-torus $T^d$), or latent spaces where certain axes encode periodic quantities (e.g., hue, phase, or direction). Circular latent encoding is motivated by the need to faithfully and efficiently encode periodic, rotational, or cyclical features, yielding more interpretable, regularized, or disentangled internal representations in various model classes.

1. Foundational Principles of Circular Latent Encoding

The defining property of circular latent encoding is the imposition of circular (or more generally, toroidal) topology on part or all of the latent space. In practice, this entails representing one or more latent dimensions as angles $\theta \in [a, a+2\pi)$, or as vectors on the unit circle via $(\cos\theta, \sin\theta)$. This approach is directly applicable when modeling inherently periodic or rotational features, such as hue in color spaces, orientation in images, or states in circular time series.

In contrast to Euclidean encoding, circular encoding preserves the invariance and continuity of periodic spaces—eliminating discontinuities or artificial boundaries and enabling meaningful smooth interpolation. The structure supports both explicit regularizers (e.g., circular spring loss, toroidal priors) and implicit representations (e.g., latent PCA subspaces associated with circular manifolds).

2. Autoencoders and Uniform Circular/Toroidal Priors

Circular latent encoding is systematically explored in generative autoencoder designs, notably the "Toroidal AutoEncoder" (Mikulski et al., 2019). Here, the latent space is explicitly parameterized as a $d$-torus, $T^d$, with latent variables $\theta=(\theta_1,\ldots,\theta_d) \in [-\pi,\pi]^d$, each interpreted modulo $2\pi$. The angular variables are embedded into $\mathbb{R}^{2d}$ using $(x_i, y_i) = (\cos\theta_i, \sin\theta_i)$ for $i=1\dots d$, resulting in a latent vector $Z\in\mathbb{R}^{2d}$ that is decoded back to data space.

A key methodological innovation is the "circular spring loss," which enforces uniformity and cyclic boundary conditions along each angular dimension within a minibatch:

$$\mathcal{L}_{\text{spring}} = \sum_{i=1}^d\Bigl[\,\bigl(\varphi_i^{O_{i,1}} + 2\pi - \varphi_i^{O_{i,S}}\bigr)^2 + \sum_{s=1}^{S-1} \bigl(\varphi_i^{O_{i,s+1}} - \varphi_i^{O_{i,s}}\bigr)^2\,\Bigr].$$

This loss encourages the encoded angles for each minibatch to be equally spaced and wrap at the boundaries, realizing a uniform distribution over the $d$-torus.

Regularization can also be extended to the radii of the embedding (i.e., $r_i = \sqrt{x_i^2 + y_i^2}$) to enforce a fixed or controlled distribution, typically via a quantile-matching loss. Decoding always uses the raw $(x_i, y_i)$ latent vector regardless of radii regularization.

This parameterization admits interpretable and topology-consistent metrics such as wrapped geodesic distances:

$$d_{T^d}(\Theta^{(1)}, \Theta^{(2)}) = \sqrt{\sum_{i=1}^d \left(\Delta\theta_i\right)^2},$$

with

$$\Delta\theta = \left((\theta^{(1)}-\theta^{(2)}+\pi)\bmod 2\pi\right)-\pi.$$

Interpolation and morphing in the latent space exploit the toroidal topology, enabling multi-path interpolations that wrap through the periodic boundaries, supporting robust generative sampling and transformation of periodic or orientation features (Mikulski et al., 2019).

3. Circular Latent Encoding in Diffusion Models and Perceptual Spaces

Circular latent encoding arises naturally in complex generative models even when not explicitly enforced. An exemplary case is found in the analysis of the Stable Diffusion's latent space (Arias et al., 10 Dec 2025). Through controlled experiments with synthetic color stimuli, principal component analysis (PCA) of the 4D latent space reveals emergent, interpretable structure:

• Channels $c_3$ and $c_4$ serve as chromatic opponent axes, jointly encoding hue along a circular manifold.
• Mean-pooled latent vectors from color images, projected into the subspace spanned by principal components $\text{PC}_2$ and $\text{PC}_3$ (dominated by $c_3$ and $c_4$), form a hue wheel: The latent hue angle is computed as $\theta_i = \mathrm{atan2}(\beta_i, \alpha_i)$, where $\alpha_i = e_2^\top z_i$, $\beta_i = e_3^\top z_i$. Comparison with the original HSV hue establishes a strong circular correlation, confirming the representation's periodic structure.

PCA eigenvalues indicate that three directions capture nearly all color variance: $$(\lambda_1, \lambda_2, \lambda_3, \lambda_4) \approx (0.5463, 0.3172, 0.1348, 0.0018)$$. Notably, $\text{PC}_1$ encodes luminance and shape, while $\text{PC}_2$ and $\text{PC}_3$ underlie chromaticity, and their joint plane is circularly organized.

This suggests that, despite the absence of an explicit circular constraint, efficient coding and the statistics of natural images lead VAEs within diffusion models to organize perceptual attributes in a partially disentangled, circular-opponent format: intensity/shape ($c_1, c_2$) versus hue ($c_3, c_4$) (Arias et al., 10 Dec 2025).

4. Bayesian Nonparametric State Space Models with Circular Latent Variables

Circular latent encoding is also central in probabilistic modeling involving time series and dynamic state spaces where latent states correspond to angular variables. Mazumder & Bhattacharya develop fully nonparametric Bayesian models where both observation and evolution are functions of latent circular variables (Mazumder et al., 2014, Mazumder et al., 2016):

• State evolution and observation equations: For latent states $x_t\in[0,2\pi)$ and observations $y_t$,

$$\begin{aligned} y_t &= \bigl(f(t, x_t) \oplus \varepsilon_t\bigr)[2\pi],\ x_t &= \bigl(g(t, x_{t-1}) \oplus \eta_t\bigr)[2\pi], \end{aligned}$$

where $f$ and $g$ are unknown circular-valued functions, noise terms $\varepsilon_t$ and $\eta_t$ are Gaussian, and $[2\pi]$ denotes reduction modulo $2\pi$.

• Wrapped Gaussian process priors: Nonparametric priors are constructed by first positing Gaussian processes for $f^*$, $g^*$ on $\mathbb{R}\times [0,2\pi)$ (using kernels such as $\exp\{-\sigma^4 (t_1-t_2)^2\}\cos(|z_1-z_2|)$), and then mapping to circular-valued functions by reducing mod $2\pi$. The wrapped GP yields circular latent transitions and observation mappings.
• Efficient posterior inference: MCMC methods combine Gibbs sampling and Metropolis-Hastings, using auxiliary integer "winding" variables to maintain tractable wrapped likelihoods. A look-up table (fixed grid) approach is used for efficient evaluation and interpolation of potentially infinite-dimensional GPs (Mazumder et al., 2016).
• Latent representation: Latent angles are stored in $[0, 2\pi)$, entering models directly or via a basis $h(t,x) = (1, t, \cos x, \sin x)'$; the circular structure is preserved throughout.

Applications include modeling wind direction, biological cycles, and animal migration—domains where latent or observed quantities are intrinsically circular.

5. Theoretical Implications: Topology, Disentanglement, and Efficient Coding

Circular latent encoding ensures that machine learning models correctly capture the topology of inherently periodic features, mitigating artifacts present when using Euclidean spaces for circular variables. Key implications include:

• Disentanglement: In both the VAE latent space of Stable Diffusion and toroidal autoencoder constructions, circular subspaces (e.g., hue) are orthogonal to linear/subspace components (e.g., luminance, shape), reflecting natural statistic separability. This factorization often emerges as a consequence of the statistical independence of luminance and hue in real images and the structure of reconstruction objectives (Arias et al., 10 Dec 2025).
• Efficient coding: The structure of circular latent encodings mirrors classic opponent-process and efficient coding theories in biological vision, where color is encoded along trichromatic axes with periodic hue geometry. The emergence of circular subspaces via PCA in generative models supports a normative account rooted in natural image statistics (Arias et al., 10 Dec 2025).
• Interpolation and geodesics: Circular encoding on the torus $T^d$ enables shortest-path interpolation (geodesics) and multi-path morphing, utilizing the periodic covering space:

$$\Phi(t) = (1-t)\Phi^{(1)} + t(\Phi^{(2)} + 2\pi k),$$

for any integer vector $k\in\mathbb{Z}^d$, thus supporting wrap-around transitions (Mikulski et al., 2019).

A plausible implication is that circular latent encoding enables efficient manipulation, control, and regularization in generative and inference models operating on periodic or orientation-invariant features.

6. Practical Implementations and Applications

The practical realization of circular latent encoding includes:

• Toroidal AutoEncoders: Employ convolutional encoders and decoders, cos/sin embeddings for each angular latent, circular spring and quantile matching losses, and optional auxiliary classification heads on angles. Decoding occurs in the original Cartesian embedding (Mikulski et al., 2019).
• Analysis of Stable Diffusion VAEs: Utilizes synthetic datasets, PCA, and channel-wise ablation to empirically probe and quantify latent subspace geometry—demonstrating that controlled manipulations of certain latent axes can be used for semantically meaningful editing (e.g., hue rotation versus shape modification) (Arias et al., 10 Dec 2025).
• Bayesian time series modeling: Employs wrapped GP priors, look-up tables, and winding variable augmentation for robust inference on time-dependent latent angles, demonstrated in wind direction, tidal cycles, and animal navigation data (Mazumder et al., 2014, Mazumder et al., 2016).

Applications span generative model control, orientation- and hue-invariant representations, periodic time series modeling, quantization schemes compatible with circular symmetry, and data compression on toroidal supports.

7. Extensions and Related Topological Encodings

Circular latent encoding generalizes to higher dimensions (tori $T^d$), spherical manifolds ($S^2$ for 3D orientation), and even the representation of special orthogonal groups (e.g., $SO(3)$ via quaternions), each requiring adapted encodings and regularization strategies.

Different topologies can be enforced by construction (embedding and regularization) or discovered empirically (e.g., via latent space exploration, PCA). The methodology is extensible to non-circular periodic features (time-of-day, biological cycles) and other domains necessitating topologically nontrivial latent spaces.

---

For detailed formulations, empirical results, and proofs refer to Mazumder & Bhattacharya (Mazumder et al., 2016, Mazumder et al., 2014), Sidorov et al. (Mikulski et al., 2019), and the latent analysis of Stable Diffusion models (Arias et al., 10 Dec 2025).