Circular Task Embeddings in CCM-AAE

• Circular task embeddings are representations defined on the one-dimensional unit circle (S¹) that capture natural cyclic and phase-based relationships among tasks.
• The CCM-AAE framework encodes task data into a normalized Euclidean space, employing an adversarial and geometric loss to enforce the manifold constraint on the unit circle.
• This approach enables effective clustering, visualization, and generative exploration of tasks with inherent cyclical structures, enhancing interpretability and modeling accuracy.

Circular task embeddings are representations of tasks as points on a one-dimensional spherical manifold—specifically, the unit circle $S^1$—with the goal of capturing cyclic, phase-based, or angular relationships among tasks. Within the adversarial autoencoder framework over constant-curvature manifolds (CCM-AAE), circular task embeddings are constructed by encoding task-level information into the circle, leveraging the geometry of constant-positive curvature to enforce a "circular" code and facilitate geometric regularities and visualizations that are not easily captured in Euclidean space (Grattarola et al., 2018).

1. Constant-Curvature Manifolds and the Spherical Geometry

A constant-curvature Riemannian manifold $M$ parameterized by curvature $\kappa \in \mathbb{R}$ can be realized extrinsically in $\mathbb{R}^{d+1}$ as

$$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$

where $\langle \cdot, \cdot \rangle$ denotes the Euclidean dot product. In the case $\kappa > 0$, the manifold is a $d$-dimensional sphere $S^d_\kappa$ of radius $r = 1/\sqrt{\kappa}$.

Core geometric operations on the hypersphere include:

• Metric tensor: $M$0.
• Geodesic distance: $M$1.
• Exponential and logarithmic maps support transition between embedded points and their tangent representations, enabling smooth optimization and interpolations.

The case $M$2 yields the unit circle $M$3, a manifold particularly apt for cyclic or periodic structures such as phase data, directions, or task sequences with a natural circular symmetry.

2. CCM-AAE Framework for Circular Embedding

The CCM-AAE (Constant-Curvature Manifold Adversarial AutoEncoder) framework enables the construction of latent representations that explicitly reside on a specified constant-curvature manifold. For circular task embeddings, the latent space is instantiated by $M$4 (i.e., a circle in $M$5).

The architecture involves:

• Encoder $M$6: Maps input task data $M$7 to $M$8, optionally projecting the resulting embedding onto $M$9 by normalization.
• Decoder $\kappa \in \mathbb{R}$0: Reconstructs the input data from the embedding, enforcing that latent codes contain information sufficient for task reconstruction or prediction.
• Discriminator $\kappa \in \mathbb{R}$1: Distinguishes between real samples drawn uniformly from $\kappa \in \mathbb{R}$2 and encoder outputs, enforcing the aggregate posterior to match the manifold's prior.

Off-manifold encodings are penalized using a Gaussian membership function,

$\kappa \in \mathbb{R}$3

which quantifies how closely $\kappa \in \mathbb{R}$4 adheres to the sphere constraint.

3. Learning Objective and Training Procedure

The joint objective function incorporates reconstruction, membership, and adversarial losses: $\kappa \in \mathbb{R}$5 where $\kappa \in \mathbb{R}$6 is a standard adversarial objective comparing real and synthetic manifold samples.

The training alternates between discriminator optimization ($\kappa \in \mathbb{R}$7) and minimization over encoder and decoder parameters ($\kappa \in \mathbb{R}$8):

• The discriminator is updated to maximize the adversarial term, distinguishing true manifold samples from encoder outputs.
• The encoder and decoder are jointly updated to minimize reconstruction error, encourage manifold membership, and fool the discriminator.

For the circular case ($\kappa \in \mathbb{R}$9), the prior $\mathbb{R}^{d+1}$0 is sampled by drawing $\mathbb{R}^{d+1}$1 and normalizing, yielding uniform distribution on the unit circle.

4. Construction of Circular Task Embeddings

Circular task embeddings are built as follows:

1. Each task $\mathbb{R}^{d+1}$2 is encoded via $\mathbb{R}^{d+1}$3 to $\mathbb{R}^{d+1}$4, then normalized to $\mathbb{R}^{d+1}$5: $\mathbb{R}^{d+1}$6.
2. The decoder reconstructs $\mathbb{R}^{d+1}$7 from $\mathbb{R}^{d+1}$8, imposing that the embedding is informative with respect to task features or metadata.
3. The discriminator trains against both $\mathbb{R}^{d+1}$9 and independently sampled $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$0 Uniform($$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$1).
4. Objective terms and hyperparameters ($$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$2, $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$3) control the balance of reconstruction, manifold regularity, and adversarial alignment.

A typical optimization step involves:

• Computing the batch of task embeddings.
• Evaluating reconstruction loss: $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$4.
• Computing the membership penalty: $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$5.
• Updating the discriminator with real and fake samples.
• Updating encoder/decoder to minimize the joint loss, including adversarial, reconstruction, and geometric terms.

5. Hyperparameterization and Practical Considerations

Relevant settings for circular task embedding include:

• Embedding dimensionality $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$6, thus ambient dimension is 2.
• Curvature $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$7, setting manifold to the unit circle.
• Regularization strength $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$8.
• Membership width $$M = \left\{ x \in \mathbb{R}^{d+1} \mid \langle x, x \rangle = \kappa^{-1} \right\},$$9.
• Learning rate (e.g., $\langle \cdot, \cdot \rangle$0), batch size (64-256).

Training proceeds as:

• For each epoch and batch, encode, normalize, and compute losses as outlined.
• The embedding $\langle \cdot, \cdot \rangle$1 after convergence offers a phase (angle) representation per task, lending itself to clustering, visualization, or generative exploration of novel task codes along the circle.

6. Applications and Interpretational Advantages

Circular task embeddings provide a natural framework for representing periodic or cyclical task relationships, such as phases, rotations, or tasks with inherent circular orderings. After training, the mapping $\langle \cdot, \cdot \rangle$2 can be employed for clustering tasks, visualizing angular task progressions, and sampling new hypothetical tasks via interpolations along the circle.

The explicit manifold constraint ensures that embeddings retain circular topology, reducing distortion relative to Euclidean latent codes when the underlying task structure is naturally cyclic. This framework enables robust probabilistic modeling, regularization, and generative modeling within a geometric regime that aligns with the data's symmetries (Grattarola et al., 2018).