Asymmetric Norms in State Embedding

• Asymmetric Norms are generalized norms that relax symmetry to capture intrinsic directional costs in statistical manifolds and control systems.
• They are constructed using divergence-induced measures, such as the KL divergence, and leverage parametric architectures like WideNorm for state embeddings in MDPs.
• Empirical evaluations demonstrate that these norms enhance planning performance by accurately modeling irreversible dynamics and directional transition costs.

Asymmetric norms are generalizations of classical norms that relax the requirement of symmetry, $\|x\| = \|-x\|$, thereby enabling faithful mathematical modeling of settings where natural “directions” or “arrows” exist in the underlying geometry or dynamics. In the context of state embedding for statistical manifolds and Markov decision process (MDP) models, asymmetric norms quantify non-reversible behavior and directional costs, ranging from information geometry (notably Kullback–Leibler divergence-induced topologies) to action-based distances in reinforcement learning.

1. Formal Definition and Construction of Asymmetric Norms

Given a vector space $V$, an asymmetric quasi-norm is a functional $p : V \to [0, \infty)$ satisfying positive homogeneity ($p(\lambda x) = \lambda p(x)$ for $\lambda>0$) and subadditivity ($p(x+y) \leq p(x) + p(y)$), but not generally symmetry ($p(x) \neq p(-x)$). For statistical manifolds, let $X$ and $Y$ be dual via $\langle x, y\rangle$ (with $V$0 representing random variables and $V$1 measures). Suppose $V$2 is closed and subdifferentiable. Define the divergence:

$V$3

Two dual asymmetric seminorms then arise:

• On $V$4:

$V$5

• On $V$6:

$V$7

These retain positive homogeneity and subadditivity, but not symmetry, so the resulting metric:

$V$8

is generally a quasipseudometric. In embedding contexts, parametric variants are also used, e.g., the WideNorm (Steccanella et al., 2023):

$V$9

where $p : V \to [0, \infty)$0 is a learned weight matrix and $p : V \to [0, \infty)$1 is the concatenation of $p : V \to [0, \infty)$2; ReLU enforces asymmetry.

2. Asymmetric Norms Induced by Information Divergences

A canonical realization employs Bregman-type divergences on probability manifolds, notably the Kullback–Leibler (KL) divergence (Belavkin, 2015). Let $p : V \to [0, \infty)$3 on $p : V \to [0, \infty)$4 (positive measures). The Bregman divergence,

$p : V \to [0, \infty)$5

induces sublevel sets: \begin{align*} M &= {u = y-z : D_{KL}[y,z] \leq 1} \subset Y-z, \ N &= {x : D_{KL}^*[x,0] \leq 1} \subset X. \end{align*} The resulting asymmetric “norms” are:

• On $p : V \to [0, \infty)$6

$p : V \to [0, \infty)$7

• On $p : V \to [0, \infty)$8

$p : V \to [0, \infty)$9

The asymmetry reflects the fundamental non-reversibility of the KL divergence: $p(\lambda x) = \lambda p(x)$0 generally, mirrored by $p(\lambda x) = \lambda p(x)$1.

3. Embedding Methods and Parametric Architectures

For state embedding in MDPs, the WideNorm construction (Steccanella et al., 2023) provides a learnable asymmetric semi-norm. A parametric state encoder $p(\lambda x) = \lambda p(x)$2 (typically an MLP or CNN) is paired with a learnable weight matrix $p(\lambda x) = \lambda p(x)$3. Given two states $p(\lambda x) = \lambda p(x)$4, the distance is:

$p(\lambda x) = \lambda p(x)$5

Because ReLU is not equivariant under sign swapping, $p(\lambda x) = \lambda p(x)$6 in general.

The embedding function and WideNorm parameters are trained end-to-end by stochastic gradient descent to approximate task-specific quasidistances, e.g., minimum action distances in MDPs.

4. Quasimetric Topologies and Topological Properties

Given a positively homogeneous, subadditive but asymmetric functional $p(\lambda x) = \lambda p(x)$7, the quasipseudometric $p(\lambda x) = \lambda p(x)$8 yields a topology with closed balls:

$p(\lambda x) = \lambda p(x)$9

The induced topology is non-symmetric: neighborhoods and convergence are “one-sided.” In the KL case (Belavkin, 2015), both $\lambda>0$0 (random variables) and $\lambda>0$1 (measures) equipped with their respective asymmetric norms become Hausdorff (T₂), T₁ (nonzero vectors have positive “length”), and $\lambda>0$2-sequentially complete (nested closed sets with vanishing diameter have nonempty intersection; Cauchy sequences converge). Classical Orlicz subspaces (with even gauge functions) embed densely and metrically inside these asymmetric normed topologies.

5. Approximation and Applications in Control, Inference, and Planning

The asymmetry of quasipseudometrics provides a faithful model when transition costs or information flows are direction-dependent. In reinforcement learning, asymmetric norms enable the approximation of the minimum action distance (MAD), defined as the least number of actions needed to transition between two states in an MDP, which is inherently non-symmetric for environments with, e.g., drift or irreversible dynamics (Steccanella et al., 2023).

The objective is to train $\lambda>0$3 and $\lambda>0$4 such that $\lambda>0$5 approximates $\lambda>0$6. Supervision comes from trajectory-derived temporal distances $\lambda>0$7. The loss function,

$\lambda>0$8

drives the learned WideNorm to match trajectory distances and upper-bound single-step transitions. No explicit symmetry or positive semi-definiteness constraints are imposed.

Once learned, the embedding supports differentiable goal-conditioned planning: a transition model $\lambda>0$9 is learned so that $p(x+y) \leq p(x) + p(y)$0. Action selection for goal-reaching exploits the asymmetric embedding metric,

$p(x+y) \leq p(x) + p(y)$1

6. Empirical Evaluation and Comparative Analysis

In environments with symmetric transitions, both symmetric ($p(x+y) \leq p(x) + p(y)$2) and asymmetric (WideNorm) parameterizations yield low mean squared error (MSE) to ground-truth $p(x+y) \leq p(x) + p(y)$3 and high planning success. In asymmetric environments—such as PointMass with drift or one-way wind—$p(x+y) \leq p(x) + p(y)$4 saturates at nonzero MSE (effectively learning $p(x+y) \leq p(x) + p(y)$5) and suffers reduced planning success (≈50%), whereas WideNorm achieves MSE approaching zero and sustains ≈90–100% planning success, matching the optimal number of actions (Steccanella et al., 2023). This consistently demonstrates the advantage of asymmetric norms when inherent directional costs or irreversibilities are present.

7. Embedding of Distributions and Implications

Equipping the statistical manifold of distributions with an asymmetric KL-norm provides a non-symmetric linear embedding into a topological vector space where the local structure encodes the forward divergence. The dual space inherits an asymmetric norm from the reverse KL, reflecting the one-way behavior of information flows. Unlike Orlicz–Banach space embeddings—which require even gauge functions and thus symmetrize—the asymmetric normed setting preserves the natural “wedge” structure, crucial for modeling cost/utility with one-sided boundedness and non-finite cumulant generating functions (Belavkin, 2015). This approach enables vector-space–style embeddings for states in control and inference tasks, retaining the intrinsic arrow of information.

A plausible implication is that asymmetric norm constructions offer a unified mathematical language for embedding spaces where symmetry would erase critical directional information, facilitating advances in control, planning, and statistical inference in environments characterized by irreversibility, non-reversibility, and asymmetry.