SigLIP2-base-16b-512: Invariant Feature Model

• SigLIP2-base-16b-512 is a model that employs algebraic invariants from iterated-integral signatures to generate invariant and compact representations of sequential data.
• It integrates tensor and free Lie algebra structures with computational tools like iisignature and C++ libraries to efficiently compute log signatures for feature extraction.
• The model enhances machine learning pipelines by providing robust, translation and reparametrization-invariant features, improving performance in time series and trajectory analysis.

SigLIP2-base-16b-512 refers to a model architecture and feature pipeline in which path or sequential data are represented through algebraic invariants derived from iterated-integral signatures and their logarithms. This approach combines advanced algebraic machinery—tensor algebras, free Lie algebras, and canonical Lyndon word bases—with efficient computational routines to produce compact, translation and reparametrization-invariant numerical features for machine learning models. In practical deployments, these log signatures enable richer structural representations of high-dimensional sequential data, such as time series or trajectories, and serve as the input to models like SigLIP2-base-16b-512, which may denote a configuration with a 16-dimensional input space and a 512-dimensional hidden or output representation.

1. Iterated-Integral Signatures in ℝᵈ

Given a continuous path $\gamma: [a, b] \rightarrow \mathbb{R}^d$, the signature of the path, $X^\gamma_{[a, b]}$, is an element of the tensor algebra $T(\mathbb{R}^d)$. It is defined as the sequence of all possible iterated integrals along the path:

$$X^\gamma_{[a, b]}(i_1i_2\ldots i_n) = \int_a^b \int_a^{t_1}\ldots\int_a^{t_{n-1}} d\gamma_{i_n}(t_n)\ldots d\gamma_{i_1}(t_1)$$

where $i_j \in \{1,\ldots,d\}$. The level-1 (n=1) component of the signature recovers the displacement in each dimension, $X^\gamma_{[a, b]}(i) = \gamma_i(b) - \gamma_i(a)$. These signatures are invariant under translation and reparametrization, satisfying Chen’s identity for concatenated paths:

$$X^{\gamma, \leq m}_{a,c} = X^{\gamma, \leq m}_{a,b} \cdot X^{\gamma, \leq m}_{b,c}$$

with truncation at level $m$. For straight-line segments, the signature can be written as a truncated exponential series:

$$X^{\gamma, \leq m}_{a,b} = \exp(\gamma(b) - \gamma(a))$$

2. Algebraic Structure and Log Signatures

The log signature provides a compressed, non-redundant form of the signature. Formally, the truncated log signature is defined by:

$$Y^\gamma_{[a, b]} = \rho^{-1}(\log(X^{\gamma, \leq m}_{a,b}))$$

where $\rho$ embeds the free Lie algebra $F^{\leq m}(\Sigma)$ into the truncated tensor algebra $T^{\leq m}(\mathbb{R}^d)$. The logarithm is computed via the power series:

$$\log(1 + x) = x - \frac{1}{2}x^2 + \frac{1}{3}x^3 - \ldots + (-1)^{m+1}\frac{1}{m}x^m$$

Basis elements for the free Lie algebra are constructed from Lyndon words via recursion: $\sigma(a) = a$ for letters $a$, and for a Lyndon word $w=uv$ (where $v$ is the longest proper Lyndon suffix), $\sigma(w) = [\sigma(u), \sigma(v)]$.

For assembling log signatures over concatenated segments, the Baker–Campbell–Hausdorff (BCH) formula is applied:

$$\log(\exp(x)\exp(y)) = x + y + \frac{1}{2}[x,y] + \frac{1}{12} [x, [x, y]] - \frac{1}{12} [y, [x, y]] + \ldots$$

This yields the recursive update for concatenated paths:

$$Y^{\gamma}_{a,c} = \log(\exp(Y^{\gamma}_{a,b}) \exp(Y^{\gamma}_{b,c}))$$

3. Computational Tools for Log Signature Calculation

Numerical evaluation of log signatures relies on dedicated software tools:

• iisignature (Python): Generates, for fixed $m$ and $d$, code to compute log signatures efficiently. Uses Lyndon word classes and precomputed BCH coefficients (e.g., bchLyndon20.dat). Supports recursive assembly of log signatures over segments for fast feature computation.
• C++ Library: Offers high-performance routines, with bch.h and bch.cpp for efficient log signature aggregation. Suitable for real-time and resource-constrained settings.
• Mathematica Visualization: Permits interactive exploration of log signature components (e.g., in $\mathbb{R}^2$ for $m\leq 4$), illustrating the geometric significance of invariants such as signed area.

These computational approaches enable the integration of log signature features into large-scale machine learning pipelines, facilitating rapid extraction and representation of complex paths.

4. Log-Signature Feature Engineering in Model Architectures

In models such as SigLIP2-base-16b-512, log signatures serve as the primary feature representation for sequential or high-dimensional input data. The workflow typically includes:

1. Input Representation: Time series or paths are encoded as sequences of displacements in $\mathbb{R}^{16}$.
2. Feature Computation: Log signatures up to level $m$ are computed using iisignature (Python) or the C++ routines.
3. Model Input: The resulting log signature—a lower-dimensional, translation and reparametrization invariant vector—is used as model input, often feeding into a hidden or output layer of size 512.

The BCH formula ensures non-commutative interactions between input dimensions are preserved, encoding geometric properties (such as generalized area and higher-order shape descriptors) that are beneficial for sequence prediction and classification tasks. This approach provides a compact yet structurally rich alternative to raw or pointwise features.

5. Practical Example: Log Signature in $\mathbb{R}^2$

Consider a path in $\mathbb{R}^2$. The first-level log signature component represents the displacement; the second-level, a commutator $[1,2]$ in the free Lie algebra, yields (up to a factor) the signed area between the path and its chord. Expanding the logarithmic series on the signature retrieves these geometric invariants, showing how higher-order features are informed by integrated path traversal properties rather than pointwise locations.

In machine learning contexts, this invariant and compressed representation improves discriminative power for data where the relational structure (rather than absolute position) encodes key information.

6. Integration and Implications for Sequential Data Analysis

The theoretical basis for using log signatures in models like SigLIP2-base-16b-512 lies in the interaction of tensor algebra, free Lie algebra structure, and Lyndon word bases, as formalized in Chen’s identity and the BCH approach. Computational tools developed for signature calculus (Python, C++, Mathematica) operationalize this theory into scalable feature engineering methods.

A plausible implication is that such log signature features, by encoding path geometry and higher-order interactions efficiently, enable machine learning models to generalize more robustly to invariances and perturbations in sequential data. This may bolster performance in time series analysis, trajectory classification, and other path-driven tasks where invariance, compactness, and geometric fidelity are crucial.

7. Key Formulas and Conceptual Summary

Formula	Description
$X^\gamma_{[a, b]}(i_1\ldots i_n)$	n-fold iterated integral component
$\exp(x) = 1 + x + \frac{1}{2}x^2 + \ldots$	Exponential for straight-line segments
$\log(1 + x)$	Log signature series expansion
BCH formula	Combination rule for log signatures

The SigLIP2-base-16b-512 approach demonstrates how deep algebraic theory, when combined with numerical software, enables the extraction of invariant, compressed, and geometry-aware features for machine learning, especially in tasks requiring robust treatment of sequential or path-dependent data.