---
title: Strips as Tokens (SATO)
url: https://www.emergentmind.com/topics/strips-as-tokens-sato
type: topic
---

# Strips as Tokens (SATO)

Strips as Tokens (SATO) refers to a methodological paradigm in which structured sequences, known as "strips," serve as atomic units or tokens for autoregressive or non-autoregressive generative modeling. Originating in geometric deep learning for 3D mesh synthesis, the SATO abstraction has informed advances across mesh generation, visual tokenization, and symbolic world modeling. The central innovation is to leverage contiguous or semantic groupings—be they mesh face strips, visual latent strips, spatial strips in vision MLPs, or sequences of grounded actions in STRIPS planning—as the tokens provided to sequence models. This preserves structural coherence, improves compression, and aligns the generation process with natural semantics or workflow patterns [2604.09132, 2603.16373, 2307.11458, 2509.13389].

## 1. Foundational Principles and Definitions

In the SATO framework, a "strip" is a contiguous chain (or semantically grouped sequence) that encodes domain structure far beyond trivial coordinate-based or per-element tokenizations:

- In mesh generation, strips are vertex sequences defining triangle or quad strips, with explicit encoding of UV (parametrization) boundaries [2604.09132].
- In visual tokenization, SATO compresses a 2D grid of latent features into a 1D token sequence ("stripifying"), each token capturing global semantic context [2603.16373].
- In vision MLPs, horizontal or vertical strips comprise contiguous rows/columns, enabling overlapping and multi-scale spatial mixing as single tokens [2307.11458].
- In STRIPS planning world modeling, SATO treats grounded action instances—ordered in applied strips—as token sequences, encapsulating causal dependencies [2509.13389].

Tokenization in SATO thus shifts from atomic, locality-agnostic units (e.g., single faces, pixels, independent actions) to structural entities that preserve local flow, boundary semantics, and global compositionality.

## 2. Canonical Implementations by Domain

### 2.1 Artist-Quality Mesh Generation

SATO for triangular and quadrilateral mesh generation proceeds as follows [2604.09132]:

- **Hierarchical quantization**: Vertices are mapped to a three-level hierarchical code $(c_1, c_2, c_3)$, yielding high precision with concise vocabularies.
- **Strip extraction**: Triangle or quad strips grow sequentially by traversing face adjacencies with a stride parameter $\delta \in \{1,2\}$ and zipper-like connectivity.
- **Structural token vocabulary**: Specialized tokens denote the start of new strips ($C_1^t$) and UV island boundaries ($C_1^{uv}$); prefix-sharing compression emits only the minimal code difference per vertex.
- **Unified decoding**: The token sequence can be interpreted with either triangle or quad decoders by adjusting stride, supporting joint training and seamless decoding into either mesh type.

### 2.2 Semantic Visual Tokenization

In image modeling, SATO is implemented by compressing $h \times w$ latent patches into a short $L$-length 1D "strip" token sequence using a dual-branch transformer encoder with cross-attention [2603.16373]. Token quantization utilizes a Binary Spherical Quantizer so that high-level semantics are encoded in short, information-rich strips. Semantic alignment constraints enforce that each strip token corresponds to meaningful global or regional content.

### 2.3 Cross-Strip MLP Mixing

For MLP-based vision architectures, SATO defines a strip as a multi-token row or column whose aggregation supports both within-strip and cross-strip information flow. This is realized through overlapping strip tokens, channel grouping, and a Cascade Group Strip Mixing Module (CGSMM) that operates efficiently even with highly reduced spatial resolutions. Local Strip Mixing Modules (LSMM) further enhance local context integration [2307.11458].

### 2.4 Symbolic World Model Learning

In propositional STRIPS planning, the SATO formalism treats each grounded action $a \in A$ as a token. Learning is cast as next-token prediction over action strips (valid or invalid traces). A domain-specific transformer with a dedicated attention head per fluent $p_\ell$ and QKV projections maps histories to the learnable precondition and (add, delete) action lists, reconstructing the symbolic world model in the process [2509.13389].

## 3. Training Methodologies and Objectives

SATO models typically employ autoregressive, sequence-to-sequence, or masked modeling objectives adapted for their structural tokens:

- **Mesh generation**: SATO models undergo a three-stage curriculum—mass pretraining on triangle meshes, UV-segmentation post-training (with $C_1^{uv}$ tokens), and quad-specific fine-tuning [2604.09132].
- **Visual tokenization**: Training is staged: a diffusion-matched objective is used for initial token learning, augmented with semantic distillation and contrastive alignment constraints, followed by a one-step refiner with pixel- and perceptual-space losses [2603.16373].
- **Vision MLPs**: Training proceeds as usual for spatial MLPs but with SATO modules replacing patch- or pixel-level mixing for layers, leveraging standard image classification or reconstruction losses [2307.11458].
- **STRIPS world models**: Cross-entropy or focal loss optimizes the next-token prediction task from labeled valid/invalid action traces, with transformer weights directly reconstructing precondition and effect lists [2509.13389].

Optimization leverages scaled AdamW, tranche-wise or random subsequence sampling, temperature sampling for diversity, and curriculum transfer in multi-stage scenarios.

## 4. Empirical Performance and Comparisons

SATO yields substantial gains in structural fidelity, compression, and efficiency across tasks:

| Domain        | Compression/Speed         | Geometric/Semantic Metrics      | User/Task Study                   |
|--------------|--------------------------|---------------------------------|-----------------------------------|
| Mesh         | Token rate: 0.283        | F1 (Objaverse): 0.503           | SATO artist rank: 2.61 > BPT 1.40 |
| UV (Mesh)    | —                        | L2 stretch: 0.979 vs 0.921      | SATO (UV): 2.60 > PartUV 2.04     |
| Quad Mesh    | —                        | F1: 0.857                       | SATO: 1.80 > CrossGen: 1.24       |
| Vision MLP   | —                        | +1–3% Top-1 acc                 | Substantial gains on small datasets [2307.11458] |
| Image Gen    | bpp 0.07–0.125           | rFID: 0.67–0.88, SOTA           | Outperforms MaskGIT, VQGAN AR      |
| STRIPS       | —                        | 100% model recovery (N≥500)     | Perfect generalization to long traces [2509.13389] |

SATO in mesh modeling consistently delivers superior geometric quality (F1, CD, NC) and UV chart regularity relative to prior patch- or coordinate-based transformers [2604.09132]. In visual tokenization, SATO compresses images to 1D semantically dense strips, outperforming prior grid/patch-based tokenizers in FID, PSNR, and LPIPS at low bpp [2603.16373]. In MLP-based vision, SATO recovers information flow under strong spatial collapsing, outperforming classic MLPs and even rivaling convolutional and transformer approaches [2307.11458]. In symbolic STRIPS modeling, the SATO architecture identifies the operator schema with perfect fidelity given enough training data [2509.13389].

## 5. Advantages, Insights, and Limitations

The SATO paradigm confers several structural and practical advantages:

- **Edge-flow and semantic coherence**: Strips align with natural boundaries and artist workflows, preserving mid-level flow and semantic UV island structure [2604.09132].
- **Unified tokenization and multi-geometry decoding**: A single representation suffices for both triangles and quads, enabling scalable and transferable training [2604.09132].
- **Compression and efficiency**: Long strips enable amortized $O(1)$ overhead per triangle/region; prefix-sharing and overlap reduce redundancy [2604.09132, 2603.16373].
- **Preservation of information under spatial reduction**: Strip tokens and cross-group mixing maintain channel and spatial diversity even for small $H, W$ in vision MLPs [2307.11458].
- **Symbolic interpretability and model extraction**: In STRIPS world learning, SATO enables direct extraction of the symbolic transition model from transformer weights [2509.13389].

Empirical limitations include reliance on quad corpus quality for mesh regularity, rare fallback triangles in odd-length strips, and potential suboptimality on near-spherical or highly non-axis-aligned data [2604.09132]. In vision MLPs, SATO requires careful tuning of strip width and grouping per task [2307.11458]. For semantically aligned visual strips, gains are maximized when semantic constraints are well matched; less structured domains may see diminished benefit [2603.16373].

## 6. Broader Applications and Theoretical Implications

SATO is not restricted to mesh, vision, or symbolic planning. The generalization of strips as structured, context-sensitive tokens implies a broader design principle: by aligning tokenization with the compositional, hierarchical, or semantic regularities of the data, sequence models can achieve improved efficiency, robustness, and interpretability. A plausible implication is that future generative and world modeling systems will increasingly adopt dynamic or learned strip-like tokenizations to unify structure and semantics, scaling across visual, geometric, and symbolic domains [2604.09132, 2603.16373, 2307.11458, 2509.13389].

Source: https://www.emergentmind.com/topics/strips-as-tokens-sato