Multi-Scale Geometry-Aware Tokenizer (MAGNO)

• MAGNO is a tokenizer that converts variable-resolution geometric inputs into a fixed latent grid, preserving critical multi-scale spatial details.
• It employs multi-scale aggregation techniques, such as ball queries and latent grid processing, to effectively encode both local and contextual geometric features.
• MAGNO’s design supports self-supervised and task-conditioned objectives, enabling robust representation learning for CAD models and complex 3D data.

Searching arXiv for the cited MAGNO-related papers to ground the article in current literature. Multi-Scale Geometry-Aware Tokenizer (MAGNO) denotes a tokenizer architecture that converts variable-size geometric inputs into a fixed or bounded latent token set while preserving multi-scale spatial structure and explicit geometric cues. In current usage, the term is most directly instantiated in Shape, where MAGNO converts a normalized CAD surface mesh into a structured $24 \times 24 \times 24$ latent token grid, and it is also articulated as a tokenizer design pattern in GeoTransolver, where multi-scale ball-query encoders build geometry, global-parameter, and boundary-condition context that is reused across transformer blocks on irregular domains (Mounmo et al., 19 Apr 2026, Adams et al., 23 Dec 2025).

1. Conceptual lineage and scope

In Shape, MAGNO is the front-end “geometry interface” that turns an arbitrary-resolution CAD surface mesh into a fixed, structured 3D grid of tokens. Shape states that this tokenizer is inherited directly from the Multiscale Attentional Graph Neural Operator encoder of GAOT, but repurposed for self-supervised representation learning rather than PDE surrogate modeling. The resulting pipeline is

$$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$

with a released configuration of $T=24^3=13{,}824$ tokens and token dimension $C=128$ (Mounmo et al., 19 Apr 2026).

GeoTransolver provides a complementary, tokenizer-centric formulation. There, the central design is to separate physical state tokens from geometry, global-parameter, and boundary-condition context; precompute this context via multi-scale ball queries; augment local state tokens with geometry-aware features; and reuse the context in every transformer block through Geometry-Aware Latent Embedding (GALE). Reinterpreted in tokenizer language, MAGNO becomes a mechanism for turning meshes, field samples, and operating conditions into geometry-aware state tokens plus a compact context token set in a shared latent “physical state” space (Adams et al., 23 Dec 2025).

Earlier and adjacent work shows that the MAGNO label names a broader design family rather than a single canonical block. The 2023 “Multi-scale Geometry-aware Transformer” for point cloud classification divided point clouds into multi-scale patches, used a sphere-mapping local feature extractor, and applied geodesic-based self-attention, thereby establishing an explicit multi-scale and geometry-aware tokenization logic for unordered 3D data (Wei et al., 2023). This suggests that MAGNO is best understood as a research program centered on geometry-aware token formation, not merely a single implementation.

2. Representational primitives and token types

A MAGNO-style tokenizer typically begins by distinguishing local signals from geometric context. In GeoTransolver, for slice $m$, the local inputs are

$$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$

while geometry is represented as

$$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$

and global parameters are

$p \in \mathbb{R}^{d_p}.$

In tokenizer terms, these correspond respectively to state tokens, geometry tokens, and global or boundary-condition tokens (Adams et al., 23 Dec 2025).

Shape specializes this abstraction to CAD surface meshes. Each mesh is canonicalized to a unit bounding box, sampled at $N=8{,}192$ surface points, and equipped with coordinates $\mathbf{P}\in\mathbb{R}^{N\times 3}$, normals $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$0, and scalar curvature $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$1. For each latent grid cell $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$2, MAGNO computes a 28-dimensional raw geometric signature $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$3 from mean, standard deviation, minimum, and maximum over three groups within a physical neighborhood: relative positions $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$4, normals $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$5, and curvature $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$6. The dimensional decomposition is $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$7, and the paper identifies this vector both as an initializer or enrichment for token embeddings and as the masked-token reconstruction target during pretraining (Mounmo et al., 19 Apr 2026).

These two formulations reveal two recurrent token types. The first is the dense set of local tokens tied to spatial support, such as latent grid cells in Shape or slice-wise state tokens in GeoTransolver. The second is a compact set of contextual tokens encoding global geometry, regime variables, or cross-scale summaries. A plausible implication is that MAGNO’s distinctive feature is not simply geometry-aware local embedding, but the coexistence of local and contextual token channels.

3. Multi-scale geometry extraction

The defining MAGNO operation is multi-scale geometry aggregation. In GeoTransolver, a set of scales

$$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$8

specifies radii $$\text{surface mesh} \xrightarrow{\text{MAGNO tokenizer}} \text{latent 3D token grid } \mathbf{Z} \xrightarrow{\text{Transformer processor}} \text{embeddings / reconstructions},$$9 and maximum neighbor counts $T=24^3=13{,}824$0. For each state point $T=24^3=13{,}824$1, geometry-to-state ball queries collect nearby geometry points $T=24^3=13{,}824$2 and aggregate scale-specific features from $T=24^3=13{,}824$3. Concatenating the outputs across scales yields a multi-scale geometry encoding $T=24^3=13{,}824$4, which is then projected and appended to the local latent state. In the reverse direction, state-to-geometry ball queries aggregate nearby state features around each geometry point and pool them into scale-wise summaries $T=24^3=13{,}824$5. Together with pooled geometry-only features $T=24^3=13{,}824$6 and global parameters $T=24^3=13{,}824$7, these form a reusable context vector

$T=24^3=13{,}824$8

Ablations on DrivAerML varied single-scale and multi-scale radii, including $T=24^3=13{,}824$9, and varied kernel size up to $C=128$0; the reported finding is that multi-scale settings consistently lower errors and that larger kernels improve performance (Adams et al., 23 Dec 2025).

Shape instantiates multi-scale geometry aggregation on a structured latent grid. For each grid cell $C=128$1, neighborhoods

$C=128$2

are built at radii $C=128$3. The tokenizer runs three AGNO layers at successively coarser radii, combining them by concatenation and linear projection. Attention from grid cells to nearby surface points uses cosine-style logits with a learned temperature: $C=128$4 and the scale-specific aggregated representation is

$C=128$5

The three scale outputs are concatenated and projected into the final token embedding $C=128$6 (Mounmo et al., 19 Apr 2026).

Across these systems, multi-scale is realized by neighborhood radii rather than by a single pyramid of uniformly downsampled grids. The literature also shows other valid realizations. GPSToken uses entropy-driven region partitioning and parameterizes each token as a 2D Gaussian with learnable center, covariance, and texture features, producing non-uniform image tokenization in which small tokens appear in complex regions and large tokens in homogeneous ones (Zhang et al., 1 Sep 2025). This suggests that MAGNO’s “multi-scale” qualifier is architectural rather than tied to one discretization scheme.

4. Interfaces to transformers and downstream reasoning

GeoTransolver’s GALE formalizes how geometry-aware tokens are consumed. Within each slice $C=128$7, self-attention operates only over the slice’s latent state tokens, while cross-attention uses the shared context $C=128$8 as keys and values. An adaptive gate

$C=128$9

blends self-attention and context-attention outputs at each layer. The effect is persistent geometry and regime conditioning at every depth, rather than one-time injection at the input (Adams et al., 23 Dec 2025).

Shape consumes MAGNO outputs with a transformer processor using grouped-query attention and RMSNorm. The $m$0 latent grid is patchified with patch size $m$1, yielding $m$2 patches, then processed by $m$3 transformer blocks with 4 attention heads and 2 KV heads. Absolute positional encoding derived from grid coordinates is used in the released configuration. A learned attention-pooling module then forms a global embedding $m$4, while per-token embeddings feed a reconstruction head (Mounmo et al., 19 Apr 2026).

A third architectural pattern appears in NDTokenizer3D, where raw point clouds are converted into multi-scale Normal Distributions Transform cells, encoded scale-wise, and then fused by a Multi-Scale NDT Decoder (MSDec). Each cell stores a 15-dimensional descriptor $m$5 consisting of mean, covariance, and projected RGB color. MSDec uses query tokens that cross-attend to one scale per decoder layer, producing a bounded set of holistic scene tokens; ablations found that about 400–850 queries are effective, and three scales $m$6 gave the best trade-off (Tang et al., 26 Nov 2025). Unlike GALE, where context remains external to the main state stream, MSDec directly produces the scene-token sequence that an LLM consumes.

5. Objectives, calibration, and interpretability

MAGNO systems have been trained under both self-supervised and task-conditioned objectives. Shape combines masked-token reconstruction of normalized geometric signatures with multi-resolution contrastive consistency. Fifty percent of latent grid tokens are masked, the reconstruction head predicts $m$7, and the target is the z-scored signature

$m$8

The reconstruction loss is Smooth-L1 on masked positions, and the global contrastive objective is symmetric InfoNCE with temperature $m$9. On a held-out split of 2,983 meshes, Shape reports reconstruction $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$0 and 98.1% top-1 retrieval under the Wang–Isola protocol. Its $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$1 ablation on loss type and target-space normalization shows that per-dimension normalization is critical: without it, performance collapses to $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$2 and top-1 $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$3, whereas with normalization both MSE and Smooth-L1 exceed $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$4 and top-1 $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$5, with Smooth-L1 offering secondary stability (Mounmo et al., 19 Apr 2026).

Shape also demonstrates a direct interpretability mechanism. Because each latent grid token is associated with a spatial cell and a geometric signature, one can run the reconstruction head without masking and compute per-token residuals

$$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$6

Mapping $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$7 back to the corresponding surface neighborhoods yields heatmaps that highlight unusual fillets, defects, or sharp edges, while typical regions remain low-error. In this formulation, explainability is not an auxiliary attribution method but a by-product of the reconstruction prior (Mounmo et al., 19 Apr 2026).

Task-conditioned tokenizers show a related but different supervision strategy. The $$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$8Tokenizer for radiology report generation uses relative positional encoding, differentiable soft top-$$X_m = \{(x_{m,i}, f_{m,i})\}_{i=1}^{N_m}, \qquad x_{m,i}\in\mathbb{R}^3,\quad f_{m,i}\in\mathbb{R}^{d_x},$$9 token selection with $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$0, and dynamic multi-scale pooling over scales $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$1; its full system improves GREEN from 0.339 under supervised fine-tuning to 0.400 after DPO (Li et al., 30 Jun 2025). This suggests that MAGNO-like tokenizers can also be optimized through downstream preference objectives rather than solely through geometric pretext tasks.

6. Variants, naming ambiguity, and broader research directions

The term MAGNO carries a mild naming ambiguity. In Shape, it explicitly refers to a “multi-scale geometry-aware tokenizer,” while the implementation is inherited from the Multiscale Attentional Graph Neural Operator encoder of GAOT (Mounmo et al., 19 Apr 2026). GeoTransolver, by contrast, presents MAGNO more as a reconstructed design recipe based on multi-scale ball queries, shared geometry/global context, and persistent cross-attentional conditioning (Adams et al., 23 Dec 2025). The literature therefore does not define a single mandatory MAGNO block.

Related work expands the design space in several directions. Hi-SAM’s Disentangled Semantic Tokenizer performs geometry-aware alignment on a shared hypersphere using a Gram-matrix volume objective, then applies coarse-to-fine residual quantization with three shared and three modality-specific codes per item; on its Industrial dataset, removing Cross-Modal Geometric Alignment drops GAUC from 0.6410 to 0.6166, underscoring the importance of geometry-aware alignment before tokenization (Pan et al., 12 Feb 2026). Galaxy Walker creates geometry tokens from Euclidean, spherical, and hyperbolic galaxy graphs and processes them with a geometry-adapter mixture-of-experts, reaching $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$2 up to $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$3 and up to $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$4 F1 improvement in challenging morphology features (Chen et al., 24 Mar 2025). GPSToken shows that geometry-aware tokenization can be continuous rather than grid-based, achieving rec.FID $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$5 and FID $$\mathcal{G} = \{(g_j,\gamma_j)\}_{j=1}^{M_g}, \qquad g_j\in\mathbb{R}^3,\quad \gamma_j\in\mathbb{R}^{d_g},$$6 on image reconstruction and generation with 128 Gaussian tokens (Zhang et al., 1 Sep 2025). NDTokenizer3D demonstrates that scene tokenization can also be built from Gaussian cell statistics in a voxel pyramid, outperforming downsampling-based alternatives on 3D VQA and referring segmentation (Tang et al., 26 Nov 2025).

Several misconceptions are clarified by this record. MAGNO is not synonymous with voxelization: GeoTransolver relies on ball-query context construction, Shape uses latent-grid queries over surface points, GPSToken uses Gaussian regions, and Hi-SAM operates in a quantized semantic space. Nor is geometry awareness reducible to positional encoding alone: the cited systems encode normals, curvature, covariance, boundary conditions, global parameters, or non-Euclidean manifold structure in addition to spatial position. Finally, “multi-scale” is not implemented uniformly across the literature; it may refer to query radii, entropy-driven partitions, pooling kernels, residual-quantization depth, or NDT voxel pyramids. This suggests that MAGNO is best treated as a unifying tokenizer principle for geometry-conditioned token formation across irregular domains, CAD, 3D scenes, medical volumes, recommendation, and multimodal scientific modeling.