VoxSet: Semi-Structured 3D Representation

• VoxSet is a semi-structured representation that combines compact latent-set encoding with coarse voxel anchors to capture both spatial support and detailed geometry in 3D assets.
• It decouples the 'where' and 'what' aspects of 3D generation, enabling arbitrary-resolution decoding and efficient transformer compatibility during inference.
• Empirical results demonstrate that VoxSet achieves high reconstruction quality and scalable performance with significantly fewer latent tokens compared to traditional methods.

to=arxiv_search.^^^^^^^^10query10^^^^^^^^ 高频彩大发快三json code {"^^^^^^^^10query10 OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^^^^^", "^^^^^^^^10max_results10 ^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10query10^^^^, "^^^^^^^^10sort_by10 "^^^^^^^^10submittedDate10 "^^^^^^^^10sort_order10 "^^^^^^^^10descending10 to=arxiv_search.^^^^^^^^10query10^^^^^^^^ 天天中彩票公众号json code {"^^^^^^^^10query10 AND all:\10^^^^^^^^"VoxSet\"", "^^^^^^^^10max_results10 ^^^^10sort_order10^^^^, "^^^^^^^^10sort_by10 "relevance", "^^^^^^^^10sort_order10 "^^^^^^^^10descending10 VoxSet is a semi-structured latent representation for ^^^^10sort_by10^^^^D assets introduced as the core representational component of LATTICE, a framework for high-fidelity ^^^^10sort_by10^^^^D generation at scale (&&&^^^^10query10^^^^&&&). It compresses a ^^^^10sort_by10^^^^D asset into a compact set of latent vectors anchored to a coarse voxel grid, with the design goal of combining the compression and transformer compatibility of latent-set methods with the positional structure of voxel-based methods. In the LATTICE formulation, VoxSet decouples coarse spatial support from detailed geometry: a sparse voxelized anchor provides the “where,” while voxel-attached latent tokens encode the “what.” This enables position-aware generation, arbitrary-resolution decoding, and strong token-level test-time scaling (&&&^^^^10query10^^^^&&&).

^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^. Definition and representational scope

VoxSet is defined as a semi-structured representation because it is neither a dense voxel tensor nor a fully unstructured latent bag (&&&^^^^10query10^^^^&&&). Its tokens are sparse and compact, as in VecSet-style latent representations, but each token is explicitly tied to the center of an active voxel intersecting the object surface. The representation therefore preserves a set-like latent interface while making token positions available at inference time.

This design addresses a recurrent difficulty in ^^^^10sort_by10^^^^D generative modeling: unlike ^^^^10max_results10^^^^D image synthesis, ^^^^10sort_by10^^^^D generation must infer both spatial support and detailed surface structure. VoxSet is presented as a way to separate those two burdens. A sparse coarse voxel structure supplies spatial anchors, and the latent vectors attached to those anchors carry local geometric content. The paper positions this as a middle point between highly structured sparse-voxel pipelines and compact but weakly grounded latent sets (&&&^^^^10query10^^^^&&&).

A direct consequence is that VoxSet is not merely a storage format for reconstructed geometry. It is also the conditioning substrate for generation. Because the latent coordinates are known, positional embeddings can be injected into the transformer that denoises or transports the latent tokens. This suggests that VoxSet is as much an inference-time control structure as it is an autoencoding representation.

^^^^10max_results10^^^^. Motivation and design rationale

The motivation for VoxSet is framed around the limitations of existing ^^^^10sort_by10^^^^D representations (&&&^^^^10query10^^^^&&&). Dense or sparse voxel methods provide explicit spatial structure, but their active token counts can become large; the paper cites Trellis as requiring active sequences of over ^^^^10max_results10query10^^^^,^^^^10query10query10query10^^^^ voxels at PRESERVED_PLACEHOLDER_^^^^10query10^^^^. By contrast, VecSet-style methods are compact and can represent a shape using as few as ^^^^10sort_by10^^^^,^^^^10query10query10max_results10^^^^ latent vectors, but the positions of those latent queries are not known at inference time when an object is generated from scratch.

The central design claim is therefore not that full structure is always preferable, but that localizable guidance at test time is especially valuable. VoxSet inherits the compactness, standard-attention implementation, and arbitrary-resolution autoencoding associated with latent-set methods, while introducing coarse voxel anchors that can be used directly by the generator. The paper explicitly argues that this combination supports four properties: arbitrary-resolution encoding and decoding, cheap progressive training by increasing token count over stages, direct positional embedding injection into the diffusion transformer, and strong token-length test-time scaling (&&&^^^^10query10^^^^&&&).

The “semi-structured” designation is important. VoxSet does not store a dense feature field over every voxel, nor does it commit to the long active sequences typical of sparse volumetric methods. Instead, it stores a compact latent sequence over only the active surface-intersecting voxel anchors. A plausible implication is that the representation is intended to capture just enough geometric locality to stabilize generation, without paying the full sequence-length cost of voxel-native transformers.

^^^^10sort_by10^^^^. Construction, encoding, and decoding

VoxSet is built on a geometry VAE whose input is a point cloud

PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^

where each point contains ^^^^10sort_by10^^^^D coordinates, a surface normal, and a binary sharpness indicator (&&&^^^^10query10^^^^&&&). The point cloud mixes uniform surface sampling and sharp-edge importance sampling, following Hunyuan^^^^10sort_by10^^^^D-^^^^10max_results10 to preserve high-frequency geometry.

The encoder uses cross-attention between encoded point features and voxel queries. The defining change relative to prior VecSet formulations is that the queries are not surface-sampled point queries but the centers of active voxels intersecting the object surface. Only those active voxels receive latent tokens. This yields a sparse set of latent codes whose coordinates are explicit and whose support is concentrated near the surface (&&&^^^^10query10^^^^&&&).

A practical problem is resolution flexibility. Rather than training separate models for many voxel resolutions, the method introduces ^^^^^^^^10query10^^^^^^^^ jitter: PRESERVED_PLACEHOLDER_^^^^10max_results10^^^^ where PRESERVED_PLACEHOLDER_^^^^10sort_by10^^^^ is the smallest supported resolution (&&&^^^^10query10^^^^&&&). The paper reports that this makes the VAE robust to voxel-^^^^^^^^10query10^^^^^^^^ decoding at arbitrary resolutions greater than PRESERVED_PLACEHOLDER_^^^^10submittedDate10^^^^.

The decoder is described symmetrically: SDF grid coordinates are used as queries in cross-attention against the latent tokens, and a mesh is extracted with Marching Cubes (&&&^^^^10query10^^^^&&&). The paper does not provide an explicit reconstruction-loss equation in the text summarized here, but it does emphasize that this decoder supports arbitrary-resolution ^^^^^^^^10query10 For generation, rotary positional embedding (RoPE) is applied to each noisy latent token, and the paper states that this is crucial for convergence (&&&^^^^10query10^^^^&&&).

^^^^10submittedDate10^^^^. Role within the LATTICE pipeline

LATTICE uses VoxSet in a two-stage pipeline (&&&^^^^10query10^^^^&&&). In the first stage, a sparse voxelized geometry anchor is obtained from an off-the-shelf pretrained ^^^^10sort_by10^^^^D generator such as Hunyuan^^^^10sort_by10^^^^D-^^^^10max_results10^^^^ or Trellis. This stage provides a coarse sparse structure rather than final geometry. In the second stage, a rectified-flow transformer generates the detailed geometry VoxSet conditioned on image features and the stage-one anchors.

This realizes the paper’s “where” versus “what” decomposition. Stage one predicts where surface-supporting structure should exist; stage two predicts what detailed geometry should occupy those locations. The generator is trained with the flow matching objective with a linear coupling plan, following SiT, although the full objective is not written explicitly in the summarized text (&&&^^^^10query10^^^^&&&).

Several implementation choices are directly tied to VoxSet’s representational assumptions. Image conditioning uses DINOv^^^^10max_results10^^^^-Giant, with the last hidden-layer embedding and no extra positional embedding added to the image features because DINO already encodes spatial information. Classifier-free guidance is implemented by dropping conditioning embeddings with probability PRESERVED_PLACEHOLDER_^^^^10sort_order10^^^^. To reduce cost, training randomly samples a fixed number of structure tokens and uses progressive token scaling, training first with ^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10query10max_results10submittedDate10^^^^ tokens and progressively scaling up to ^^^^10descending10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10submittedDate10submittedDate10^^^^ (&&&^^^^10query10^^^^&&&).

This pipeline positions VoxSet as more than a latent codec. It is the interface that makes coarse geometric scaffolding and detailed latent generation compatible inside a pure-transformer ^^^^10sort_by10^^^^D pipeline.

^^^^10sort_order10^^^^. Empirical behavior, scaling, and reconstruction quality

The paper reports strong reconstruction results for VoxSet on LATTICE-Bench(R), measured by Chamfer Distance (CD, lower is better) and F-score at threshold ^^^^10query10^^^^.^^^^10query10query10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ (F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^, higher is better), with CD multiplied by PRESERVED_PLACEHOLDER_^^^^10descending10^^^^ and F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ by PRESERVED_PLACEHOLDER_^^^^10query10^^^^ (&&&^^^^10query10^^^^&&&).

Method and latent size	CD	F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^
Hunyuan^^^^10sort_by10^^^^D-^^^^10max_results10 PRESERVED_PLACEHOLDER_^^^^10ti:\10^^^^	^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10max_results10^^^^.^^^^10sort_by10sort_order10^^^^	^^^^10ti:\10max_results10^^^^.^^^^10query10ti:\10
Hunyuan^^^^10sort_by10^^^^D-^^^^10max_results10 PRESERVED_PLACEHOLDER_^^^^10 AND all:\10^^^^	^^^^10 AND all:\10^^^^.^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10sort_order10query10^^^^	^^^^10 AND all:\10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^.^^^^10sort_order10query10^^^^
SparseFlex, PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10query10^^^^	^^^^10ti:\10^^^^.^^^^10query10max_results10query10	^^^^10 AND all:\10query10^^^^.^^^^10 AND all:\10submittedDate10^^^^
Direct^^^^10sort_by10^^^^D-s^^^^10max_results10 PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^	^^^^10submittedDate10^^^^.^^^^10 AND all:\10ti:\10query10^^^^	^^^^10 AND all:\10query10^^^^.^^^^10submittedDate10descending10^^^^
LATTICE / VoxSet, PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10max_results10^^^^	^^^^10sort_order10^^^^.^^^^10sort_by10max_results10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^	^^^^10 AND all:\10sort_order10^^^^.^^^^10sort_by10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^
LATTICE / VoxSet, PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10sort_by10^^^^	^^^^10max_results10^^^^.^^^^10 AND all:\10query10 AND all:\10^^^^	^^^^10 AND all:\10ti:\10^^^^.^^^^10sort_order10sort_by10^^^^
LATTICE / VoxSet, PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10submittedDate10^^^^	^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^.^^^^10ti:\10 AND all:\10sort_by10^^^^	^^^^10 AND all:\10 AND all:\10^^^^.^^^^10sort_order10 AND all:\10^^^^

These results are used to argue that VoxSet offers a stronger compactness-quality tradeoff than Hunyuan^^^^10sort_by10^^^^D-^^^^10max_results10^^^^ and can outperform voxel-native methods at much smaller latent sizes in some settings (&&&^^^^10query10^^^^&&&). The paper also presents an ablation on ^^^^^^^^10query10^^^^^^^^ jitter with ^^^^10submittedDate10query10 AND all:\10descending10^^^^ tokens and voxel queries: compared with a baseline of CD PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10sort_order10^^^^, F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10descending10^^^^ at resolution ^^^^10descending10submittedDate10^^^^ and CD PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10query10^^^^, F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10ti:\10^^^^ at resolution ^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10max_results10ti:\10^^^^, adding ^^^^^^^^10query10^^^^^^^^ jitter improves performance to CD PRESERVED_PLACEHOLDER_^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10 AND all:\10^^^^, F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ PRESERVED_PLACEHOLDER_^^^^10max_results10query10^^^^ at resolution ^^^^10descending10submittedDate10^^^^, CD PRESERVED_PLACEHOLDER_^^^^10max_results10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^, F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ PRESERVED_PLACEHOLDER_^^^^10max_results10max_results10^^^^ at resolution ^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10max_results10ti:\10^^^^, and CD PRESERVED_PLACEHOLDER_^^^^10max_results10sort_by10^^^^, F^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^ PRESERVED_PLACEHOLDER_^^^^10max_results10submittedDate10^^^^ at resolution ^^^^10max_results10sort_order10descending10^^^^ (&&&^^^^10query10^^^^&&&).

For image-to-geometry generation, LATTICE-^^^^10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^.^^^^10 AND all:\10^^^^B achieves PRESERVED_PLACEHOLDER_^^^^10max_results10sort_order10^^^^, PRESERVED_PLACEHOLDER_^^^^10max_results10descending10^^^^, PRESERVED_PLACEHOLDER_^^^^10max_results10query10^^^^, and PRESERVED_PLACEHOLDER_^^^^10max_results10ti:\10^^^^, compared with Hunyuan^^^^10sort_by10^^^^D ^^^^10max_results10^^^^.^^^^10query10^^^^ at PRESERVED_PLACEHOLDER_^^^^10max_results10 AND all:\10^^^^, PRESERVED_PLACEHOLDER_^^^^10sort_by10query10^^^^, PRESERVED_PLACEHOLDER_^^^^10sort_by10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^, and PRESERVED_PLACEHOLDER_^^^^10sort_by10max_results10^^^^, and Trellis at PRESERVED_PLACEHOLDER_^^^^10sort_by10sort_by10^^^^, PRESERVED_PLACEHOLDER_^^^^10sort_by10submittedDate10^^^^, PRESERVED_PLACEHOLDER_^^^^10sort_by10sort_order10^^^^, and PRESERVED_PLACEHOLDER_^^^^10sort_by10descending10^^^^ (&&&^^^^10query10^^^^&&&). The paper notes that the semantic-metric gains are modest numerically, while visual gains in smoothness, regularity, and detail are more pronounced.

A major claim concerns scaling. VoxSet models scale from PRESERVED_PLACEHOLDER_^^^^10sort_by10query10^^^^B to PRESERVED_PLACEHOLDER_^^^^10sort_by10ti:\10^^^^B to PRESERVED_PLACEHOLDER_^^^^10sort_by10 AND all:\10^^^^B parameters with increasingly detailed and regular structures, and a model trained with PRESERVED_PLACEHOLDER_^^^^10submittedDate10query10^^^^ tokens can be evaluated with PRESERVED_PLACEHOLDER_^^^^10submittedDate10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^, PRESERVED_PLACEHOLDER_^^^^10submittedDate10max_results10^^^^, and even PRESERVED_PLACEHOLDER_^^^^10submittedDate10sort_by10^^^^ tokens (&&&^^^^10query10^^^^&&&). The paper further states that the ^^^^10max_results10^^^^B base model can be effectively trained in under ^^^^10max_results10submittedDate10^^^^ hours on ^^^^10descending10submittedDate10^^^^ GPUs. This suggests that VoxSet is intended not only for fidelity but also for low-cost progressive training and flexible inference.

^^^^10descending10^^^^. Relation to adjacent methods and terminological ambiguity

VoxSet is closely related to VecSet-style latent representations but differs primarily in its ^^^^^^^^10query10^^^^^^^^ set (&&&^^^^10query10^^^^&&&). VecSet uses learnable or surface point queries; VoxSet uses voxel centers of active surface-intersecting voxels. The paper argues that this reduces the train-test gap because the token positions are available during generation, enabling RoPE-based positional conditioning and stronger test-time scaling. A stated limitation is that this locality depends on a first-stage coarse voxel structure.

Relative to sparse voxel latents such as SLAT- or XCube-like formulations, VoxSet is presented as more compact and easier to implement with standard transformer operations, without sparse convolutions or specialized sparse attention (&&&^^^^10query10^^^^&&&). Relative to dense voxels or triplanes, its emphasis is on sparse surface-focused tokens rather than dense structured fields.

The term “VoxSet” is also susceptible to confusion. It should not be conflated with “VoxSeT” (Voxel Set Transformer), which is a voxel-based architecture for ^^^^10sort_by10^^^^D object detection from point clouds rather than a latent ^^^^10sort_by10^^^^D asset representation (&&&^^^^10max_results10all:VoxSet OR ti:VoxSet OR ti:LATTICE 3D asset generation semi-structured representation10^^^^&&&). Nor should it be conflated with the CVT-focused singing dataset introduced in “A Dataset for Automatic Vocal Mode Classification,” which the paper explicitly says is not officially named “VoxSet” (&&&^^^^10max_results10max_results10^^^^&&&). In current arXiv usage, “VoxSet” refers specifically to the semi-structured ^^^^10sort_by10^^^^D representation introduced in LATTICE (&&&^^^^10query10^^^^&&&).

In that sense, VoxSet occupies a specific place in the ^^^^10sort_by10^^^^D generation literature: it is a representation designed to make latent-set compression and spatially grounded generation compatible at scale. The paper’s broader interpretation is that ^^^^10sort_by10^^^^D generation has lacked a representation that is simultaneously compact, structured, transformer-friendly, and scalable, and VoxSet is proposed as that missing middle layer (&&&^^^^10query10^^^^&&&).