CSG Autoencoder: Neural Shape Reconstruction

• CSG-AE is a neural autoencoder that reconstructs shapes by encoding inputs into discrete, interpretable CSG programs composed of geometric primitives and Boolean operations.
• It leverages CNN/RNN encoder–decoder architectures to translate 2D images or 3D voxel grids into structured CSG trees that facilitate symbolic execution.
• The approach bridges autoencoding, inverse graphics, and program induction to deliver improved reconstruction accuracy with a clear, symbolic latent representation.

Searching arXiv for the cited CSG-related papers to ground the article in the current record. In the literature on neural shape reconstruction, “CSG Autoencoder” most naturally denotes an autoencoder whose latent representation is an explicit constructive solid geometry program or decomposition rather than an opaque continuous code. In that sense, the term is not the established title of a single canonical geometric model: a 2023 survey states that “CSG-AE is not explicitly mentioned anywhere in the survey,” and places the closest neural precedents in the family of CSG-Net and UCSG-Net (Fayolle et al., 2023). Within that family, the central idea is to encode a 2D image, 3D voxel grid, or point-cloud-derived shape into primitives and Boolean operations whose execution reconstructs the target, thereby making the bottleneck discrete, symbolic, and interpretable rather than purely vectorial (Sharma et al., 2019).

1. Definition and conceptual scope

In constructive solid geometry, a shape is represented by recursively composing primitives with Boolean operations such as union, intersection, and difference. The relevant autoencoding problem is therefore not merely shape compression, but reconstruction into a structured symbolic program whose leaves are primitives and whose internal nodes are Boolean operators. The survey formulates the underlying representation as a CSG tree expression $\Phi$ over a primitive-set $P$ and a set of binary operations $R$, with the solid fully described when $|\Phi(P)| = S$ (Fayolle et al., 2023).

Within this framing, a CSG autoencoder is best understood as an encoder–decoder model in which the decoder does not directly emit pixels, voxels, or points. Instead, it emits either a CSG program or a differentiable CSG-like decomposition, and reconstruction is obtained by symbolic execution, occupancy evaluation, or physics-informed rendering. This places the topic at the intersection of autoencoding, program induction, inverse graphics, and geometric modeling (Sharma et al., 2019).

A useful distinction runs through the literature. One branch predicts discrete token sequences or parse trees, as in CSGNet and the later “Neural Shape Parsers for Constructive Solid Geometry,” where the symbolic program itself is the interpretable bottleneck (Sharma et al., 2017). A second branch, exemplified by UCSG-Net, learns primitives and Boolean composition without parse-tree supervision and is explicitly evaluated on “2D and 3D autoencoding tasks” (Kania et al., 2020). A third, complementary line concerns differentiable execution for fixed CSG structure, as in DiffCSG, which makes continuous CSG parameters trainable from image losses but does not itself provide a full encoder–decoder autoencoder (Yuan et al., 2024).

2. Symbolic bottlenecks and program representations

The defining departure from a conventional autoencoder is the nature of the latent representation. In the constructive-solid-geometry view articulated for CSGNet, the reconstruction path is

$$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$

rather than the standard

$$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$

The important difference is that the latent representation is partly or wholly symbolic: the output program $y$ itself is the interpretable bottleneck (Sharma et al., 2019).

For sequence-based systems, the program space is typically defined by a context-free grammar. The generic grammar used in the neural shape parser work is

$S \rightarrow E$

$E \rightarrow E~E~T \;|\; P$

$$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$

$P$0

Programs are serialized in postfix notation, so binary CSG trees become reverse-Polish token sequences such as

$P$1

This matters algorithmically because postfix syntax aligns naturally with stack-based execution: primitive tokens push shapes to a stack, and operator tokens pop the top two elements and push their Boolean result (Sharma et al., 2019).

The survey situates this representation problem inside a difficult combinatorial search space. For binary trees with $P$2 internal nodes, the Catalan factor is

$P$3

and the number of CSG trees with $P$4 inner nodes is given as

$P$5

Across multiple tree sizes, the search space grows as

$P$6

This is one reason neural CSG autoencoding usually restricts grammar, primitive vocabulary, depth, or composition form (Fayolle et al., 2023).

3. Sequence-to-program autoencoding: CSGNet and neural shape parsers

CSGNet and its later stack-augmented formulation instantiate a shape-to-program encoder–decoder system in which the bottleneck is a discrete CSG program. The input is either a binary $P$7 image in 2D or a binary $P$8 voxel grid in 3D. The parser is written as

$P$9

where $R$0 is the convolutional encoder and $R$1 the recurrent decoder (Sharma et al., 2017).

In the synthetic 2D setting, the concrete grammar is

$R$2

$R$3

$R$4

$R$5

$R$6

The paper reports $R$7 unique 2D instructions total: $R$8 primitive instructions plus $R$9 Boolean ops plus EOS. In the synthetic 3D setting, the primitives are sphere, cube, and cylinder, with a 3D vocabulary of $|\Phi(P)| = S$0 instructions: $|\Phi(P)| = S$1 primitive parameterizations, the three Boolean operators, and EOS (Sharma et al., 2019).

The decoder produces programs autoregressively. The supervised objective is token log-likelihood,

$|\Phi(P)| = S$2

with inference by greedy decoding or beam search. Weak supervision is handled by REINFORCE because the executor is non-differentiable. The reward is based on rendered-shape similarity via Chamfer distance on boundary point sets, with invalid programs receiving reward zero and reward shaping $|\Phi(P)| = S$3 using $|\Phi(P)| = S$4 (Sharma et al., 2019).

The stack-augmented architecture is especially important for CSG-autoencoder design. In that variant, the input at time $|\Phi(P)| = S$5 includes both the target shape and the top-$|\Phi(P)| = S$6 intermediate execution states of the CSG stack, concatenated along the channel dimension:

$|\Phi(P)| = S$7

for 2D, and

$|\Phi(P)| = S$8

for 3D. This means the model conditions not only on the target but also on a rendered summary of what the current partial program has already built. The memory is symbolic and non-differentiable: it is the actual renderer execution stack rather than a learned soft stack (Sharma et al., 2019).

Empirically, the stack formulation improves reconstruction. On synthetic 2D data, supervised CSGNetStack with beam size $|\Phi(P)| = S$9 reaches $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$0 IoU and $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$1 CD versus $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$2 IoU and $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$3 CD for CSGNet; nearest neighbor obtains $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$4 IoU and $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$5 CD. On synthetic 3D, with beam size $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$6, 3D-CSGNetStack reaches $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$7 IoU versus $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$8 for 3D-CSGNet and $$I \xrightarrow{\text{CNN encoder} \Phi(I)} \xrightarrow{\text{RNN decoder}} y \xrightarrow{\text{CSG executor } \mathcal{R}} \hat I,$$9 for nearest neighbor (Sharma et al., 2019).

These systems are autoencoder-like rather than textbook autoencoders in nomenclature. They reconstruct through a symbolic bottleneck and a non-neural executor, and they rely heavily on discretized primitive parameterization, beam search, and optional post-hoc continuous refinement. The survey correspondingly treats CSG-Net as the clearest example of neural program generation for CSG, while also noting that it “supports only fully decomposable models” and “seems limited to very simple shapes” (Fayolle et al., 2023).

4. Unsupervised constructive solid geometry autoencoding: UCSG-Net

UCSG-Net is the strongest explicit realization of a constructive solid geometry autoencoder in the provided literature. Its stated contribution is to extract a CSG parse tree without any supervision, and it is evaluated on “2D and 3D autoencoding tasks” (Kania et al., 2020).

The overall pipeline is

$$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$0

Here $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$1 is the encoder, $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$2 is the latent code, $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$3 predicts primitive parameters, each primitive induces an SDF $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$4, SDFs are converted to soft/binary occupancy $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$5, and a stack of CSG layers produces the final occupancy $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$6 (Kania et al., 2020).

The latent dimensionality is explicitly

$$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$7

For 2D inputs UCSG-Net uses a 2D convolutional neural network; for 3D inputs it uses a 3D convolutional neural network. The primitive decoder is an MLP that outputs intrinsic parameters $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$8, translations $$I \xrightarrow{\text{encoder}} z \xrightarrow{\text{neural decoder}} \hat I.$$9, and rotations $y$0. In experiments, supported primitives are rectangles and circles in 2D, and boxes and spheres in 3D (Kania et al., 2020).

A key architectural difference from token-sequence models is that UCSG-Net does not autoregressively decode a grammar. Instead, it discovers structure through fixed-depth CSG layers with learned operand routing. At CSG layer $y$1, left and right operands are selected by learned key matrices

$y$2

with softmax scores

$y$3

and Gumbel-Softmax relaxation

$y$4

The selected operands are soft mixtures of candidate occupancy fields, after which the layer materializes all Boolean results rather than classifying a single operator (Kania et al., 2020).

UCSG-Net also makes a distinctive choice about Boolean computation. It does not use soft min/max SDF composition; instead, it converts each primitive SDF into a clipped occupancy indicator

$y$5

with $y$6 learnable and encouraged toward zero. Boolean operations are then performed in occupancy space:

$y$7

$y$8

$y$9

This is exact for binary occupancy and acts as a differentiable surrogate for soft occupancy (Kania et al., 2020).

Training is explicitly two-stage. The reconstruction objective is MSE between predicted and ground-truth occupancy over sampled points,

$S \rightarrow E$0

The first-stage total loss is

$S \rightarrow E$1

with $S \rightarrow E$2. Once $S \rightarrow E$3, a second-stage term drives Gumbel temperatures toward zero:

$S \rightarrow E$4

with $S \rightarrow E$5 (Kania et al., 2020).

The paper’s 2D results are unusually strong for unsupervised CSG autoencoding. On the CAD dataset, UCSG-Net unsupervised with $S \rightarrow E$6, $S \rightarrow E$7 reports Chamfer Distance $S \rightarrow E$8, compared with $S \rightarrow E$9 for CSG-NetStack supervised with $E \rightarrow E~E~T \;|\; P$0, $E \rightarrow E~E~T \;|\; P$1, and $E \rightarrow E~E~T \;|\; P$2 for CSG-NetStack RL with $E \rightarrow E~E~T \;|\; P$3, $E \rightarrow E~E~T \;|\; P$4. In 3D, UCSG-Net obtains CD $E \rightarrow E~E~T \;|\; P$5, compared with $E \rightarrow E~E~T \;|\; P$6 for VP, $E \rightarrow E~E~T \;|\; P$7 for SQ, $E \rightarrow E~E~T \;|\; P$8 for BAE, and $E \rightarrow E~E~T \;|\; P$9 for BSP-Net (Kania et al., 2020).

From an encyclopedia perspective, UCSG-Net is the clearest case in which “CSG autoencoder” is not merely an interpretive label but a technically faithful description: it has an encoder, a latent bottleneck, a programmatic CSG decoder, and a reconstruction objective, while explicitly discovering the parse tree without symbolic supervision (Kania et al., 2020).

5. Differentiable execution, refinement, and fixed-structure decoding

A recurrent obstacle for CSG autoencoding is that symbolic execution and Boolean rendering are usually non-differentiable. DiffCSG addresses a different but complementary subproblem: differentiable rendering for fixed-structure 3D CSG programs (Yuan et al., 2024).

Its setting is explicit. The number and type of primitives are fixed, Boolean operators are fixed, and tree/program structure is fixed; what is optimized are the continuous parameters $$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$0 controlling those primitives. The method renders CSG models directly during rasterization via the Goldfeather algorithm, rather than constructing the final Booleaned mesh with black-box mesh-processing libraries. The optimization objective is written as

$$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$1

with pixel-wise $$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$2 or $$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$3 loss (Yuan et al., 2024).

The critical contribution for CSG-autoencoder pipelines is the treatment of primitive intersection edges. Ordinary differentiable rasterization provides gradients mainly at silhouettes and visibility boundaries, but CSG introduces additional discontinuities where primitives intersect under Boolean operations. DiffCSG detects pairwise primitive intersection edges, rasterizes them separately, and feeds crossed pixels and edge endpoints to an anti-aliasing stage so that gradients can backpropagate from pixels to intersection points, from intersection points to primitive vertices, and from primitive vertices to primitive parameters (Yuan et al., 2024).

For fixed-template CSG autoencoders, this directly enables a decoder that predicts continuous primitive parameters while relying on image-space reconstruction loss. The paper is explicit, however, that it does not solve encoder design, latent representation design, discrete program induction, differentiable structure search, or variable operator selection. It is therefore best viewed as a differentiable execution backend rather than a complete CSG-AE (Yuan et al., 2024).

This suggests a natural hybrid architecture: discrete structure discovery by a parser or unsupervised CSG tree model, followed by differentiable parameter refinement through a renderer such as DiffCSG. The provided literature describes that combination as plausible rather than already unified in a single canonical system (Yuan et al., 2024).

6. Taxonomy, capabilities, and recurring limitations

The survey places CSG-autoencoder-like systems in the branch “CSG extraction from unstructured data → CSG generation → Deep learning based approaches,” alongside CSG-Net, UCSG-Net, CvxNet, BSP-Net, CSG-Stump, and CAPRI-Net (Fayolle et al., 2023). These methods share an encoder-to-structured-representation design, but they differ sharply in how much explicit symbolic structure they recover.

The following summary organizes the major systems discussed in the record.

System	Role in CSG-AE context	Core characterization
CSGNet / CSGNe	Proto-CSG-autoencoder	CNN/3D-CNN encoder + GRU decoder to postfix CSG program
UCSG-Net	Explicit CSG autoencoder	Unsupervised primitive prediction and parse-tree discovery
DiffCSG	Differentiable execution backend	Fixed-structure CSG rendering with gradients to continuous parameters
Survey taxonomy	Literature placement	Locates CSG-AE nearest to UCSG-Net and CSG-Net

A persistent misconception is that “CSG autoencoder” denotes a standard latent-variable autoencoder with direct differentiable reconstruction. The literature does not support that generalization. In CSGNet, the symbolic bottleneck is discrete, the executor is non-differentiable, and weak supervision requires policy gradient (Sharma et al., 2019). In UCSG-Net, differentiability is regained by relaxing occupancy and operand selection, but the decoder is still a structured CSG generator rather than a deconvolutional pixel decoder (Kania et al., 2020). In DiffCSG, differentiability is available only for continuous parameters under fixed structure (Yuan et al., 2024).

The recurrent limitations are also consistent across sources. Primitive vocabularies are small and often heavily discretized; CSG-Net uses only three primitive families in 2D and three in 3D, with short programs and coarse parameter grids (Sharma et al., 2019). UCSG-Net is limited to rectangles and circles in 2D, and boxes and spheres in 3D (Kania et al., 2020). The survey repeatedly emphasizes limited primitive vocabulary, uncertain out-of-distribution behavior, difficulty scaling to complex exact trees, and a tendency to recover approximate or CSG-like structures rather than exact CAD-grade programs (Fayolle et al., 2023).

A second recurring limitation is search-space restriction. Sequence models constrain length and grammar; layered models constrain depth and reuse; mask-based models constrain the composition form. The survey’s main point is that neural methods often succeed only by restricting the search space through fixed primitive counts, restricted grammar, decomposable models, fixed-depth CSG-like structures, or masks instead of unrestricted symbolic trees (Fayolle et al., 2023).

A third limitation concerns supervision and optimization. CSGNet’s practical recipe relies on synthetic supervised pretraining before RL fine-tuning (Sharma et al., 2019). UCSG-Net removes parse-tree supervision but depends on careful staged discretization of occupancy and routing (Kania et al., 2020). DiffCSG avoids discrete search entirely but assumes the program structure is fixed (Yuan et al., 2024). Taken together, these systems suggest that no single method in the supplied record fully solves unconstrained, exact, end-to-end CSG autoencoding.

7. Terminological ambiguity of “CSG-AE”

The abbreviation “CSG-AE” is ambiguous across fields. In geometric modeling and inverse CSG, the survey states that “CSG-AE is not explicitly mentioned anywhere in the survey,” and the closest context is the family of neural CSG reconstruction models, especially UCSG-Net and secondarily CSG-Net (Fayolle et al., 2023). In this geometric sense, “CSG autoencoder” is therefore a descriptive label for systems that reconstruct shapes into constructive solid geometry programs or decompositions.

A different use appears in wireless communications, where “CSG Autoencoder (CSG-AE)” is the explicit name of the model proposed for Channel Space Gridization. That model consists of a trainable RSRP-to-CAPS encoder,

$$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$4

a sparse codebook quantizer,

$$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$5

and a fixed physics-informed decoder,

$$T \rightarrow OP_1 \;|\; OP_2 \;|\; \ldots \;|\; OP_m$$6

Its task is channel-centric clustering rather than constructive solid geometry (Wang et al., 21 Jul 2025).

This ambiguity matters because the two literatures are unrelated except for the abbreviation. The constructive-solid-geometry usage concerns symbolic shape reconstruction from images, voxels, or point clouds; the wireless-communications usage concerns channel estimation and gridization from beam-level RSRP. A precise reading of “CSG Autoencoder” therefore requires domain context (Wang et al., 21 Jul 2025).

Within the geometric literature itself, the most defensible encyclopedic conclusion is narrow: there is no single standard model universally named “CSG Autoencoder,” but there is a coherent class of systems in which an encoder maps unstructured shape observations to explicit CSG primitives and Boolean composition, with CSGNet providing the canonical sequence-to-program parser, UCSG-Net providing the clearest unsupervised autoencoding realization, and DiffCSG supplying a differentiable execution layer for fixed-structure continuous refinement (Sharma et al., 2017).