S2C-3D: Reconstruction & Compression

• S2C-3D is a label for two distinct 3D methods: one for reconstructing complete scenes from sparse viewpoints and one for compressing voxel-based point clouds.
• In the reconstruction approach, a diffusion model with view-consistency conditioning and camera trajectory planning is used to recover high-fidelity, artifact-free 3D scenes from as few as six to eight images.
• For voxel compression, the method employs stage-wise space-to-channel context, geometry residual coding, and spherical quantization to efficiently encode sparse point cloud geometry.

S2C-3D is a label used in recent arXiv literature for two distinct 3D methods. In "Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes," S2C-3D denotes a sparse-view 3D reconstruction framework for recovering high-fidelity and complete scenes from as few as six to eight images by combining a specialized diffusion model, training-free view-consistency conditioned sampling, camera trajectory planning, and 3D Gaussian optimization (Shen et al., 7 May 2026). In a separate technical description associated with "Voxel-based Point Cloud Geometry Compression with Space-to-Channel Context," the same label is used for a voxel-based point cloud geometry compression system built around stage-wise and level-wise Space-to-Channel context models, Geometry Residual Coding, Residual Probability Approximation, and spherical quantization (Liu et al., 24 Mar 2025). The term therefore does not designate a single canonical algorithm; it denotes two systems that address sparse 3D information in different ways.

1. Terminological scope and disambiguation

The two uses of the label differ in task definition, input modality, and optimization target.

Usage of S2C-3D	Problem setting	Core mechanism
Sparse-to-Complete	Sparse-view scene reconstruction from 6–8 posed RGB images	Diffusion-based restoration, view-consistency conditioning, trajectory planning, 3D Gaussians
Space-to-Channel	Voxel-based point cloud geometry compression	Stage-wise and level-wise S2C context, GRC, RPA, spherical coordinates

In the reconstruction setting, S2C-3D is explicitly introduced as a framework for "high-fidelity and complete scene reconstruction from as few as six to eight images," with the objective of producing scenes "free from missing regions, blurring, or other artifacts" under very sparse inputs (Shen et al., 7 May 2026). In the compression setting, S2C-3D is presented as an end-to-end system for voxelized geometry coding in which spatial occupancy dependencies are reorganized into feature channels so that sparse-convolution receptive fields are preserved or enlarged across coding stages (Liu et al., 24 Mar 2025).

A common source of confusion is to treat these two usages as variants of a single method. The available literature does not support that interpretation. They are separate systems operating on different data types and optimizing different objectives.

2. Sparse-to-Complete S2C-3D: reconstruction problem and representation

In the sparse-view reconstruction formulation, the input is a sparse set of posed color images

$\{(I_j, K_j, E_j)\}_{j=1}^M,\quad M = 6 \ldots 8,$

where $I_j \in \mathbb{R}^{H \times W \times 3}$ is the $j$-th RGB image, $K_j$ is the camera intrinsics, and $E_j = (R_j, t_j) \in SE(3)$ is the extrinsics (Shen et al., 7 May 2026).

The target is a continuous 3D scene representation

$$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$

where $G^*$ is composed of $N$ anisotropic Gaussians,

$G^* = \{g_i\}_{i=1}^N,$

and each primitive is parameterized as

$g_i = \{\mu_i, s_i, q_i, \alpha_i, c_i\}.$

Here $I_j \in \mathbb{R}^{H \times W \times 3}$0 is the position, $I_j \in \mathbb{R}^{H \times W \times 3}$1 the scale, $I_j \in \mathbb{R}^{H \times W \times 3}$2 the orientation, $I_j \in \mathbb{R}^{H \times W \times 3}$3 the opacity, and $I_j \in \mathbb{R}^{H \times W \times 3}$4 the view-dependent color coefficients (Shen et al., 7 May 2026).

This representation places S2C-3D within the 3D Gaussian scene modeling family, but the paper’s stated goal is not merely faithful reprojection of observed views. It is to recover a scene that remains complete even in previously unseen regions. That emphasis on completion is structurally reflected in the combination of restoration, consistency guidance, and trajectory expansion.

3. Scene-specific diffusion restoration and view-consistency conditioning

A central component is a specialized diffusion model derived from a pretrained single-step diffusion prior, Difix3D+. Its base denoiser $I_j \in \mathbb{R}^{H \times W \times 3}$5 is a U-Net operating in latent space under a frozen VAE encoder/decoder. To adapt the model to a target scene, the method fine-tunes only low-rank adapters in the decoder (Shen et al., 7 May 2026).

The training pairs are constructed scene-specifically. For each input view, a degraded version is rendered by first adding Gaussian noise to $I_j \in \mathbb{R}^{H \times W \times 3}$6's parameters and then randomly masking out a subset of Gaussians to simulate occlusions. The corresponding ground truth is the original input image. The repair loss is

$I_j \in \mathbb{R}^{H \times W \times 3}$7

where $I_j \in \mathbb{R}^{H \times W \times 3}$8 is the diffusion denoiser plus decoder (Shen et al., 7 May 2026).

The baseline reverse diffusion step is written as

$I_j \in \mathbb{R}^{H \times W \times 3}$9

S2C-3D then adds a view-consistency conditioned sampling mechanism to mitigate multi-view conflicts among repaired images. For a rendered view $j$0, neighboring views are unprojected into 3D via their depth maps and reprojected onto view $j$1 to define a consistency energy

$j$2

Under a single-step approximation, the guided update becomes

$j$3

with $j$4 a scaling factor for the consistency gradient (Shen et al., 7 May 2026).

Algorithmically, the process alternates between rendering images from $j$5 along a planned trajectory, unconditional sampling of repaired images, optimization of $j$6 against these repaired views, consistency-gradient correction of the repaired images, and re-optimization. The paper characterizes this as a training-free conditioning process because the consistency term is injected into sampling of the frozen diffusion model without additional training.

A plausible implication is that S2C-3D treats scene restoration and geometric optimization as a coupled loop rather than as two separable stages. The repaired images are not merely outputs of a prior; they serve as intermediate supervision for the evolving Gaussian scene.

4. Camera trajectory planning and 3D Gaussian optimization

S2C-3D includes an explicit camera trajectory planning scheme designed to maximize coverage of occluded regions. Given two poses $j$7 and $j$8, the pose distance is defined as

$j$9

where $K_j$0 and $K_j$1 are unit quaternions for $K_j$2 and $K_j$3 (Shen et al., 7 May 2026).

A newly sampled camera pose is connected to its two nearest neighbors, and the method interpolates along both edges,

$K_j$4

Coverage is evaluated by an information-gain criterion. If $K_j$5 denotes the sphere-sample points visible in the frustum of camera $K_j$6, then the marginal gain is

$K_j$7

The planning loop repeatedly samples $K_j$8 in the OBB, finds its two nearest neighbors by the distance above, builds the interpolation path, computes the gain over all cameras in that path, and retains the path only if the gain exceeds a threshold $K_j$9 (Shen et al., 7 May 2026).

Once repaired and consistency-guided images are available, the Gaussian scene is optimized by matching rendered images $E_j = (R_j, t_j) \in SE(3)$0 to the target repaired images $E_j = (R_j, t_j) \in SE(3)$1:

$E_j = (R_j, t_j) \in SE(3)$2

The Gaussian parameters $E_j = (R_j, t_j) \in SE(3)$3 are refined by gradient descent via automatic differentiation (Shen et al., 7 May 2026).

The trajectory module is important because the framework is not restricted to the original sparse viewpoints. It actively constructs a better observation path for optimization. This distinguishes the method from pipelines that only densify or repair views at fixed camera poses.

5. Empirical performance of Sparse-to-Complete S2C-3D

The reconstruction framework is evaluated on ScanNet++ with 6–8 views, Replica with 4–6 views, and S2C-Scene with 4–6 views. Comparisons are reported against DNGaussian, Difix3D+, GenFusion, and VD-3DGS, using PSNR$E_j = (R_j, t_j) \in SE(3)$4, SSIM$E_j = (R_j, t_j) \in SE(3)$5, and LPIPS$E_j = (R_j, t_j) \in SE(3)$6 as metrics (Shen et al., 7 May 2026).

On the 6-view ScanNet++ exemplar, the reported quantitative results are as follows: 3DGS baseline achieves PSNR 13.06, SSIM 0.496, LPIPS 0.465; DNGaussian 12.83, 0.420, 0.571; Difix3D+ 13.32, 0.549, 0.405; GenFusion 16.61, 0.626, 0.371; VD-3DGS 16.54, 0.644, 0.395; and S2C-3D 17.78, 0.683, 0.344 (Shen et al., 7 May 2026).

The ablation study on Replica 6-view isolates the contribution of each component. Removing noise-based fine-tuning reduces performance to PSNR 19.81, SSIM 0.714, LPIPS 0.295. Removing mask-based fine-tuning gives 18.52, 0.653, 0.336. Removing all fine-tuning, corresponding to pure Difix3D+, gives 18.16, 0.640, 0.349. Removing camera trajectory planning gives 18.25, 0.645, 0.342. Removing view-consistency conditioning gives 18.13, 0.639, 0.353. The paper states that visual results confirm that each component is critical to achieving high fidelity, completeness, and multi-view coherence under extremely sparse inputs (Shen et al., 7 May 2026).

These results support a specific interpretation of the framework’s design: scene-specific diffusion adaptation, view-consistency conditioning, and trajectory planning are not interchangeable refinements. The reported ablations indicate that each addresses a distinct failure mode—artifact repair, cross-view coherence, and scene coverage, respectively. The source code and data are reported as available at https://gapszju.github.io/S2C-3D.

6. S2C-3D as Space-to-Channel geometry compression

In the compression literature, S2C-3D is described as a voxel-based point cloud geometry compression method that replaces spatial upsampling with channel expansion so that all 3D sparse convolutions remain at the same level resolution (Liu et al., 24 Mar 2025). At level $E_j = (R_j, t_j) \in SE(3)$7, each occupied voxel has a feature vector $E_j = (R_j, t_j) \in SE(3)$8, and the occupancies of its eight child sub-voxels are represented as

$E_j = (R_j, t_j) \in SE(3)$9

The joint distribution is factorized as a channel-wise autoregression,

$$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$0

For high-level sparse point clouds, the method introduces a level-wise S2C context with Geometry Residual Coding. Each voxel predicts the one occupied sub-voxel among its eight octants through a residual index $$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$1, trained with a cross-entropy loss. To enlarge the receptive field, Residual Probability Approximation uses large-kernel sparse convolution, specifically a $$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$2 kernel within an L-DFA block, and a two-group autoregression in which the voxel set is split into odd and even indices (Liu et al., 24 Mar 2025).

The method also uses spherical coordinates,

$$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$3

arguing that for LiDAR point clouds this yields a more compact representation than Cartesian voxelization. On KITTI and Ford, the description reports that by level 12 in $$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$4 one already has nearly the full point count, versus level 15–18 in $$G^* = \arg\min_G \mathcal{L}_{\text{scene}}(G;\{I_j,K_j,E_j\}_{j=1}^M),$$5 (Liu et al., 24 Mar 2025).

Reported dense lossless geometry compression results on MPEG 8i and Owlii give an average of 0.542 bpp for the stage-wise S2C method, compared with 0.567 for SparsePCGC and 0.920 for G-PCC. Encoding and decoding are also reported as faster than SparsePCGC: on MPEG 8i, 1.16 s encode and 1.06 s decode versus 1.60 s and 1.29 s; on Owlii, 2.84 s and 2.68 s versus 3.72 s and 3.17 s (Liu et al., 24 Mar 2025).

For sparse LiDAR, the level-wise S2C model is reported to save up to 16.4% on KITTI and 15.4% on Ford over SparsePCGC at high bitrates while matching or improving PSNR by 0.5–1.0 dB. In lossless KITTI compression, total bitrate is reported as 15.24 bpp, decomposed into 7.47 for stage, 7.31 for residual, and 0.46 for points, compared with 16.39 for SparsePCGC (Liu et al., 24 Mar 2025).

This alternative use of the acronym changes the meaning of "S2C" entirely. Here it denotes Space-to-Channel context rather than Sparse-to-Complete reconstruction. The shared label is therefore nominal rather than methodological.