Combined Geometry Encoding Volume (CGEV)

• CGEV is a multiscale volumetric representation that combines explicit, learned, and analytic geometric descriptors into a unified tensor or hierarchical structure.
• It enables geometry-aware processing across visual computing tasks, such as neural implicit surface reconstruction, stereo matching, and volumetric point cloud compression.
• The approach improves efficiency and accuracy through multi-scale feature fusion, sparse high-resolution grids, and iterative optimization, reducing memory use and accelerating convergence.

The Combined Geometry Encoding Volume (CGEV) paradigm refers to a class of multiscale volumetric representations that encode scene geometry, context, and feature information by combining multiple geometric descriptors (explicit, learned, or analytic) within a unified tensor or hierarchical structure. This approach enables geometry-aware processing across visual computing subfields including neural implicit surface reconstruction, dense correspondence estimation, and volumetric point cloud compression. CGEV methods can realize a richer, more query-efficient, and regularized geometric embedding than traditional single-scale or purely neural-network-based encodings. Recent instantiations include hierarchical multimode grid volumes in neural implicit models, tensor concatenations of geometry-aware and correlation-based features in stereo matching, and multiresolution B-spline wavelet transforms for unified geometry–attribute coding.

1. Formal Definitions and Scope

CGEV is defined, in the context of neural implicit surface reconstruction, as a fusion of explicit 3D feature volumes at various spatial scales, rather than reliance on a single MLP whose parameters provide an implicit, nonlocal encoding. Each level of the hierarchy captures distinct spatial frequencies: low-resolution volumes impart global consistency and smoothing, while high-resolution (fine) grids preserve high-frequency geometric details. The multi-scale features are typically concatenated to form the final geometry embedding, which can be consumed by a downstream decoder (e.g., SDF or color MLP) (Gu et al., 3 Aug 2024).

In deep stereo matching, CGEV is instantiated as a 4D tensor: $$\mathrm{CGEV}\;=\;V(x, y, d) \in \mathbb{R}^{C \times H \times W \times D}$$ where each channel includes geometry encoding features (from 3D UNet-regularized costs), local all-pairs correlation slices, and their multi-scale pooled variants. This tensor is indexed and updated iteratively within a recurrent optimization loop to yield correspondence estimates with enhanced accuracy and convergence (Xu et al., 2023).

In volumetric point cloud compression, the paradigm is realized by representing geometry as the zero-level set of a signed distance field (SDF) encoded via a B-spline wavelet basis at multiple resolutions, providing critically sampled, Lagrangian-optimized octree wavelet codes for both geometry and attributes (Krivokuća et al., 2018).

2. Methodological Implementations

Neural Implicit Surface Reconstruction (HIVE)

The HIVE architecture encodes geometry using a hierarchical stack of 3D feature volumes:

• Number of Levels: 8 dense (resolutions $2^3,\ldots,256^3$) for global-to-mid-level context; up to 2 additional sparse high-resolution volumes (512³, 1024³) for local details.
• Sparse Memory Structure: High-res stages store only near-surface voxels in a sparse embedded list, indexed by a table $T_i$ for each spatial location, minimizing memory use while preserving detail.
• Trilinear Feature Extraction: For a grid volume $V^\ell$, interpolation at point $x$ uses:

$$F^\ell(x) = \sum_{i,j,k} V^\ell_{i,j,k}(1 - |x_x-i|)(1 - |x_y-j|)(1 - |x_z-k|)$$

• Feature Fusion: The encoding at $x$ is $\mathcal{F}(x) = [F^1(x) \| \ldots \| F^L(x)]$, concatenating features from all levels.
• Regularization: Two losses are imposed: volume total variation (TV) to smooth learned embeddings and a normal-gradient term to regularize SDF curvature along rays (Gu et al., 3 Aug 2024).

Stereo Matching (IGEV-Stereo)

• Volume Construction: Combines geometry-encoding features (3D UNet filtered) and all-pairs correlation volumes with feature pooling along the disparity axis, yielding a multi-scale, context-rich 4D tensor.
• Indexing and Updates: The current disparity estimate indexes the CGEV at every step, extracting a local "slice" used by ConvGRU-based iterative refinement.
• Initialization and Losses: Soft-argmin on the geometry-encoding slice provides a high-quality initialization, followed by recurrent L1 losses weighted by an exponential schedule to optimize convergence (Xu et al., 2023).

Volumetric Point Cloud Compression (BV-SDF)

• SDF Level Set Representation: Surface as the zero-level set of an SDF estimated at spatial grid knots.
• B-spline Wavelet Transform: Hierarchical B-spline basis (orders $p\geq1$) is used to encode both geometry and attributes in a multi-resolution volumetric function space.
• Critical Sampling and Quantization: Analysis/synthesis filterbanks are applied recursively, with scalar quantization in the loop, ensuring bounded geometric/attribute error.
• Octree Pruning: Rate–distortion-optimized pruning reduces bitrates while guaranteeing perceptual fidelity (Krivokuća et al., 2018).

3. Regularization, Efficiency, and Quantitative Footprint

CGEV architectures incorporate specific design elements for regularization and computational efficiency:

• Volume TV suppresses high-frequency noise ("salt-and-pepper" artifacts).
• Normal-gradient regularization controls surface curvature in mesh extraction.
• Sparse high-resolution grids (in HIVE) restrict memory growth to sublinear in grid size: e.g., while a dense $1024^3$ grid at 4 channels would require $\sim$16 GB, the HIVE sparse/Banded architecture maintains total memory use under $\sim$1.5 GB (for all volumes), with empirically measured 10x-20x reductions at high resolution (Gu et al., 3 Aug 2024).
• ConvGRU-based updating with CGEV in stereo matching yields faster convergence and higher accuracy per update compared to all-pairs-only baselines: on Scene Flow, CGEV achieves EPE = 0.47 px after 32 iterations, compared to 0.56 for prior volumetric correlation-only approaches (Xu et al., 2023).
• In BV-based point cloud coding, critically sampled B-spline wavelets achieve up to 2 dB PSNR gains at low bitrate for attributes and substantially reduced geometry MSE at fixed bitrates, relative to MPEG G-PCC and Haar-based transforms (Krivokuća et al., 2018).

4. Multiscale Feature Fusion in CGEV

A core property is the fusion of multi-frequency (spatial bandwidth) descriptors in a unified representation. In HIVE, low-frequency (global) codes stabilize surface topology and smoothness, whereas high-frequency (local) codes permit sharp, detailed reconstruction near surfaces. In stereo, the concatenation within CGEV allows context-regularized, geometry-aware cost volumes to coexist with local correlation evidence, benefiting from both non-local and raw matching signals. In volumetric compression, B-spline multiresolution supports Lagrangian-optimized, wavelet-sparse codes that adapt per spatial block, balancing compressibility and appearance/perceptual fidelity.

A summary of CGEV feature sources in different domains:

Domain	Low-Resolution Content	High-Resolution Content	Fusion Strategy
Neural implicit reconstruction	Global shape priors	Local geometric detail	Feature concatenation
Stereo 4D matching	Regularized geometry (GEV)	Local all-pairs correlations	Multi-scale channel stacking
Point cloud compression	SDF low-pass coefficients	Wavelet detail coefficients	Level-wise B-spline transforms

5. Comparative Advantages and Empirical Results

CGEV consistently outperforms single-mode and single-resolution volumes:

• Neural Implicit Reconstruction: HIVE (CGEV) yields smoother, more detailed, and artifact-free reconstructions relative to pure MLP-based NeRF-like methods, with qualitative improvements demonstrated across diverse datasets (DTU, EPFL, BlendedMVS) (Gu et al., 3 Aug 2024).
• Stereo Matching: CGEV-based IGEV-Stereo improves endpoint error and reduces the number of recurrent updates required for convergence. For example, IGEV achieves comparable accuracy in 3-4 updates (∼100 ms) as previous methods did in 32 (∼440 ms) (Xu et al., 2023).
• Point Cloud Coding: B-spline (BV) volumetric transform achieves higher geometry PSNR at equivalent or lower bitrate, outperforming prior region-adaptive Haar and standard MPEG G-PCC pipelines (Krivokuća et al., 2018).

6. Context, Related Paradigms, and Practical Considerations

CGEV generalizes and extends several trends in geometry-driven machine perception:

• Explicit volumetric encodings address over-smoothing and "memorization" risks of pure MLP architectures by anchoring geometry to spatial tensors or control functions at multiple scales.
• Hierarchical and context-augmented volumes reconcile the need for both local fidelity (e.g., detailed disparity, color, surface detail) and global consistency (e.g., topological smoothness, error resilience).
• Practical deployments exploit sparsity structures, efficient interpolation, and joint quantization/sharing strategies to meet hardware limits and minimize latency or memory footprints. For instance, B-spline volumetric transforms naturally fit octree partitioning and multi-level entropy coding frameworks (Krivokuća et al., 2018).

A plausible implication is that future CGEV designs will further hybridize neural and analytic bases, exploiting task-driven multi-scale fusion for improved generalization, efficiency, and perceptual quality across vision, graphics, and robotics applications.

• HIVE: HIerarchical Volume Encoding for Neural Implicit Surface Reconstruction (Gu et al., 3 Aug 2024).
• Iterative Geometry Encoding Volume for Stereo Matching (Xu et al., 2023).
• A Volumetric Approach to Point Cloud Compression (Krivokuća et al., 2018).