Differentiable G-PCC Surrogate Model

• The paper introduces a surrogate model that replaces non-differentiable octree operations with neural approximations, enabling gradient-based optimization in G-PCC.
• It neutralizes quantization and discrete coding challenges using both voxelization network integration and Gaussian smoothing to promote smooth rate-distortion optimization.
• Empirical results reveal significant BD-rate reductions and minimal CPU overhead, validating the surrogate's efficacy in maintaining decoder performance while optimizing encoder parameters.

Searching arXiv for the cited papers and closely related G-PCC surrogate work. A differentiable G-PCC surrogate model is a learned proxy for the non-differentiable rate–distortion behaviour of geometry-based point cloud compression (G-PCC). In one formulation, the surrogate is integrated with a compression-oriented voxelization network and emulates octree-based geometry coding so that both rate and distortion can participate in end-to-end gradient propagation during training; at inference time only the lightweight voxelization network remains in front of the standard, unmodified G-PCC encoder, and no changes are made to the decoder (Ma et al., 3 Aug 2025). In a second formulation, adapted from the ZeroGrads framework, a differentiable model is fit online to a black-box G-PCC objective $f(\theta)=D(\theta)+\beta R(\theta)$ under local Gaussian smoothing, and its analytic gradient is then used to update codec parameters even when the true codec objective has undefined or zero gradients (Fischer et al., 2023).

1. Definition and conceptual scope

The literature describes two closely related uses of a differentiable surrogate around G-PCC. One use is codec emulation: the surrogate replaces non-differentiable internal stages of octree geometry coding by differentiable neural modules and supplies proxy rate and distortion terms during training. The other use is black-box optimization: a surrogate is trained locally against codec evaluations and used only as a differentiable approximation of the objective near the current parameter iterate. These formulations address different optimization loci—one at the level of learned point-cloud preprocessing, the other at the level of codec-parameter descent—but both target the same obstacle: the absence of usable gradients in the original pipeline (Ma et al., 3 Aug 2025, Fischer et al., 2023).

Formulation	Optimization target	Differentiability mechanism
Joint training with voxelization network	Input point cloud preprocessing before standard G-PCC	Neural replacements for octree-coding steps, plus BCE-based rate proxy
ZeroGrads-style local surrogate	Codec parameters $\theta$ in $f(\theta)=D(\theta)+\beta R(\theta)$	Online learned local objective surrogate under Gaussian active smoothing

The scope of the surrogate is therefore broader than a single architectural pattern. In the joint-training setting, the surrogate is part of a cascaded training system with a learned front-end. In the ZeroGrads-style setting, the surrogate is an auxiliary differentiable model that is re-fit online without pre-computed data or pre-trained models. This suggests that “differentiable G-PCC surrogate model” denotes a family of methods defined by function—making G-PCC-compatible optimization differentiable—rather than by one fixed network topology.

2. Sources of non-differentiability in G-PCC

The need for a surrogate arises because G-PCC contains highly non-continuous operations. In the black-box formulation, the cited obstacles include block-wise quantization, prediction-mode selection, and octree or connectivity flag bits; because G-PCC internally applies quantization and discrete branch decisions, $\nabla f(\theta)$ is undefined or zero in large parts of $\mathbb R^n$ (Fischer et al., 2023). In the joint-training formulation, the non-differentiable stages are described more structurally as context expansion through hand-crafted neighbour patterns $N$, context concentration through dynamic OBUF mapping, and arithmetic coding of each occupancy bit (Ma et al., 3 Aug 2025).

A key representational device is the multiscale sparse-tensor representation shared by the voxelization network and the surrogate. This representation admits a one-to-one correspondence with the octree nodes used by G-PCC, so that every learned operation—such as deleting a leaf node in the sparse tensor—corresponds exactly to removing an octree node in G-PCC (Ma et al., 3 Aug 2025). That correspondence is central: it constrains the differentiable model to operate on structures that remain meaningful to the deployed standard codec.

The two formulations differ in how they neutralize discontinuity. In the joint-training surrogate, non-differentiable steps are replaced by small neural sub-networks with Sigmoid outputs. In the ZeroGrads-style formulation, the codec remains a black box and Gaussian perturbations $\delta$ are used to smooth out quantization discontinuities in expectation, while the surrogate learns the local behaviour of the resulting smoothed objective (Ma et al., 3 Aug 2025, Fischer et al., 2023).

3. Mathematical formulations of the surrogate objective

For black-box G-PCC parameter optimization, the objective is written as

$f(\theta)=D(\theta)+\beta R(\theta),$

where $D(\theta)\ge 0$ is geometry-reconstruction error, $R(\theta)\ge 0$ is the number of bits used by the encoder, and $\theta$0 controls the rate–distortion trade-off. A differentiable surrogate $\theta$1 is trained on a locally smoothed version of $\theta$2 around the current iterate $\theta$3:

$\theta$4

The locality regularizer is chosen as

$\theta$5

which penalizes higher-order deviations and focuses network capacity on first-order behaviour around $\theta$6 (Fischer et al., 2023).

For the joint-training surrogate, the rate and distortion proxies are defined over occupancy probabilities. Let $\theta$7 be ground-truth occupancy bits of the scaled cloud $\theta$8, let $\theta$9 be the voxel-network occupancy probabilities, let $f(\theta)=D(\theta)+\beta R(\theta)$0 be the occupancy bits of the voxelized cloud $f(\theta)=D(\theta)+\beta R(\theta)$1, and let $f(\theta)=D(\theta)+\beta R(\theta)$2 be the surrogate’s predicted occupancy probabilities. With binary cross-entropy

$f(\theta)=D(\theta)+\beta R(\theta)$3

the differentiable distortion and rate terms are

$f(\theta)=D(\theta)+\beta R(\theta)$4

and the combined loss is

$f(\theta)=D(\theta)+\beta R(\theta)$5

Equivalently, with latent representation $f(\theta)=D(\theta)+\beta R(\theta)$6 produced by the voxelization network,

$f(\theta)=D(\theta)+\beta R(\theta)$7

Since $f(\theta)=D(\theta)+\beta R(\theta)$8 is a deterministic function of $f(\theta)=D(\theta)+\beta R(\theta)$9 via the frozen surrogate, $\nabla f(\theta)$0 remains differentiable in the voxelization-network weights $\nabla f(\theta)$1 (Ma et al., 3 Aug 2025).

The surrogate also preserves the causal structure of octree occupancy coding. The paper states

$\nabla f(\theta)$2

and replaces it by

$\nabla f(\theta)$3

where $\nabla f(\theta)$4 is a learned parent-node feature and $\nabla f(\theta)$5 are learned sibling-node features (Ma et al., 3 Aug 2025).

4. Architectural realizations

In the joint-training system, the full training architecture consists of two neural modules in cascade: the “versatile voxelization network” and the “differentiable G-PCC surrogate.” The voxelization network manipulates the input point cloud through global scaling, local pruning, and point-level editing. Its “Scale” branch applies a learned global quantization factor $\nabla f(\theta)$6 to the raw cloud $\nabla f(\theta)$7 to produce a scaled point cloud $\nabla f(\theta)$8. Its “Prune / edit” branch uses a sequence of sparse-conv downsampling blocks and a back-loaded sparse-conv upsampling block, ending in a per-leaf-node Sigmoid classifier that predicts occupancy probabilities $\nabla f(\theta)$9 (Ma et al., 3 Aug 2025).

Discretization is handled by straight-through rounding. The formulation is

$\mathbb R^n$0

in the forward pass, with $\mathbb R^n$1 in the backward pass. This STERound mechanism preserves the discrete octree structure while keeping the training graph differentiable (Ma et al., 3 Aug 2025).

The differentiable surrogate receives the binary occupancy bits $\mathbb R^n$2 produced by the voxelization network. A “Feature extraction” block replaces G-PCC’s hand-crafted neighbourhood context and produces a parent-node feature $\mathbb R^n$3. “Context expansion” and “feature aggregation” blocks replace dynamic OBUF and arithmetic coding by sequentially absorbing sibling-node features and emitting occupancy probabilities $\mathbb R^n$4 for each child bit. The resulting surrogate is differentiable end-to-end because neighbourhood-pattern lookup, dynamic OBUF mapping, and arithmetic coding are each replaced by small neural sub-networks with Sigmoid outputs (Ma et al., 3 Aug 2025).

In the ZeroGrads-style blueprint for G-PCC, the surrogate has a different role and a simpler stated realization. The codec remains untouched and is queried only through black-box evaluations of $\mathbb R^n$5 and $\mathbb R^n$6. The surrogate network $\mathbb R^n$7 is described as a small MLP, for example with 3 hidden layers of width 256, trained only around the current iterate $\mathbb R^n$8. Because it is fitted locally and regularized by $\mathbb R^n$9, it need only capture the complexity of the objective in a small patch of parameter space; the blueprint states that this locality makes the method scale to high dimensions, with $N$0 on the order of thousands, such as one quant-step per block (Fischer et al., 2023).

5. Training procedures and gradient flow

The joint-training formulation adopts a two-stage procedure. First, the surrogate is pretrained on true G-PCC outputs: a large set of point clouds is generated, the standard G-PCC voxelizer is run to obtain ground-truth occupancy bits $N$1, the sparse-tensor representation of $N$2 is passed into the surrogate, and the pretraining loss

$N$3

is minimized. After approximately 10 epochs, the surrogate is reported to have $N$4 that closely matches the real G-PCC bit-probabilities. Second, the surrogate is frozen and the voxelization network is jointly trained: raw cloud $N$5 voxelization network $N$6 and $N$7 via STERound, then $N$8 is computed from $N$9, $\delta$0 is computed by feeding $\delta$1 into the frozen surrogate, and the total loss $\delta$2 is back-propagated through both the voxelization network and the frozen surrogate branch into $\delta$3 (Ma et al., 3 Aug 2025).

The gradient path in this system is explicit. Because STERound uses a straight-through backward rule and all convolutional and Sigmoid layers are differentiable, gradients flow back to the voxelization-network weights that control global scaling, pruning, and editing. The surrogate therefore stands in for the real G-PCC encoder plus arithmetic coder: its cross-entropy loss provides a smooth $\delta$4 indicating how the occupancy pattern should change to reduce bit-cost, while the distortion term constrains fidelity to the original geometry (Ma et al., 3 Aug 2025).

The ZeroGrads-style formulation interleaves surrogate fitting and parameter descent online. At iteration $\delta$5, one draws a mini-batch of $\delta$6 perturbations $\delta$7, evaluates

$\delta$8

by running the G-PCC encoder/decoder, forms the surrogate-training batch $\delta$9, computes

$f(\theta)=D(\theta)+\beta R(\theta),$0

updates $f(\theta)=D(\theta)+\beta R(\theta),$1 by Adam or SGD, computes the surrogate gradient $f(\theta)=D(\theta)+\beta R(\theta),$2 at $f(\theta)=D(\theta)+\beta R(\theta),$3 by auto-diff, updates $f(\theta)=D(\theta)+\beta R(\theta),$4, and decreases the smoothing width by

$f(\theta)=D(\theta)+\beta R(\theta),$5

Typical budgets in the blueprint are $f(\theta)=D(\theta)+\beta R(\theta),$6 codec calls per iteration, $f(\theta)=D(\theta)+\beta R(\theta),$7, $f(\theta)=D(\theta)+\beta R(\theta),$8, decay $f(\theta)=D(\theta)+\beta R(\theta),$9, and $D(\theta)\ge 0$0 (Fischer et al., 2023).

6. Validation, efficiency, and interpretive issues

The empirical validation reported for the joint-training surrogate emphasizes end-to-end rate–distortion outcomes. The paper reports a 38.84% average BD-rate reduction over G-PCC. It also evaluates the surrogate as a lossless codec: on Owlii-basketball_player, G-PCC (octree) is reported at $D(\theta)\ge 0$1 bpp and the surrogate at $D(\theta)\ge 0$2 bpp, corresponding to a 28.3% bit-saving; across 12 test point clouds, the surrogate achieves 2–18% bpp reductions compared to octree G-PCC and up to approximately 58% over V-PCC. An ablation further states that replacing the trained surrogate with a naïve uniform model completely destroyed the RD improvement, whereas the trained surrogate yielded gradient steps that corresponded to real bit-savings (Ma et al., 3 Aug 2025).

The design trade-offs are also explicitly quantified. Back-loaded upsampling moves the costly transposed sparse-conv stage to the end of the voxelization network and reduces FLOPs by approximately 85.5% with only a 1.02% BD-rate penalty compared to a standard upsampling-in-the-middle design. Pretraining the surrogate to convergence and freezing it during joint optimization is described as preventing instabilities, specifically oscillation of surrogate parameters. Straight-through rounding is reported to preserve the discrete octree structure better than additive uniform-noise relaxation and to yield higher PSNR at matched bit-rates. At test time, only the voxelization network runs before the standard G-PCC encoder, the decoder is unchanged, and the system incurs only a few milliseconds of extra CPU time in voxelization with no change to the decoder pipeline (Ma et al., 3 Aug 2025).

For the ZeroGrads-style G-PCC blueprint, validation is presented as an evaluation plan rather than as reported G-PCC results. Success is defined along two axes: surrogate accuracy, measured by

$D(\theta)\ge 0$3

on held-out perturbations together with gradient alignment measured as the correlation between $D(\theta)\ge 0$4 and a finite-difference $D(\theta)\ge 0$5; and end-to-end rate–distortion gain, assessed by the RD-curve $D(\theta)\ge 0$6 versus $D(\theta)\ge 0$7 over $D(\theta)\ge 0$8 and by convergence speed in codec calls against derivative-free baselines such as CMA-ES, SPSA, and Nelder–Mead. The blueprint further proposes ablations on $D(\theta)\ge 0$9-smoothing and $R(\theta)\ge 0$0, and states expected outcomes such as surrogate collapse when $R(\theta)\ge 0$1 and an operating range $R(\theta)\ge 0$2 that balances overfitting against an overly rigid linear surrogate (Fischer et al., 2023).

Two misconceptions recur around this topic. First, a differentiable G-PCC surrogate does not imply differentiating through the deployed standard codec itself. In the joint-training formulation, the surrogate is used during training while inference uses the standard encoder and unchanged decoder; in the ZeroGrads-style formulation, the method never differentiates inside the codec and instead uses Gaussian perturbations to smooth discontinuities in expectation (Ma et al., 3 Aug 2025, Fischer et al., 2023). Second, the acronym should not be conflated with the conditional generalized Polynomial Chaos (gPC) surrogate model used for PDE-state approximation, where differentiability arises from GPR-conditioned KL expansions and Hermite-polynomial bases rather than from octree occupancy modeling (Li et al., 2019).