---
title: Differentiable G-PCC Surrogate Model
url: https://www.emergentmind.com/topics/differentiable-g-pcc-surrogate-model
type: topic
---

# Differentiable G-PCC Surrogate Model

Searching arXiv for the cited papers and closely related G-PCC surrogate work.
A differentiable G-PCC surrogate model is a learned proxy for the non-differentiable rate–distortion behaviour of geometry-based point cloud compression (G-PCC). In one formulation, the surrogate is integrated with a compression-oriented voxelization network and emulates octree-based geometry coding so that both rate and distortion can participate in end-to-end gradient propagation during training; at inference time only the lightweight voxelization network remains in front of the standard, unmodified G-PCC encoder, and no changes are made to the decoder [2508.01633]. In a second formulation, adapted from the ZeroGrads framework, a differentiable model is fit online to a black-box G-PCC objective $f(\theta)=D(\theta)+\beta R(\theta)$ under local Gaussian smoothing, and its analytic gradient is then used to update codec parameters even when the true codec objective has undefined or zero gradients [2308.05739].

## 1. Definition and conceptual scope

The literature describes two closely related uses of a differentiable surrogate around G-PCC. One use is codec emulation: the surrogate replaces non-differentiable internal stages of octree geometry coding by differentiable neural modules and supplies proxy rate and distortion terms during training. The other use is black-box optimization: a surrogate is trained locally against codec evaluations and used only as a differentiable approximation of the objective near the current parameter iterate. These formulations address different optimization loci—one at the level of learned point-cloud preprocessing, the other at the level of codec-parameter descent—but both target the same obstacle: the absence of usable gradients in the original pipeline [2508.01633][2308.05739].

| Formulation | Optimization target | Differentiability mechanism |
|---|---|---|
| Joint training with voxelization network | Input point cloud preprocessing before standard G-PCC | Neural replacements for octree-coding steps, plus BCE-based rate proxy |
| ZeroGrads-style local surrogate | Codec parameters $\theta$ in $f(\theta)=D(\theta)+\beta R(\theta)$ | Online learned local objective surrogate under Gaussian active smoothing |

The scope of the surrogate is therefore broader than a single architectural pattern. In the joint-training setting, the surrogate is part of a cascaded training system with a learned front-end. In the ZeroGrads-style setting, the surrogate is an auxiliary differentiable model that is re-fit online without pre-computed data or pre-trained models. This suggests that “differentiable G-PCC surrogate model” denotes a family of methods defined by function—making G-PCC-compatible optimization differentiable—rather than by one fixed network topology.

## 2. Sources of non-differentiability in G-PCC

The need for a surrogate arises because G-PCC contains highly non-continuous operations. In the black-box formulation, the cited obstacles include block-wise quantization, prediction-mode selection, and octree or connectivity flag bits; because G-PCC internally applies quantization and discrete branch decisions, $\nabla f(\theta)$ is undefined or zero in large parts of $\mathbb R^n$ [2308.05739]. In the joint-training formulation, the non-differentiable stages are described more structurally as context expansion through hand-crafted neighbour patterns $N$, context concentration through dynamic OBUF mapping, and arithmetic coding of each occupancy bit [2508.01633].

A key representational device is the multiscale sparse-tensor representation shared by the voxelization network and the surrogate. This representation admits a one-to-one correspondence with the octree nodes used by G-PCC, so that every learned operation—such as deleting a leaf node in the sparse tensor—corresponds exactly to removing an octree node in G-PCC [2508.01633]. That correspondence is central: it constrains the differentiable model to operate on structures that remain meaningful to the deployed standard codec.

The two formulations differ in how they neutralize discontinuity. In the joint-training surrogate, non-differentiable steps are replaced by small neural sub-networks with Sigmoid outputs. In the ZeroGrads-style formulation, the codec remains a black box and Gaussian perturbations $\delta$ are used to smooth out quantization discontinuities in expectation, while the surrogate learns the local behaviour of the resulting smoothed objective [2508.01633][2308.05739].

## 3. Mathematical formulations of the surrogate objective

For black-box G-PCC parameter optimization, the objective is written as
$$
f(\theta)=D(\theta)+\beta R(\theta),
$$
where $D(\theta)\ge 0$ is geometry-reconstruction error, $R(\theta)\ge 0$ is the number of bits used by the encoder, and $\beta>0$ controls the rate–distortion trade-off. A differentiable surrogate $s_\phi:\mathbb R^n\to\mathbb R$ is trained on a locally smoothed version of $f$ around the current iterate $\theta_t$:
$$
L_s(\phi;\theta_t)=
E_{\delta\sim\mathcal N(0,\sigma_t^2 I)}
\bigl[(s_\phi(\theta_t+\delta)-f(\theta_t+\delta))^2\bigr]
+\lambda\,R_{\text{locality}}(\phi;\theta_t).
$$
The locality regularizer is chosen as
$$
R_{\text{locality}}(\phi;\theta_t)=
E_{\delta\sim\mathcal N(0,\sigma_t^2I)}
\left\|s_\phi(\theta_t+\delta)-\left[s_\phi(\theta_t)+\nabla_\theta s_\phi(\theta_t)\cdot\delta\right]\right\|^2,
$$
which penalizes higher-order deviations and focuses network capacity on first-order behaviour around $\theta_t$ [2308.05739].

For the joint-training surrogate, the rate and distortion proxies are defined over occupancy probabilities. Let $b_i^s\in\{0,1\}$ be ground-truth occupancy bits of the scaled cloud $P^s$, let $p_i^c$ be the voxel-network occupancy probabilities, let $b_j^v\in\{0,1\}$ be the occupancy bits of the voxelized cloud $P^v$, and let $p_j^s$ be the surrogate’s predicted occupancy probabilities. With binary cross-entropy
$$
B(b,p)=-\,b\log p-(1-b)\log(1-p),
$$
the differentiable distortion and rate terms are
$$
\hat D(\theta)=\sum_{i=1}^{N_s} B(b_i^s,p_i^c), \qquad
\hat R(\theta)=\sum_{j=1}^{N_v} B(b_j^v,p_j^s),
$$
and the combined loss is
$$
L(\theta)=\hat D(\theta)+\lambda\cdot \hat R(\theta).
$$
Equivalently, with latent representation $z$ produced by the voxelization network,
$$
\hat D(z)=\sum_i B\bigl(b_i^s,p_i^c(z)\bigr),\qquad
\hat R(z)=\sum_j B\bigl(b_j^v,p_j^s(z)\bigr),\qquad
L(\theta)=\hat D(z(\theta))+\lambda\,\hat R(z(\theta)).
$$
Since $p^s$ is a deterministic function of $z$ via the frozen surrogate, $L$ remains differentiable in the voxelization-network weights $\theta$ [2508.01633].

The surrogate also preserves the causal structure of octree occupancy coding. The paper states
$$
H(b_0\ldots b_7\mid N)=H(b_0\mid N)\prod_{i=1}^7 H(b_i\mid N,b_0\ldots b_{i-1})
$$
and replaces it by
$$
H(b_0\ldots b_7\mid f_N)=H(b_0\mid f_N)\prod_{i=1}^7 H(b_i\mid f_N,f_{b_0}\ldots f_{b_{i-1}}),
$$
where $f_N$ is a learned parent-node feature and $f_{b_k}$ are learned sibling-node features [2508.01633].

## 4. Architectural realizations

In the joint-training system, the full training architecture consists of two neural modules in cascade: the “versatile voxelization network” and the “differentiable G-PCC surrogate.” The voxelization network manipulates the input point cloud through global scaling, local pruning, and point-level editing. Its “Scale” branch applies a learned global quantization factor $s$ to the raw cloud $P$ to produce a scaled point cloud $P^s$. Its “Prune / edit” branch uses a sequence of sparse-conv downsampling blocks and a back-loaded sparse-conv upsampling block, ending in a per-leaf-node Sigmoid classifier that predicts occupancy probabilities $p_i^c$ [2508.01633].

Discretization is handled by straight-through rounding. The formulation is
$$
S(p_i^c)=
\begin{cases}
1 & \text{if } p_i^c\ge 0.5,\\
0 & \text{otherwise,}
\end{cases}
$$
in the forward pass, with $\partial S/\partial p_i^c = 1$ in the backward pass. This STERound mechanism preserves the discrete octree structure while keeping the training graph differentiable [2508.01633].

The differentiable surrogate receives the binary occupancy bits $b_j^v$ produced by the voxelization network. A “Feature extraction” block replaces G-PCC’s hand-crafted neighbourhood context and produces a parent-node feature $f_N$. “Context expansion” and “feature aggregation” blocks replace dynamic OBUF and arithmetic coding by sequentially absorbing sibling-node features and emitting occupancy probabilities $p_i^s$ for each child bit. The resulting surrogate is differentiable end-to-end because neighbourhood-pattern lookup, dynamic OBUF mapping, and arithmetic coding are each replaced by small neural sub-networks with Sigmoid outputs [2508.01633].

In the ZeroGrads-style blueprint for G-PCC, the surrogate has a different role and a simpler stated realization. The codec remains untouched and is queried only through black-box evaluations of $D(\theta)$ and $R(\theta)$. The surrogate network $s_\phi$ is described as a small MLP, for example with 3 hidden layers of width 256, trained only around the current iterate $\theta_t$. Because it is fitted locally and regularized by $R_{\text{locality}}$, it need only capture the complexity of the objective in a small patch of parameter space; the blueprint states that this locality makes the method scale to high dimensions, with $n$ on the order of thousands, such as one quant-step per block [2308.05739].

## 5. Training procedures and gradient flow

The joint-training formulation adopts a two-stage procedure. First, the surrogate is pretrained on true G-PCC outputs: a large set of point clouds is generated, the standard G-PCC voxelizer is run to obtain ground-truth occupancy bits $b_j^v$, the sparse-tensor representation of $b^v$ is passed into the surrogate, and the pretraining loss
$$
L_p=\sum_{j=1}^{N_v} B(b_j^v,p_j^s)
$$
is minimized. After approximately 10 epochs, the surrogate is reported to have $p^s$ that closely matches the real G-PCC bit-probabilities. Second, the surrogate is frozen and the voxelization network is jointly trained: raw cloud $\rightarrow$ voxelization network $\rightarrow p_i^c$ and $b_j^v$ via STERound, then $\hat D$ is computed from $p_i^c$, $\hat R$ is computed by feeding $b^v$ into the frozen surrogate, and the total loss $L=\hat D+\lambda\hat R$ is back-propagated through both the voxelization network and the frozen surrogate branch into $p^c$ [2508.01633].

The gradient path in this system is explicit. Because STERound uses a straight-through backward rule and all convolutional and Sigmoid layers are differentiable, gradients flow back to the voxelization-network weights that control global scaling, pruning, and editing. The surrogate therefore stands in for the real G-PCC encoder plus arithmetic coder: its cross-entropy loss provides a smooth $\partial L/\partial z$ indicating how the occupancy pattern should change to reduce bit-cost, while the distortion term constrains fidelity to the original geometry [2508.01633].

The ZeroGrads-style formulation interleaves surrogate fitting and parameter descent online. At iteration $t$, one draws a mini-batch of $M$ perturbations $\delta_i\sim \mathcal N(0,\sigma_t^2 I)$, evaluates
$$
f_i=f(\theta_t+\delta_i)=D(\theta_t+\delta_i)+\beta R(\theta_t+\delta_i)
$$
by running the G-PCC encoder/decoder, forms the surrogate-training batch $\{(\theta_t+\delta_i,f_i)\}$, computes
$$
\nabla_\phi L_s(\phi;\theta_t)\approx
\frac{1}{M}\sum_{i=1}^M 2\,(s_\phi(\theta_t+\delta_i)-f_i)\,\nabla_\phi s_\phi(\theta_t+\delta_i)
+\lambda\,\nabla_\phi R_{\text{locality}}(\phi;\theta_t),
$$
updates $\phi\leftarrow \phi-\alpha_\phi \nabla_\phi L_s$ by Adam or SGD, computes the surrogate gradient $g_t=\nabla_\theta s_\phi(\theta)$ at $\theta_t$ by auto-diff, updates $\theta_{t+1}=\theta_t-\alpha_\theta g_t$, and decreases the smoothing width by
$$
\sigma_{t+1}=\max(\sigma_{\min},\rho \sigma_t).
$$
Typical budgets in the blueprint are $M=20\ldots 100$ codec calls per iteration, $\alpha_\phi\approx 10^{-3}$, $\sigma_0\sim 10^{-1}$, decay $\rho\approx 0.99$, and $\sigma_{\min}\sim 10^{-3}$ [2308.05739].

## 6. Validation, efficiency, and interpretive issues

The empirical validation reported for the joint-training surrogate emphasizes end-to-end rate–distortion outcomes. The paper reports a 38.84% average BD-rate reduction over G-PCC. It also evaluates the surrogate as a lossless codec: on Owlii-basketball_player, G-PCC (octree) is reported at $0.646$ bpp and the surrogate at $0.464$ bpp, corresponding to a 28.3% bit-saving; across 12 test point clouds, the surrogate achieves 2–18% bpp reductions compared to octree G-PCC and up to approximately 58% over V-PCC. An ablation further states that replacing the trained surrogate with a naïve uniform model completely destroyed the RD improvement, whereas the trained surrogate yielded gradient steps that corresponded to real bit-savings [2508.01633].

The design trade-offs are also explicitly quantified. Back-loaded upsampling moves the costly transposed sparse-conv stage to the end of the voxelization network and reduces FLOPs by approximately 85.5% with only a 1.02% BD-rate penalty compared to a standard upsampling-in-the-middle design. Pretraining the surrogate to convergence and freezing it during joint optimization is described as preventing instabilities, specifically oscillation of surrogate parameters. Straight-through rounding is reported to preserve the discrete octree structure better than additive uniform-noise relaxation and to yield higher PSNR at matched bit-rates. At test time, only the voxelization network runs before the standard G-PCC encoder, the decoder is unchanged, and the system incurs only a few milliseconds of extra CPU time in voxelization with no change to the decoder pipeline [2508.01633].

For the ZeroGrads-style G-PCC blueprint, validation is presented as an evaluation plan rather than as reported G-PCC results. Success is defined along two axes: surrogate accuracy, measured by
$$
\text{MSE}_{\text{surr}}=E_\delta\bigl[(s_\phi(\theta_t+\delta)-f(\theta_t+\delta))^2\bigr]
$$
on held-out perturbations together with gradient alignment measured as the correlation between $\nabla_\theta s_\phi(\theta_t)$ and a finite-difference $\nabla_\theta f(\theta_t)$; and end-to-end rate–distortion gain, assessed by the RD-curve $D(\theta_t)$ versus $R(\theta_t)$ over $t$ and by convergence speed in codec calls against derivative-free baselines such as CMA-ES, SPSA, and Nelder–Mead. The blueprint further proposes ablations on $\sigma$-smoothing and $\lambda$, and states expected outcomes such as surrogate collapse when $\sigma_t=0$ and an operating range $\lambda\in[10^{-2},10^{-1}]$ that balances overfitting against an overly rigid linear surrogate [2308.05739].

Two misconceptions recur around this topic. First, a differentiable G-PCC surrogate does not imply differentiating through the deployed standard codec itself. In the joint-training formulation, the surrogate is used during training while inference uses the standard encoder and unchanged decoder; in the ZeroGrads-style formulation, the method never differentiates inside the codec and instead uses Gaussian perturbations to smooth discontinuities in expectation [2508.01633][2308.05739]. Second, the acronym should not be conflated with the conditional generalized Polynomial Chaos (gPC) surrogate model used for PDE-state approximation, where differentiability arises from GPR-conditioned KL expansions and Hermite-polynomial bases rather than from octree occupancy modeling [1908.00424].

Source: https://www.emergentmind.com/topics/differentiable-g-pcc-surrogate-model