Textured Surfel Octree (TeSO) for 3D Streaming

• Textured Surfel Octree (TeSO) is a 3D representation that organizes surfels within an adaptive octree, each linked to a per-surfel texture patch for enhanced surface detail.
• It adapts surfel size based on local curvature, reducing the number of primitives by 5–10x while maintaining high-frequency appearance through precise texture mapping.
• TeSO integrates a compression scheme for both geometry and textures via learned entropy models and standard codecs, achieving 20–30% bitrate savings at comparable perceptual quality.

Searching arXiv for the specified paper and closely related baseline context. Textured Surfel Octree (TeSO) is a 3D representation for visual content streaming that is built from point clouds and designed to improve rendering quality, surface definition, and compressibility relative to point clouds, meshes, and 3D Gaussians. It represents a 3D scene as cube-bounded surfels organized on an octree, with each surfel associated with a texture patch. The central idea is to approximate smooth surface regions with larger surfels at coarser octree levels while retaining high-frequency appearance through per-surfel texture maps, and to couple this representation with a compression scheme for geometry and texture that leverages the octree structure. In the reported evaluation, the representation combined with its compression scheme achieves higher rendering quality at lower bit-rates than multiple point cloud and 3D Gaussian-based baselines (Hu et al., 9 Aug 2025).

1. Representation and geometric primitives

At the core of TeSO is an octree whose occupied leaf nodes each carry one surfel. A surfel at octree level $\ell$ is associated with an axis-aligned cube of side-length

$b_\ell = \frac{B}{2^\ell}$

where $B$ is the side-length of the root cube. Within that cube, TeSO stores three geometric attributes: an offset vector $o_\ell = P_\ell - c_\ell^A$, where $P_\ell$ is the surfel center and $c_\ell^A$ is the cube’s anchor corner; a unit-normal vector $n_\ell$, estimated as the average of per-point normals in that cube; and a radius $r_\ell$, chosen to cover all points in the cube. The surfel is thus parameterized as $(o_\ell, n_\ell, r_\ell)$, and its world-space center is

$c_\ell = c_\ell^A + o_\ell .$

This construction makes TeSO a hybrid between explicit point-derived geometry and hierarchical spatial partitioning. A plausible implication is that the representation separates local surface support from global scene organization: the cube provides the octree-aligned support region, while the surfel parameters encode the local geometric approximation.

2. Multilevel approximation and octree organization

TeSO is constructed from a fine, full-resolution point cloud, exemplified in the summary by a 10-level octree. The encoder greedily tests at each cube whether a single surfel $b_\ell = \frac{B}{2^\ell}$0 can approximate the local surface to within a D1–PSNR threshold $b_\ell = \frac{B}{2^\ell}$1. If the test succeeds, the cube becomes a leaf and all its points are marked “visited.” If it fails, the cube is split into its eight children and the test repeats at level $b_\ell = \frac{B}{2^\ell}$2.

The resulting hierarchy is adaptive. On smooth, nearly planar regions, the process terminates quickly at small $b_\ell = \frac{B}{2^\ell}$3, so a single large surfel covers a large area. On high-curvature or high-detail regions, cubes subdivide further. In practice, this reduces the number of primitives by $b_\ell = \frac{B}{2^\ell}$4–$b_\ell = \frac{B}{2^\ell}$5 compared to one-surfels-per-point (Hu et al., 9 Aug 2025).

Each node carries two bits: an occupancy bit, with $b_\ell = \frac{B}{2^\ell}$6 and $b_\ell = \frac{B}{2^\ell}$7, and, if occupied, a split-flag, with $b_\ell = \frac{B}{2^\ell}$8 and $b_\ell = \frac{B}{2^\ell}$9. Leaf surfels are stored only at levels $B$0 chosen by the encoder, with the summary giving $B$1 as an example. All unvisited points in leaf cubes are removed from further subdivision. Morton, or Z-, ordering is used for linear indexing of occupied cubes and later for texture-patch packing.

This organization yields what the summary describes as a compact, multiscale geometric representation that automatically adapts to local curvature, with explicit surfel connectivity implied by the octree and no holes. This suggests that TeSO is intended not merely as a storage structure, but as a geometry model whose hierarchy is directly tied to approximation error and coding efficiency.

3. Texture patches and 3D-to-2D parameterization

Every leaf surfel at level $B$2 carries a square $B$3D texture patch $B$4 of size $B$5 pixels, covering a $B$6 region on the surfel’s tangent plane. The reported experimental configuration allows

$B$7

so coarser surfels carry larger patches and finer surfels carry smaller ones.

The tangent frame $B$8 is constructed by rotating the canonical $B$9-axis onto $o_\ell = P_\ell - c_\ell^A$0. One efficient method uses the quaternion

$o_\ell = P_\ell - c_\ell^A$1

Rotating $o_\ell = P_\ell - c_\ell^A$2 by $o_\ell = P_\ell - c_\ell^A$3 yields $o_\ell = P_\ell - c_\ell^A$4.

A texel with integer coordinates $o_\ell = P_\ell - c_\ell^A$5 maps to world space as

$o_\ell = P_\ell - c_\ell^A$6

To fill $o_\ell = P_\ell - c_\ell^A$7, the method finds the $o_\ell = P_\ell - c_\ell^A$8 nearest points from the original point cloud around that plane location, with $o_\ell = P_\ell - c_\ell^A$9, weights them by inverse distance, and averages their colors (Hu et al., 9 Aug 2025).

The significance of this design is that geometric simplification and appearance fidelity are decoupled. TeSO reduces the number of geometric primitives through larger surfels on smooth regions, yet preserves high-frequency texture details through the attached texture maps. A plausible implication is that this is the mechanism by which the representation avoids the usual trade-off in which aggressive geometric simplification directly erodes visual detail.

4. Geometry and texture compression

TeSO includes a compression scheme for both geometry and texture. For geometry, the base octree up to level $P_\ell$0 is coded with the standard G-PCC occupancy codec. For levels $P_\ell$1, the method losslessly codes, in order per node: the occupancy bit, predicted by a sparse $P_\ell$2D CNN over a rasterized context volume of neighboring nodes; the split/leaf flag, coded via CABAC on the Morton-ordered leaf bits; the offset $P_\ell$3, with each coordinate quantized to step $P_\ell$4 over $P_\ell$5; the normal $P_\ell$6, using octahedral quantization to two components $P_\ell$7 with step $P_\ell$8; and the radius $P_\ell$9, quantized with step $c_\ell^A$0 over $c_\ell^A$1.

Each of these elements is entropy-coded by an adaptive arithmetic coder using probabilities estimated by the same sparse CNN, conditioned on previously decoded attributes. The summary also formulates a possible rate–distortion optimization objective,

$c_\ell^A$2

where $c_\ell^A$3 might be average D1–PSNR or rendering error, but notes that in practice TeSO fixes $c_\ell^A$4 and chooses $c_\ell^A$5.

Two alternatives were explored for texture encoding. The first packs all $c_\ell^A$6 patches in Morton order into a single large atlas image and codes that image with AV1 intra $c_\ell^A$7. The second treats every texel $c_\ell^A$8 as a colored $c_\ell^A$9D point, using the known location $n_\ell$0 and applying G-PCC to encode only the color attribute. In the reported experiments, the second method gave slightly better R–D but higher CPU decoding cost, while AV1 was much faster in software.

For geometry, the lossless bit-cost lower bound is expressed as the cross-entropy

$n_\ell$1

with

$n_\ell$2

For textures, distortion can be measured by MSE or PSNR on rendered images:

$n_\ell$3

However, LPIPS is used as the primary perceptual distortion metric for rendering quality.

5. Rendering model

TeSO is rendered through a surfel rasterization pipeline operating on camera rays $n_\ell$4. For each pixel, the renderer first collects all surfels whose cubes might intersect the ray, using a simple z-buffer of surfel centers. These surfels are then sorted by distance and tested in turn. The intersection parameter is computed as

$n_\ell$5

The renderer checks whether $n_\ell$6 lies within the cube bounds, with a small “soft” margin, and whether $n_\ell$7. If the hit is “solid,” meaning inside the cube, it picks the bilinearly interpolated color from the patch $n_\ell$8 at the corresponding $n_\ell$9 in tangent-plane coordinates and stops. If the hit lies in the “soft” skirt region, outside the cube face but within a Gaussian fall-off, the renderer blends by a factor

$r_\ell$0

and continues testing.

The summary characterizes the resulting behavior as real-time rendering with soft-skirt blending at arbitrary viewpoints. The mention of a simplified but watertight surface indicates that the renderer is designed to avoid the gap and overlap artifacts that often accompany direct point splatting, while still permitting smooth transitions at surfel boundaries (Hu et al., 9 Aug 2025).

6. Empirical comparison and reported advantages

TeSO was compared against three baselines: MPEG G-PCC with lossy geometry and lossy color plus OpenGL splatting; G-PCC plus learned $r_\ell$1D-Gaussian rendering, identified as P2ENet; and the end-to-end learned “Bits-to-Photon” (B2P), which decodes $r_\ell$2D Gaussians. On four 8iVFB human scans, at resolutions $r_\ell$3 and $r_\ell$4, TeSO achieved approximately $r_\ell$5–$r_\ell$6 lower LPIPS at the same bpp, or $r_\ell$7–$r_\ell$8 bitrate savings at iso-quality (Hu et al., 9 Aug 2025).

The visual comparisons reported in the summary attribute distinct failure modes to the baselines. Standard splatting leaves gaps or overlaps; learned Gaussians blur high-frequency textures; and B2P over-smooths complex clothing patterns. By contrast, TeSO is described as preserving sharp textures on a simplified but watertight surface.

The conclusion in the summary enumerates four consequences of the design: a compact, multiscale geometric representation that automatically adapts to local curvature; explicit surfel connectivity implied by the octree, with no holes; very dense color detail via attached texture; and efficient end-to-end compression using a learned octree entropy model plus standard image and point-cloud codecs. It further states that TeSO compresses to $r_\ell$9–$(o_\ell, n_\ell, r_\ell)$0 bpp while providing better perceptual quality than prior point-cloud or learned-Gaussian approaches. Within the scope of the reported experiments, these results position TeSO as a representation oriented toward low-latency streaming in $(o_\ell, n_\ell, r_\ell)$1D telepresence and AR/VR systems.

A common misconception would be to treat TeSO as merely an octree of points with textures attached. The summary instead presents it as a representation in which the surfel, not the original point, is the operative geometric primitive, and in which texture patches are tied to tangent-plane parameterizations rather than to per-point attributes. Another possible misconception would be to reduce its performance claims to geometry compression alone; the reported evaluation emphasizes perceptual rendering quality, with LPIPS as the primary distortion metric, indicating that the method is assessed at the level of rendered appearance rather than solely point-set fidelity.