Multi-Res Hexplane Encoder

• Multi-resolution Hexplane encoder is a 4D scene representation method that decomposes space-time features into six orthogonal 2D planes using wavelet analysis.
• It employs multi-scale resolution pyramids and sparse hard-thresholding with hash-map compression to drastically reduce storage while preserving image fidelity.
• The architecture seamlessly integrates with NeRF and Gaussian Splatting renderers, achieving faster training and robust dynamic view synthesis across static and dynamic scenes.

A multi-resolution Hexplane structure encoder is a compact and efficient architecture for representing 4D dynamic scenes, based on decomposing space-time features into six orthogonal two-dimensional feature planes, with each plane encoding multi-scale information via wavelet analysis. This methodology is central to recent advances in dynamic novel view synthesis and 3D neural scene representations, where the primary goals are improved training speed, reduced storage, and high-fidelity view synthesis across both static and dynamic scenes (Azzarelli et al., 2023, Cao et al., 2023).

1. Multi-Resolution Hexplane Feature Architecture

The core principle of a Hexplane encoder is to organize a volumetric 4D function $f(x, y, z, t)$ into six explicit 2D feature planes, corresponding to each unique pair among the four coordinates: $$C=\{\textrm{xy}, \textrm{xz}, \textrm{yz}, \textrm{xt}, \textrm{yt}, \textrm{zt}\}$$ (Azzarelli et al., 2023, Cao et al., 2023). Each feature plane stores trainable vectors. In multi-resolution variants, these planes are further organized into a resolution pyramid across several levels $r=1,\dots,L$, with increasingly finer spatial and temporal grids.

For each level $r$, the six planes (e.g., $$\textrm{Plane}_{\textrm{XY}}^{(r)} \in \mathbb{R}^{S_x^{(r)} \times S_y^{(r)} \times F_r}$$) store feature vectors at discretized 2D grids. Coordinate normalization is used so that input points $(x, y, z, t)$ are mapped into $[0, 1]^4$, and then to integer grid coordinates at each resolution via $u_x^{(r)} = \hat{x} \cdot (S_x^{(r)} - 1)$, etc. (Cao et al., 2023).

2. Wavelet Domain Parameterization and Inverse Synthesis

WavePlanes (Azzarelli et al., 2023) innovates by storing each plane not as a dense feature grid, but as a set of multi-scale wavelet coefficient tensors. For each plane $c \in C$, the representation consists of:

• A low-pass ("father") coefficient tensor $$\Omega_{c0} \in \mathbb{R}^{B \times \lceil H/2^N \rceil \times \lceil W/2^N \rceil}$$
• For $s=1,...,N$, "mother" wavelet detail tensors $$\Omega_{cs} \in \mathbb{R}^{B \times 3 \times \lceil H/2^{N-s+1} \rceil \times \lceil W/2^{N-s+1} \rceil}$$ (three directions: LH, HL, HH)

The feature plane $P_c(x, y)$ is reconstructed via a multi-scale inverse discrete wavelet transform (IDWT):

$$P_c(x,y) = \sum_{i,j} a_0 \phi_{i,j}(x,y) + \sum_{s=1}^N \sum_{m \in \{\textrm{LH, HL, HH}\}} \sum_{u,v} d_{sm,u,v} \psi_{m,s;u,v}(x,y)$$

where $a_0 = \Omega_{c0}[i,j]$, $d_{sm,u,v} = \Omega_{cs}[m,u,v]$.

Trainable per-scale scaling controls gradient flow to each band. Space-time planes ($c \in \{\textrm{xt}, \textrm{yt}, \textrm{zt}\}$) have a +1 bias to encourage a default empty value (Azzarelli et al., 2023).

3. Sparse Hard-Thresholding and Hash-Map Compression

Since wavelet domains exhibit signal concentration, hard-thresholding is applied after training:

$$\Omega_{cs}[i,j] \leftarrow \Omega_{cs}[i,j] \cdot 1\left[|\Omega_{cs}[i,j]| > \tau\right]$$

Typically $\tau \approx 0.1$ effectively eliminates over 90% of coefficients with negligible PSNR loss. Only nonzero coefficients and their linear indices are stored in per-plane, per-scale hash-maps. This achieves $10$–$15\times$ compression (e.g., $<20$ MB for a dynamic scene at $256^2$ resolution compared to $200$ MB dense) (Azzarelli et al., 2023).

A final pass using lossless codecs (LZMA superior to BZ2, GZIP, or Pickle) further reduces storage. The precise number of surviving coefficients $K$ is typically $5$–$10\%$ of original, so storage is $O(K)$ floats plus $O(K)$ indices.

4. Feature Extraction, Fusion, and NeRF Integration

For each query $(x, y, z, t)$, projections $\pi_c(q)$ map the point onto each 2D plane, yielding six coordinates. Bilinear interpolation samples feature vectors from fine and coarse planes at each projection. Features across resolutions and planes are fused.

WavePlanes introduces the Zero-Agreement Mask (ZAM) fusion:

$$f(q) = \text{concat}_{r=0,1}\left[\frac{1}{3} \sum_{ct \in \{\textrm{xt},\textrm{yt},\textrm{zt}\}} f_{cr,ct}(q) \cdot \prod_{c \in C} f_{cr,c}(q)\right]$$

Only if all three space-time planes agree on zero does the fused representation become zero, preserving sharp motion boundaries (Azzarelli et al., 2023).

The fused feature vector $f(q)\in\mathbb{R}^{6B}$ feeds a scene-specific basis-expansion MLP for density $\sigma$ and view-dependent color $c$. Standard NeRF volume rendering $\int T(t)\sigma(q(t))c(q(t), d)dt$ yields rendered colors. The encoder can also drive a Gaussian Splatting renderer; the hexplane logic remains, only the final scene integrator changes (Azzarelli et al., 2023).

5. Empirical Results and Ablations

Detailed experiments on LLFF, synthetic D-NeRF, and real DyNeRF scenes demonstrate that with only $6 \times 2$ planes (fine and coarse) at $256^2$ resolution and $B=64$, WavePlanes matches or outperforms K-Planes in PSNR/SSIM, while reducing storage requirements by $5$–$15\times$ and incurring no extra training time on a single RTX 3090 GPU (Azzarelli et al., 2023).

Key ablation findings include:

Variable	Options/Ablation	Storage/Quality Impact
Wavelet family	Haar (db1), db6, coif4	Coif4 (PSNR ≈ 31.44) maxes regularity/support
Decomposition depth	$N=2$, $N=3$	$N=2$ gives $\approx$10% less GPU time, no PSNR loss
Threshold $\tau$	$0.1$, $0.0$	$>90\%$ coeff. pruned at $\tau=0.1$, PSNR loss $<0.02$
Codec	LZMA, GZIP, BZ2, Pickle	LZMA yields $<$18MB serializations

6. Connections to Prior Encoding Paradigms

Hexplane encoders generalize prior plane-based feature architectures (e.g., K-Planes, HexPlane) by extending explicit 4D spatial-temporal representations to multi-resolution and wavelet-compressed domains (Azzarelli et al., 2023, Cao et al., 2023). The hash-map sparsification leverages wavelet coefficient sparsity (analogous to SBHex/BBHex on hexagonal grids in 2D image coding (Rashid et al., 2021)) for rate–distortion optimality at low storage.

The mathematical basis of multi-scale decomposition follows bi-orthogonal wavelet theory, e.g., the Cohen–Schlenker bi-orthogonal bases, but is adapted for Cartesian rather than hexagonal lattice filtering—though hexagonal sampling retains advantages in 2D coding contexts (Rashid et al., 2021).

7. Performance Considerations and Applicability

WavePlanes and its multi-resolution hexplane encoder provide dramatic gains in storage efficiency (10–15×), faster training/inference versus classical implicit MLP NeRFs, and robustness for both static and dynamic view synthesis. The encoder preserves scene fidelity across a variety of wavelet families and decomposition depths, and the zero-prioritizing fusion method maintains edge sharpness under motion. These approaches integrate seamlessly with NeRF or Gaussian Splatting, and the design supports further improvements with more advanced coding/quantization, region-of-interest decoding, or lossless lifting extensions, as explored in related image coding work (Azzarelli et al., 2023, Rashid et al., 2021, Cao et al., 2023).