Sparse Volume Textures (SVT) in Real-Time 3D Rendering

• Sparse Volume Textures (SVT) are a tiling-based data representation that partitions massive 3D volumes into non-empty, padded tiles for efficient in-core rendering.
• The method leverages mipmap pyramids and a sparse 3D page table to ensure continuous lighting and accurate trilinear interpolation across tile boundaries.
• Key challenges include managing 32-bit integer overflows, GPU memory limits, and developing out-of-core tile streaming to scale beyond a 4 GiB data transfer cap.

Sparse Volume Textures (SVT) are a memory- and bandwidth-efficient data representation for large, sparse 3D volume datasets, supporting efficient in-core rendering at high resolutions in real-time engines such as Unreal Engine 5 (UE5). SVT achieves scalability by partitioning volumetric data into small fixed-size tiles, densely packing only non-empty tiles, and managing virtual-to-physical tile mappings via a sparse 3D page table. This design enables real-time rendering of gigavoxel-scale scientific and visual effects datasets, notably in UE5’s Heterogeneous Volumes (HV) system, and addresses major challenges in chunk-based approaches, including discontinuous lighting across chunk boundaries and virtual memory limitations (Schlüter et al., 10 Apr 2025).

1. Data Structures and Memory Layout

SVT organizes a 3D volume of size $V_x \times V_y \times V_z$ into a regular grid of tiles, each of size $T \times T \times T$ (with $T=16$ in UE5). The total tile grid has dimensions:

$$N_x = \lceil V_x / T \rceil,\quad N_y = \lceil V_y / T \rceil,\quad N_z = \lceil V_z / T \rceil$$

Only tiles containing at least one occupied (nonzero) voxel are retained. These are tightly packed in a single 3D "physical tile data texture." Metadata is maintained in a 3D page table of size $N_x \times N_y \times N_z$; each entry stores either a flattened index to its tile within the physical texture or a sentinel value if empty.

Every tile contains its full mipmap pyramid (successively halved versions down to $1\times 1\times 1$), concatenated alongside base-level data in the physical tile texture. To support correct trilinear interpolation at tile boundaries, each tile (at every mip level) is stored with a one-voxel border on all sides, yielding a practical per-tile storage of $(T+2)^3$ voxels (e.g., $18^3=5832$ voxels at base level for $T=16$).

Addressing formulas for sampling and mip-level fetching use a combination of:

• Discrete tile coordinates $\mathbf{t}=(i,j,k)$ and in-tile offsets $\mathbf{u}=(u,v,w)$.
• Page table lookup $p=\text{PageTable}[i,j,k]$ to retrieve the physical tile index.
• Concatenated mip-levels tracked by precomputed offsets $M_\ell$.

For sampling at mip-level $\ell$, the physical texture coordinate is:

$$\mathbf{P}_{\rm phys}^{(\ell)} = (p + M_\ell) \, T_{\rm stride}^{(\ell)} + \mathbf{u}^{(\ell)} + \mathbf{pad}^{(\ell)}$$

where $T_{\rm stride}^{(\ell)}$ is the tile layout stride for mip-level $\ell$ and $\mathbf{u}^{(\ell)}$ maps local coordinates at the down-sampled resolution.

2. Compression and Storage Limits

SVT leverages sparse storage to maximize in-memory and on-disk efficiency:

• On the CPU, an 8-bit occupancy mask or non-empty voxel count is computed per tile, filtering only non-empty tiles for upload.
• Data on disk (e.g., OpenVDB format) is maintained with a hierarchical grid structure. Upon import into SVT, voxel trees are flattened into tile lists and the associated page table.

Each tile stores all mip-levels with appropriate padding, resulting in a per-tile voxel count:

$$N_{\rm voxels\,per\,tile} = \sum_{\ell=0}^L (\lceil T/2^\ell \rceil + 2 )^3, \qquad L = \lfloor \log_2 T \rfloor$$

Maximum storage capacity is governed by several constraints:

• UE5’s implementation on DirectX 12 supports up to a $2048^3 \approx 8\,\textrm{Gvoxels}$ virtual texture reservation.
• Given tile padding and mip pyramids (total mip-factor $\sum_{\ell=0}^\infty (1/2^\ell)^3 \approx 1.143$), the practical net usable payload is:

$$2048^3 \times \frac{16^3}{18^3} \div 1.143 \approx 4.9\,\textrm{Gvoxels}$$

• CPU–GPU upload buffer size and GPU ByteAddressBuffer offsets are constrained to 32-bit integers, capping the total transferred/compressed SVT data per upload at approximately 4 GiB.

For example, the Kolumbo dataset with $4211 \times 935 \times 1501 \approx 6.0\times10^9$ voxels but only $2.6$ G non-empty voxels fits the SVT pipeline after tiling, padding, and mip-generation, remaining within the imposed 4 GiB upload limit.

3. Engineering Challenges in SVT Implementation

SVT development encountered multiple high-impact implementation issues:

• Integer Overflow: With large datasets (tile counts approaching $2^{31}$), signed int32 fields in the import and compression routines led to crashes. Replacement with unsigned int32 (tile counts) and uint64 (total occupied voxels) eliminated the overflow risk for large inputs:

  - int32 TileCount; // overflow at >2³¹ + uint32 TileCount; // safe up to 2³²−1 - int32 TotalOccupiedVoxels; // overflow at >2³¹ + uint64 TotalOccupiedVoxels; // safe for massive datasets

• Texture Memory Size Reporting: The RHI memory size computation previously returned a 32-bit value, preventing correct handling for physical tile textures ⩾4 GiB. Conversion to a uint64 return type enabled support for larger allocations.
• Lighting Artifacts at Chunk Boundaries: Chunked approaches (e.g. TBRayMarcher or Niagara) require splitting volumes into sub-datasets <1 Gvoxel each, each with an independent illumination cache. At chunk seams, missing information introduces lighting discontinuities (bright seams). By contrast, the HV/SVT system maintains a single global illumination cache and unified field representation, guaranteeing continuous lighting and trilinear filtering across all tile boundaries.

4. Performance Characteristics

SVT delivers interactive rendering performance within in-core (fully resident in VRAM) constraints:

• On an RTX 3500 Ada laptop GPU, a 6 Gvoxel (2.6 G non-empty) HV dataset renders at approximately 20 fps at WQXGA resolution with DownsampleFactor=4 and MaxStepCount=1024.
• On an RTX 5000 desktop GPU, the same scene achieves ~10 fps, with differences arising from GPU driver overhead and clock speeds rather than VRAM availability.
• Performance predominantly depends on raymarching step count, lighting cache resolution, and output pixel resolution, not strictly on total data size, provided all tiles are loaded in VRAM.

The SVT system thus amortizes data access and rendering costs over sparse, compactly stored non-empty portions of a vast 3D domain.

5. Out-of-Core Operation and Scalability Prospects

Current SVT deployments are limited by the 4 GiB CPU-GPU transfer and addressing boundaries. To exceed in-core limits, an out-of-core tile streaming extension is necessary:

• The existing page table mechanism would be extended to support a CPU/GPU-driven streaming manager. Tiles would be paged into VRAM on-demand, managed by a sliding window over a smaller GPU buffer, and transferred in discrete chunks.
• 64-bit arithmetic in GPU tile-addressing shaders (e.g., UpdateSparseVolumeTexture.usf) becomes mandatory to handle offsets greater than 4 GiB. This incurs a performance penalty (approximately 4–8× slower than 32-bit), but this can be restricted to streaming rather than per-frame material evaluation.
• A hybrid design, combining DX12 reserved resources for virtual-texture backing and an out-of-core tile cache manager, could remove both the 4 GiB upload constraint and the $~5$ Gvoxel storage ceiling, thereby enabling volumes up to the metadata-imposed $32{\rm k}\times 32{\rm k}\times 16{\rm k}$.

6. Comparative Advantages and Limitations

SVT with the HV approach fundamentally improves volume rendering scalability and visual quality compared to chunked/naïve systems:

• Continuity: One global logical field and lighting cache eliminates discontinuities and interpolation artifacts at chunk boundaries.
• Storage Efficiency: Only non-empty tiles are stored and transferred, with physical memory proportional to features of interest.
• Scalability Constraints: While SVT enables massive datasets in-core, hard limits from 32-bit pointer ranges must be overcome via tile streaming and 64-bit address support.

A plausible implication is that future SVT developments will enable seamless handling of petavoxel-scale volumes for both visualization and scientific computation as VRAM and graphics API architectures evolve (Schlüter et al., 10 Apr 2025).