--- title: TSDF-to-Dense ESDF Perception url: https://www.emergentmind.com/topics/tsdf-to-dense-esdf-perception type: topic --- # TSDF-to-Dense ESDF Perception TSDF-to-Dense ESDF Perception, also known as the transformation from Truncated Signed Distance Fields (TSDF) to dense Euclidean Signed Distance Fields (ESDF), constitutes the algorithmic foundation for accurate, real-time distance-based perception in robotics and spatial AI. This pipeline is central to tasks requiring dense 3D environmental understanding, such as motion planning, collision avoidance, and surface reconstruction. Modern approaches employ both explicit (e.g., volumetric grids, voxel hashing with distance transforms) and implicit (learned neural representations) pipelines, enabling trade-offs among memory footprint, computational throughput, and sub-voxel map fidelity. Two notable systems that exemplify state-of-the-art TSDF-to-ESDF methodologies are LGSDF [2404.05187] and cuRoboV2 [2603.05493]. ## 1. TSDF Construction and Fusion The TSDF aggregates raw observations into a volumetric field, assigning each voxel a signed distance to the nearest observed surface, truncated to a maximum band $\tau$. In conventional pipelines, per-frame depth images are projected into world coordinates. For each voxel center $x$, the truncated signed distance is computed as $$ d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right) $$ where $z(u)$ is the observed depth at pixel $u$, and $p$ is the optical center. cuRoboV2 maintains a persistent, block-sparse TSDF allocation at millimeter resolution, fusing depth sensor data and analytic primitives, with each voxel storing both a depth-fused SDF and an analytic SDF channel. Voxels are updated with a weighted running average, where measurement confidence is adaptively scaled by the voxel’s projected area in the image, favoring observations closer to the camera. To account for scene dynamics, total voxel weights are recursively decayed and sparsely recycled [2603.05493]. LGSDF uses active 3D point sampling per incoming RGB-D frame, guided by pixel-block irregularity, and fuses the result into axis-aligned grids. Each volume cell $v$ stores an average distance $D_t(v)$, an accumulated weight $W_t(v)$, and an average gradient $G_t(v)$. Updates are performed using a heuristic exponential weight function on the signed distance. Over time, $D_t(v)$ converges to the locally fused SDF value at that grid location [2404.05187]. ## 2. Dense ESDF Generation After TSDF fusion, ESDF construction involves propagating signed Euclidean distances from surface boundaries throughout the workspace, yielding a field that can be queried at arbitrary points or voxel centers. cuRoboV2 employs a three-stage approach: 1. **Site seeding**: Detects zero-crossings in the TSDF at the ESDF grid resolution via gather-based sampling. 2. **Distance transform**: Utilizes the Parallel Banding Algorithm (PBA⁺) to perform exact, separable 3D distance propagation (forward and backward 1D passes along $X$, $Y$, $Z$ axes). This yields per-voxel squared distances and associated nearest-site indices. 3. **Sign recovery**: Assigns the ESDF sign, using the TSDF sign within the truncation band and analytic SDF signs for primitives, with default positivity for unobserved/missing blocks. This pipeline is implemented as a lock-free, GPU-native kernel with block-sparse storage and trilinear lookups for $O(1)$ query latency [2603.05493]. LGSDF approaches ESDF construction as a continual, implicit regression task: a compact multi-layer perceptron (MLP) is trained to approximate the fused SDF values and gradients throughout the workspace. The MLP, $f_\theta: \mathbb{R}^3 \to \mathbb{R}$, is conditioned on positional encodings of query locations and is updated incrementally via self-supervised gradient descent. Each update batch contains both current frame grid cells and randomly subsampled historically updated cells, preserving map plasticity and stability [2404.05187]. ## 3. Algorithmic and Hardware Optimizations cuRoboV2’s GPU pipeline uses block-sparse hashing (8³-voxel blocks), gather-based seeding to eliminate atomics in site selection, and CUDA Graph capture for all ESDF computation passes. Parallel key deduplication, lock-free block allocation (using compare-and-swap), and on-demand recycling ensure memory efficiency and rapid adaptation to changing scenes. Synchronization utilizes warp-level primitives ($\_\_syncwarp()$) throughout core kernels. Performance on NVIDIA RTX 4090 demonstrates a per-frame ESDF generation time of 1.69 ms (for 10 mm resolution over the full coverage zone) and up to $\sim$7.3× less memory usage than previous state-of-the-art (e.g., nvblox), with up to 99% collision recall [2603.05493]. LGSDF achieves both lightweight storage ($\sim$1 MB for network weights plus hash-grid) and rapid update rates ($\sim$0.2 s per incoming frame with $N_I=10$ gradient steps), supporting real-time mapping scenarios. By locally fusing raw TSDF signals into grid proxies and distilling them into MLPs, the system yields smoothly varying, densely queryable ESDFs for planning and mesh extraction [2404.05187]. ## 4. Loss Functions and Continual Learning In LGSDF, the ESDF neural field is optimized using a composite loss: - **Free-space loss** for $|D|\geq T$: $$ \mathcal{L}_{fs}(f_\theta(v), D) = \max \big(0, e^{-\beta f_\theta(v)} - 1, f_\theta(v) - D\big) $$ - **Near-surface loss** for $|D|