TSDF-to-Dense ESDF Perception

• TSDF-to-Dense ESDF Perception is a transformation pipeline that converts truncated signed distance fields into dense, Euclidean distance fields for precise spatial understanding.
• It integrates explicit volumetric fusion and implicit neural regression methods to balance memory efficiency, computational speed, and map fidelity.
• Optimized for real-time applications, it supports tasks like motion planning, collision avoidance, and surface reconstruction in dynamic robotic environments.

TSDF-to-Dense ESDF Perception, also known as the transformation from Truncated Signed Distance Fields (TSDF) to dense Euclidean Signed Distance Fields (ESDF), constitutes the algorithmic foundation for accurate, real-time distance-based perception in robotics and spatial AI. This pipeline is central to tasks requiring dense 3D environmental understanding, such as motion planning, collision avoidance, and surface reconstruction. Modern approaches employ both explicit (e.g., volumetric grids, voxel hashing with distance transforms) and implicit (learned neural representations) pipelines, enabling trade-offs among memory footprint, computational throughput, and sub-voxel map fidelity. Two notable systems that exemplify state-of-the-art TSDF-to-ESDF methodologies are LGSDF (Yue et al., 2024) and cuRoboV2 (Sundaralingam et al., 5 Mar 2026).

1. TSDF Construction and Fusion

The TSDF aggregates raw observations into a volumetric field, assigning each voxel a signed distance to the nearest observed surface, truncated to a maximum band $\tau$. In conventional pipelines, per-frame depth images are projected into world coordinates. For each voxel center $x$, the truncated signed distance is computed as

$$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$

where $z(u)$ is the observed depth at pixel $u$, and $p$ is the optical center.

cuRoboV2 maintains a persistent, block-sparse TSDF allocation at millimeter resolution, fusing depth sensor data and analytic primitives, with each voxel storing both a depth-fused SDF and an analytic SDF channel. Voxels are updated with a weighted running average, where measurement confidence is adaptively scaled by the voxel’s projected area in the image, favoring observations closer to the camera. To account for scene dynamics, total voxel weights are recursively decayed and sparsely recycled (Sundaralingam et al., 5 Mar 2026).

LGSDF uses active 3D point sampling per incoming RGB-D frame, guided by pixel-block irregularity, and fuses the result into axis-aligned grids. Each volume cell $v$ stores an average distance $D_t(v)$, an accumulated weight $W_t(v)$, and an average gradient $G_t(v)$. Updates are performed using a heuristic exponential weight function on the signed distance. Over time, $x$0 converges to the locally fused SDF value at that grid location (Yue et al., 2024).

2. Dense ESDF Generation

After TSDF fusion, ESDF construction involves propagating signed Euclidean distances from surface boundaries throughout the workspace, yielding a field that can be queried at arbitrary points or voxel centers.

cuRoboV2 employs a three-stage approach:

1. Site seeding: Detects zero-crossings in the TSDF at the ESDF grid resolution via gather-based sampling.
2. Distance transform: Utilizes the Parallel Banding Algorithm (PBA⁺) to perform exact, separable 3D distance propagation (forward and backward 1D passes along $x$1, $x$2, $x$3 axes). This yields per-voxel squared distances and associated nearest-site indices.
3. Sign recovery: Assigns the ESDF sign, using the TSDF sign within the truncation band and analytic SDF signs for primitives, with default positivity for unobserved/missing blocks.

This pipeline is implemented as a lock-free, GPU-native kernel with block-sparse storage and trilinear lookups for $x$4 query latency (Sundaralingam et al., 5 Mar 2026).

LGSDF approaches ESDF construction as a continual, implicit regression task: a compact multi-layer perceptron (MLP) is trained to approximate the fused SDF values and gradients throughout the workspace. The MLP, $x$5, is conditioned on positional encodings of query locations and is updated incrementally via self-supervised gradient descent. Each update batch contains both current frame grid cells and randomly subsampled historically updated cells, preserving map plasticity and stability (Yue et al., 2024).

3. Algorithmic and Hardware Optimizations

cuRoboV2’s GPU pipeline uses block-sparse hashing (8³-voxel blocks), gather-based seeding to eliminate atomics in site selection, and CUDA Graph capture for all ESDF computation passes. Parallel key deduplication, lock-free block allocation (using compare-and-swap), and on-demand recycling ensure memory efficiency and rapid adaptation to changing scenes. Synchronization utilizes warp-level primitives ($x$6) throughout core kernels. Performance on NVIDIA RTX 4090 demonstrates a per-frame ESDF generation time of 1.69 ms (for 10 mm resolution over the full coverage zone) and up to $x$77.3× less memory usage than previous state-of-the-art (e.g., nvblox), with up to 99% collision recall (Sundaralingam et al., 5 Mar 2026).

LGSDF achieves both lightweight storage ($x$81 MB for network weights plus hash-grid) and rapid update rates ($x$90.2 s per incoming frame with $$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$0 gradient steps), supporting real-time mapping scenarios. By locally fusing raw TSDF signals into grid proxies and distilling them into MLPs, the system yields smoothly varying, densely queryable ESDFs for planning and mesh extraction (Yue et al., 2024).

4. Loss Functions and Continual Learning

In LGSDF, the ESDF neural field is optimized using a composite loss:

• Free-space loss for $$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$1:

$$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$2

• Near-surface loss for $$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$3:

$$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$4

• Gradient loss (cosine distance):

$$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$5

• Eikonal loss (enforcing $$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$6 in free space):

$$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$7

The full loss is

$$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$8

where $$d_t(x) = \mathrm{clamp}_{[-\tau, \tau]} \left( z(u) - \|x - p\| \right)$$9 selects $z(u)$0 or $z(u)$1 depending on $z(u)$2.

This continual learning scheme ensures that the ESDF model remains accurate across both previously observed and novel regions while memory constraints are decoupled from scene complexity (Yue et al., 2024).

5. Evaluation Metrics and Comparative Analysis

Key quantitative metrics for ESDF perception include:

• SDF error [cm]: Mean absolute difference between predicted ESDF values and ground-truth distances at sampled points.
• Mesh completion [cm]: Mean nearest distance from dense ground-truth mesh vertices to the predicted zero-level-set surface.

LGSDF demonstrates lowest SDF error across benchmarks in ReplicaCAD and ScanNet scenes (e.g., 2.95 cm for LGSDF vs. 3.46 cm for iSDF), and best mesh completion scores (e.g., 9.82 cm vs. 11.15 cm), outperforming explicit Voxblox/VDB-EDT and implicit iSDF/SHINE baselines (Yue et al., 2024).

cuRoboV2 achieves up to 99% collision recall over relevant motion planning ROIs, 7.5× speedup and 7.3× memory reduction for ESDF generation compared to nvblox, and supports full dense distance field coverage required for high-DoF robotic manipulation and humanoid planning (Sundaralingam et al., 5 Mar 2026).

Method	SDF Error [cm]	Mesh Completion [cm]	Memory (ESDF gen)	Speedup
LGSDF	2.95	9.82	~1MB + grid	–
iSDF	3.46	11.15	(implicit)	–
cuRoboV2	n/a	n/a	1.63 GB (full)	7.5× vs nvblox

LGSDF’s ablation studies indicate that the pixel-regularity-guided sampling and grid-based fusion both contribute 10–20% accuracy improvement (Yue et al., 2024).

6. Applications and Extensions

TSDF-to-dense ESDF pipelines underpin a range of downstream applications:

• Real-time robotic motion generation with high-DoF manipulators and humanoids, supporting safe, collision-free trajectory planning even with dynamic scenes (Sundaralingam et al., 5 Mar 2026).
• Accurate surface mesh extraction for mapping and localization, with robust zero-level set determination in partially observed environments (Yue et al., 2024).
• Autonomous policy learning where sub-voxel ESDF gradients inform continuous control and optimal collision avoidance.

cuRoboV2 demonstrates scalability for manipulation and locomotion policies, achieving low tracking error, high collision-free IK success, and efficient memory utilization (Sundaralingam et al., 5 Mar 2026). LGSDF provides globally smooth, artifact-free ESDFs suitable for entire-scene semantic reasoning and long-horizon planning (Yue et al., 2024).

7. Limitations and Future Prospects

A persistent challenge remains balancing map update speed, memory requirements, and map completion accuracy as workspace or sensor resolution scales. Explicit pipelines such as cuRoboV2 achieve low latency and memory via block-sparse layouts and fixed-kernel parallel distance transforms but depend on explicit storage proportional to the explored volume. Implicit methods like LGSDF offer sublinear memory for global representations, but training speed and query latency remain bottlenecks, especially for large-scale continual learning.

A plausible implication is that hybrid pipelines—explicit grid fusion as front-end and neural ESDF regression as back-end—provide robustness against sensor conflicts while allowing for memory-efficient, densely queryable map representations. Further, block recycling and adaptive sampling strategies are likely to remain essential for handling dynamic scenes and mitigating map drift.

Both cuRoboV2 and LGSDF provide reference implementations for the community, indicating an ongoing evolution toward fully GPU-native, scalable, and dynamics-aware TSDF-to-dense ESDF perception stacks (Yue et al., 2024, Sundaralingam et al., 5 Mar 2026).