Neural Terrain Maps (NTM): Research Overview

• Neural Terrain Maps are continuous neural representations that model terrain geometry as implicit functions, allowing precise off-grid querying and analytical derivative computation.
• They replace conventional raster DEMs with learned parameterizations, supporting diverse applications from GIS analysis to robotic local mapping and neural rendering.
• Recent approaches incorporate spectral control, adaptive sampling, and differentiable rendering, achieving high accuracy, efficient compression, and real-time performance.

Searching arXiv for papers on Neural Terrain Maps and closely related implicit terrain representations. arxiv_search(query="Neural Terrain Maps terrain implicit neural representation DEM ImplicitTerrain ImplicitTerrainV2", max_results=10, sort_by="submittedDate") Searching arXiv for neural elevation models and terrain mapping via neural rendering. arxiv_search(query="terrain mapping neural elevation models neural volume rendering digital terrain map", max_results=10, sort_by="submittedDate") Neural Terrain Maps (NTMs) are neural representations of terrain geometry that replace or augment conventional raster digital elevation models (DEMs) with learned functions over space. In the DEM setting, one precise formulation is an implicit neural function $f_\theta:[0,1]^2\to\mathbb R$ that reproduces grid-sampled elevations at DEM coordinates, generalizes smoothly off-grid, admits analytical derivatives, and stores terrain as learned parameters rather than as a dense raster (Feng et al., 21 May 2026). In the cited literature, the same label is also used for NeRF-style terrain recovery from multi-view satellite imagery, self-supervised DEM inpainting networks, real-time dense elevation mapping with uncertainty, and sparse 4D scene reconstruction for locomotion (Biberstein et al., 2 Aug 2025, Stölzle et al., 2021, Yang et al., 2022, Hoeller et al., 2022). The term therefore denotes a research family rather than a single canonical architecture.

1. Terminology, scope, and relation to conventional terrain models

Raster DEMs remain the dominant representation in GIS, but they require interpolation for off-grid sampling and finite-difference operators for derivative-based analysis (Feng et al., 21 May 2026). NTM formulations replace those discrete procedures with continuous neural fields, coordinate-based decoders, or learned completion modules. In the terrain-analysis literature, the central abstraction is a height field $z=f(x,y)$, typically realized by an MLP; in robotic mapping, the same idea is often coupled to uncertainty estimation, scene completion, or temporal fusion; in neural rendering, it is embedded in a differentiable image-formation model (Feng et al., 2024, Yang et al., 2022, Biberstein et al., 2 Aug 2025).

A common misconception is that NTM denotes a single standardized data structure. The cited papers indicate otherwise: some NTMs are compact implicit terrain tiles for GIS, some are robot-centric local maps, and some are view-consistent terrain models learned directly from imagery. This suggests that the unifying characteristic is not a specific network family, but the replacement of explicit sampled terrain grids by neural parameterizations that support terrain queries, reconstruction, or downstream optimization.

Formulation	Representative papers	Defining characteristic
Implicit DEM tile	(Feng et al., 21 May 2026, Feng et al., 2024)	Continuous height field with derivative access
Neural elevation + radiance field	(Dai et al., 2024)	Joint height and appearance within a NeRF framework
Satellite neural terrain map	(Biberstein et al., 2 Aug 2025)	Height and texture learned from multi-view satellite imagery
Self-supervised DEM inpainting	(Stölzle et al., 2021)	U-Net completion of occluded elevation cells
Real-time dense elevation mapping	(Yang et al., 2022)	Bayesian-GAN reconstruction with pixel-wise uncertainty
Sparse 4D scene reconstruction	(Hoeller et al., 2022)	Robot-centric 3D terrain completion with temporal feedback

2. Continuous implicit NTMs for DEM tiles

The most explicit NTM-as-data-format formulation appears in ImplicitTerrainV2, where an NTM is “an implicit neural function” $f_\theta:[0,1]^2\to\mathbb R$ that exactly reconstructs grid-sampled elevations, supports off-grid height and derivative queries in closed form, and compresses to a few bits per pixel via standard quantization and entropy coding (Feng et al., 21 May 2026). The model uses a cascade of two SIREN-based networks: a shape network $s_\phi$ for a Gaussian-smoothed manifold and a geometry network $g_\psi$ for the residual. Both share a first layer of the form

$h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$

with frozen $(W_0,b_0)$ encoding an integer 2D Fourier basis and $M(x)$ acting as a spatial mask. By freezing the first layer on a grid of integer frequencies up to bandlimit $\mathcal B$, the framework obtains direct control over the representable Fourier spectrum (Feng et al., 21 May 2026).

ImplicitTerrain, the earlier continuous surface model, uses a related SIREN formulation $z=f(x,y)$ with a two-stage Surface-plus-Geometry (SPG) procedure. Stage A fits a surface model $z=f(x,y)$0 to a progressively smoothed DEM; Stage B fits a geometry compensation model $z=f(x,y)$1 to the residual $z=f(x,y)$2, yielding $z=f(x,y)$3 (Feng et al., 2024). In the reported experiments, the model uses $z=f(x,y)$4 layers, each hidden layer of width $z=f(x,y)$5, Adam with learning rate $z=f(x,y)$6, and $z=f(x,y)$7 steps per stage (Feng et al., 2024).

ImplicitTerrainV2 extends that line in four directions. First, it introduces explicit spectral control through the frozen Fourier first layer. Second, it adds a wavelet complexity field (WCF), computed from a two-level Haar stationary wavelet transform of the residual, with feature vector

$z=f(x,y)$8

A small CNN decoder maps the normalized features to a scalar complexity field $z=f(x,y)$9, which is tile-normalized to $f_\theta:[0,1]^2\to\mathbb R$0 and converted into strictly nested band masks

$f_\theta:[0,1]^2\to\mathbb R$1

through cumulative-softplus thresholds (Feng et al., 21 May 2026). Third, the same field drives adaptive sampling with

$f_\theta:[0,1]^2\to\mathbb R$2

with $f_\theta:[0,1]^2\to\mathbb R$3, concentrating training on rugged regions (Feng et al., 21 May 2026). Fourth, the pipeline applies post-training mixed-precision quantization and arithmetic coding, with shape weights at $f_\theta:[0,1]^2\to\mathbb R$4, geometry weights and WCF decoder at $f_\theta:[0,1]^2\to\mathbb R$5, plus a 4-bit quantized low-resolution complexity field (Feng et al., 21 May 2026).

The reported results make the data-format interpretation concrete. On $f_\theta:[0,1]^2\to\mathbb R$6 Swiss terrain tiles, ImplicitTerrainV2 reaches end-to-end PSNR $f_\theta:[0,1]^2\to\mathbb R$7 $f_\theta:[0,1]^2\to\mathbb R$8, MAE $f_\theta:[0,1]^2\to\mathbb R$9, MaxAE $s_\phi$0, with model size $s_\phi$1 K parameters, training time $s_\phi$2 s/tile on a single RTX A5000, and compressed storage $s_\phi$3 bpp at a $s_\phi$4 PSNR drop (Feng et al., 21 May 2026). ImplicitTerrain reports full SPG-to-input fitting PSNR $s_\phi$5, SSIM $s_\phi$6, $s_\phi$7 MB of weights, and $s_\phi$8 compression versus a $s_\phi$9 MB GeoTIFF on $g_\psi$0 SwissALTI3D tiles at $g_\psi$1 resolution (Feng et al., 2024).

3. Differential structure, topology, and GIS analysis

A defining property of implicit NTMs is that terrain derivatives are model outputs rather than post hoc finite-difference estimates. In ImplicitTerrainV2, the SIREN composition yields closed-form derivatives; for each hidden layer,

$g_\psi$2

By caching the $g_\psi$3 terms in the forward pass, the implementation can compute elevation and Jacobian $g_\psi$4 simultaneously at arbitrary continuous coordinates, with “over $g_\psi$5 million simultaneous elevation+gradient queries per second” reported for a fused custom GPU kernel via torch.compile (Feng et al., 21 May 2026).

Derivative fidelity is explicitly supervised in the shape stage of ImplicitTerrainV2 through gradient matching. The loss combines elevation and gradient terms,

$g_\psi$6

where DEM gradients come from central finite differences and model gradients from autodiff through the SIREN (Feng et al., 21 May 2026). The paper states that this “enforce[s] the smooth manifold structure of terrain DEMs for improved derivative fidelity” (Feng et al., 21 May 2026). ImplicitTerrain similarly emphasizes that autodiff-based gradients and higher derivatives are stable enough for topological analysis without an additional curvature regularizer (Feng et al., 2024).

That differentiability enables classical terrain-analysis operators to be applied directly to the neural surface. ImplicitTerrain locates critical points by minimizing $g_\psi$7, classifies minima, saddles, and maxima using the Hessian $g_\psi$8 and Sylvester’s criterion, traces separatrix lines from saddles along Hessian eigenvectors, and builds a Morse Incidence Graph (MIG) weighted by persistence $g_\psi$9 (Feng et al., 2024). Ridges, valleys, watershed boundaries, slope, aspect, normal vectors, and mean curvature are all written as functions of derivatives of $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$0; for example,

$h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$1

The reported topological alignment to a discrete Forman baseline yields critical-point $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$2 and a Wasserstein-distance ratio $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$3–$h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$4 (Feng et al., 2024).

In this terrain-analysis setting, NTMs function as more than compressed height fields. They also serve as differentiable carriers of terrain morphology, permitting continuous-query evaluation of slope, aspect, curvature, persistence, and separatrix structure at arbitrary resolution. A plausible implication is that the representation shifts a substantial fraction of GIS terrain analysis from raster operators and mesh-specific numerical fixes toward query-time differential geometry on a learned manifold.

4. Robotics, inpainting, dense mapping, and traversability

In robotics, NTM has been used for terrain completion and short-horizon local mapping rather than only for compact terrain storage. One self-supervised NTM formulation is an encoder-decoder U-Net that inpaints missing cells in a robot-centric elevation map patch of size $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$5 pixels (Stölzle et al., 2021). The input is an occluded height patch together with a binary occlusion mask, and the network outputs a fully reconstructed height patch. Training is self-supervised: starting from an already incomplete DEM, the method artificially re-occludes visible cells by 2D Bresenham ray casting from a random vantage point until the new occlusion ratio lies in $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$6, then optimizes

$h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$7

with $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$8 and $h_1(x)=\sin(\omega_0\cdot(W_0x+b_0))\odot M(x),$9 (Stölzle et al., 2021). Reported real-world results include ETH stairs with $(W_0,b_0)$0, MSE $(W_0,b_0)$1, PSNR $(W_0,b_0)$2, and Tenerife with $(W_0,b_0)$3, MSE $(W_0,b_0)$4, PSNR $(W_0,b_0)$5; inference on $(W_0,b_0)$6 maps runs at about $(W_0,b_0)$7 Hz on a single CPU core and about $(W_0,b_0)$8 Hz on an NVIDIA Titan Xp GPU (Stölzle et al., 2021).

A second robotic line is Neural Dense Elevation Mapping (N.D.E.M.), which constructs a $(W_0,b_0)$9-channel per-cell statistical tensor from multiple sparse LiDAR frames and feeds it to a fully convolutional two-task generator with a shared encoder, a $M(x)$0-block ResNet bottleneck, and two decoders, supervised by a pair of Patch-GAN discriminators (Yang et al., 2022). The height decoder outputs both a mean elevation and a log-variance map, so the model estimates aleatoric uncertainty through a heteroscedastic Gaussian likelihood (Yang et al., 2022). On an NVIDIA Jetson AGX Xavier, point-cloud preprocessing takes about $M(x)$1 ms for $M(x)$2–$M(x)$3k points, generator inference is about $M(x)$4 ms for an $M(x)$5 patch and about $M(x)$6 ms for a $M(x)$7 patch, and the full pipeline sustains at least $M(x)$8 Hz (Yang et al., 2022). Simulation results report masked MAE $M(x)$9 mm, masked MGD $\mathcal B$0 m/m, PSNR $\mathcal B$1, and SSIM $\mathcal B$2 (Yang et al., 2022).

A third formulation reconstructs local terrain as a dense point cloud in a sparse $\mathcal B$3D fully convolutional architecture. The robot-centred scene is discretized into a $\mathcal B$4 voxel grid over a $\mathcal B$5 m cube, with a time-channel index indicating current measurement or previous estimate (Hoeller et al., 2022). The network fuses current depth measurements and transformed prior predictions,

$\mathcal B$6

uses sparse 4D convolutions via Minkowski Engine, and runs end-to-end in about $\mathcal B$7 ms on a Jetson Xavier, yielding a $\mathcal B$8 Hz update rate alongside a $\mathcal B$9 Hz locomotion controller (Hoeller et al., 2022). On the ANYmal stairs scenario, the reported validation metrics are precision $z=f(x,y)$0, recall $z=f(x,y)$1, F1 $z=f(x,y)$2, and MAE $z=f(x,y)$3 cm (Hoeller et al., 2022).

A related but more semantically specialized extension is TRAIL, which uses an implicit neural representation of traversability rather than only elevation (Jia et al., 22 Nov 2025). Query points are Fourier-encoded, concatenated with bilinearly sampled BEV features, and passed through several MLP decoders that predict ground elevation range, overhang indicator, minimum ceiling height, and normalized bumpiness (Jia et al., 22 Nov 2025). Because each decoder is differentiable in spatial coordinates, gradients such as $z=f(x,y)$4 and $z=f(x,y)$5 become inputs to a gradient-based trajectory optimizer that jointly adjusts path geometry and speed profile (Jia et al., 22 Nov 2025). The reported hardware timing is $z=f(x,y)$6 ms for the encoder plus about $z=f(x,y)$7s per query point for the decoder on an RTX 4090, with $z=f(x,y)$8 optimization iterations taking $z=f(x,y)$9 ms (Jia et al., 22 Nov 2025).

5. Neural rendering, image-based reconstruction, and terrain synthesis

Another branch of NTM research recovers terrain directly from imagery rather than from a pre-existing DEM. Neural Elevation Models (NEMos) adapt Neural Radiance Fields to a $z=f(x,y)$00D continuous and differentiable terrain model by jointly training a radiance field $z=f(x,y)$01 and a height field $z=f(x,y)$02 (Dai et al., 2024). The height branch is a hash-grid-encoded MLP, and training combines volumetric rendering with a quantile regression loss

$z=f(x,y)$03

while masking out density above the predicted surface (Dai et al., 2024). Because $z=f(x,y)$04 is differentiable, NEMos supports a continuous cost functional for path planning,

$z=f(x,y)$05

optimized by backpropagating through a differentiable spline or flatness integrator (Dai et al., 2024). On the KT-22 scene, path refinement reduces distance from $z=f(x,y)$06 to $z=f(x,y)$07 and smoothness from $z=f(x,y)$08 to $z=f(x,y)$09; on Red Rocks, distance falls from $z=f(x,y)$10 to $z=f(x,y)$11 and smoothness from $z=f(x,y)$12 to $z=f(x,y)$13 (Dai et al., 2024).

For planetary and satellite-scale mapping, NTM has been defined as a neural volume-rendering framework that learns a height map $z=f(x,y)$14 and texture map $z=f(x,y)$15 directly from multi-view satellite imagery, requiring only the locus of each image pixel and no explicit depth priors, RPC fitting, or handcrafted outlier removal (Biberstein et al., 2 Aug 2025). Along a ray $z=f(x,y)$16, the method evaluates the signed-distance surrogate

$z=f(x,y)$17

and converts it into opacity with a logistic-SDF-style mapping before rendering color by quadrature (Biberstein et al., 2 Aug 2025). Implementation details include a height MLP with $z=f(x,y)$18 fully connected layers of $z=f(x,y)$19 units, a color MLP with $z=f(x,y)$20 layers of $z=f(x,y)$21 units, InstantNGP-style multi-resolution hash-grid encoding with $z=f(x,y)$22 levels and $z=f(x,y)$23 features per level, Adam with learning rate $z=f(x,y)$24, batch size $z=f(x,y)$25 rays, and $z=f(x,y)$26k iterations on an NVIDIA RTX 6000 Ada GPU, with runtime of about $z=f(x,y)$27 hours per scene (Biberstein et al., 2 Aug 2025). Reported DTM statistics include Gale Crater mean error $z=f(x,y)$28 m and standard deviation $z=f(x,y)$29 m at $z=f(x,y)$30 m GSD, Jezero Crater mean error $z=f(x,y)$31 m and standard deviation $z=f(x,y)$32 m at $z=f(x,y)$33 m GSD, and synthetic Google Earth Studio Everest mean error $z=f(x,y)$34 m and standard deviation $z=f(x,y)$35 m at $z=f(x,y)$36 m GSD (Biberstein et al., 2 Aug 2025).

The term NTM also appears in terrain synthesis rather than terrain recovery. A procedural terrain generation pipeline combines multi-layered smoothed Gaussian noise or fractional Perlin noise with feed-forward Neural Style Transfer over a frozen VGG-19 network (Merizzi, 2024). The candidate map is optimized from an initial procedural map $z=f(x,y)$37 under a weighted sum of content loss, style loss using Gram matrices, and total variation loss, with weights $z=f(x,y)$38, $z=f(x,y)$39, and $z=f(x,y)$40 over $z=f(x,y)$41 SGD iterations (Merizzi, 2024). Quantitatively, SSIM between generated and source real-world maps improves from $z=f(x,y)$42 to $z=f(x,y)$43 for mountain terrain, from $z=f(x,y)$44 to $z=f(x,y)$45 for river terrain, and from $z=f(x,y)$46 to $z=f(x,y)$47 for desert terrain, with runtime about $z=f(x,y)$48 s on a single RTX 3060 GPU (Merizzi, 2024).

6. Limitations, misconceptions, and open research directions

A persistent misconception is that NTMs automatically solve all issues associated with DEMs simply by being continuous. The literature is more specific. ImplicitTerrain notes per-tile training cost, lack of global continuity between adjacent tiles, and current GPU/CPU inference speed as limitations; the paper explicitly suggests overlap-based consistency losses, quantization, pruning, or multiresolution hash encodings as mitigations (Feng et al., 2024). The satellite NTM literature identifies shadow bias, slope-smearing on vertical cliffs or steep crater walls, camera pose uncertainty, finite-difference through the height field during opacity computation, and the need to scale to very large baselines (Biberstein et al., 2 Aug 2025). The structured-terrain locomotion model reports dynamic obstacles, edge outliers from depth sensors, and limited generalization to highly irregular natural terrains as unresolved issues (Hoeller et al., 2022).

A second misconception is that all NTMs are intrinsically $z=f(x,y)$49D height fields. Many are, but not all. The $z=f(x,y)$50D restriction is explicit in DEM compression, NEMos, and satellite neural rendering (Feng et al., 21 May 2026, Dai et al., 2024, Biberstein et al., 2 Aug 2025). By contrast, the sparse $z=f(x,y)$51D locomotion model reconstructs a local $z=f(x,y)$52D scene around the robot rather than a single-valued surface (Hoeller et al., 2022). This suggests that “terrain map” in the neural setting can refer either to a differentiable height field or to a neuralized local scene representation, depending on application constraints.

The open research directions recorded in the cited work are correspondingly diverse. ImplicitTerrain proposes meta-learned weight initializations for “few-shot terrain,” hierarchical INR designs with coarse global MLPs and local refinements, joint multi-task training of elevation plus vector layers, and real-time streaming with progressive refinement for web deployment (Feng et al., 2024). Satellite NTM work proposes physical illumination models, multi-date imagery, pose bundle adjustment during training, custom CUDA kernels, multi-band reflectance modeling, and hierarchical or tile-based NTMs for $z=f(x,y)$53 global mosaics (Biberstein et al., 2 Aug 2025). ImplicitTerrainV2 already points toward a compact neural terrain data format that is “resolution-independent,” “differentiable in closed form,” and competitive with established DEM codecs while also supporting continuous queries and derivative evaluation (Feng et al., 21 May 2026). Taken together, the surveyed literature suggests that NTMs are evolving simultaneously as compression formats, analysis substrates, perception modules, and optimization backbones.