---
title: 'TOrtho-Gaussian: 3D Orthographic Splatting'
url: https://www.emergentmind.com/topics/tortho-gaussian
type: topic
---

# TOrtho-Gaussian: 3D Orthographic Splatting

TOrtho-Gaussian, introduced in “Tortho-Gaussian: Splatting True Digital Orthophoto Maps,” is a 3D Gaussian Splatting–based method for generating True Digital Orthophoto Maps (TDOMs) directly from aerial imagery by orthographically splatting optimized anisotropic Gaussian kernels onto a 2D image plane. It is positioned as a replacement for the conventional photogrammetric TDOM pipeline, in which geometry such as a DSM or DBM is first reconstructed and then used for explicit occlusion detection, differential rectification, mosaicking, and often radiometric balancing. In the paper’s framing, the central change is to treat orthophoto generation as differentiable orthographic rendering of a learned Gaussian field, thereby bypassing explicit DSM construction and explicit occlusion detection while targeting improved building-boundary accuracy, reduced facade artifacts, better behavior in weak-texture and reflective regions, and greater scalability for large urban scenes [2411.19594].

## 1. Problem setting and methodological position

True Digital Orthophoto Maps are described as essential products for digital twins and Geographic Information Systems. The paper distinguishes ordinary digital orthophotos from true orthophotos by emphasizing that traditional DOM generation removes terrain-induced perspective distortion but still suffers from building lean and facade projection, whereas TDOM workflows attempt to remove these effects through DSM-based visibility reasoning. The stated failure modes of classical pipelines include inaccurate DSMs, degenerated occlusion detections, ghosting, repeated mapping, wrong building boundaries, visual artifacts in weak texture regions and reflective surfaces, and persistent or jagged facade remnants. The review section also notes sensitivity to DSM ground sampling distance and the “M-portion problem” on elongated linear structures [2411.19594].

TOrtho-Gaussian is situated between classical photogrammetric orthorectification and recent neural rendering approaches. Relative to classical methods, it avoids explicit surface-model construction and explicit visibility testing. Relative to NeRF- and Plenoxel-based orthophoto generation such as Ortho-NeRF, it retains a differentiable-rendering formulation but adopts the explicit primitives and efficient rasterization associated with 3D Gaussian Splatting. The paper states that vanilla 3DGS is attractive because NeRF-style methods are too slow to train and render for operational orthophoto production, while 3DGS supports efficient training and rendering at more than \(100\) FPS. At the same time, vanilla 3DGS is characterized as perspective-oriented and limited on large-scale aerial scenes, reflective water, and thin structures. TOrtho-Gaussian is therefore presented as an adaptation of 3DGS from perspective novel-view synthesis to scale-uniform orthographic map production.

## 2. Orthographic rendering formulation

The method starts from the standard 3DGS scene model: a set of 3D Gaussian ellipsoids parameterized by mean position, anisotropic covariance, opacity, and color represented through spherical harmonics. To ensure positive semidefiniteness of the covariance, the paper uses the standard factorization
$$
\Sigma = R S S^T R^T,
$$
where \(R\) is a rotation matrix and \(S\) is a scaling matrix [2411.19594].

In vanilla 3DGS, each Gaussian is projected under perspective projection, and its image-plane covariance is approximated by
$$
\Sigma' = J W \Sigma W^T J^T,
$$
where \(W\) is the world-to-view transformation and \(J\) is the Jacobian of the perspective projection. When multiple Gaussians intersect a pixel, the rendered color is computed by front-to-back alpha compositing,
$$
C=\sum_{n=1}^{|\mathcal{N}|} c_n \alpha'_n \prod_{j=1}^{n-1}(1-\alpha'_j),
$$
with splatted opacity
$$
\alpha'_n = \alpha_n \exp\left( -\frac{1}{2}(x'-\mu'_n)^T \Sigma_n'^{-1}(x'-\mu'_n) \right).
$$

TOrtho-Gaussian keeps this Gaussian-field representation and compositing rule but replaces perspective splatting with orthographic splatting. The orthographic projection matrix is given as
$$
P_o=
\begin{pmatrix}
\frac{2}{r-l} & 0 & 0 & -\frac{r+l}{r-l}\\
0 & \frac{2}{t-b} & 0 & -\frac{t+b}{t-b}\\
0 & 0 & -\frac{2}{z_f-z_n} & -\frac{z_f+z_n}{z_f-z_n}\\
0 & 0 & 0 & 1
\end{pmatrix},
$$
where \(l,r,b,t\) are the left, right, bottom, and top bounds of the viewing box, and \(z_n,z_f\) are the near and far planes. Unlike perspective projection, no division by depth occurs, so points at different heights retain the same map scale.

The corresponding orthographic Jacobian is obtained by differentiating the orthographic projection,
$$
J_o=
\begin{pmatrix}
\frac{2}{r-l} & 0 & 0\\
0 & \frac{2}{t-b} & 0\\
0 & 0 & 0
\end{pmatrix}.
$$
This is the defining geometric distinction of the method. Under orthographic projection, the footprint of a Gaussian depends on its world-space covariance projected onto the horizontal plane rather than on camera depth. The paper argues that this affine, scale-uniform splatting is what makes the formulation natural for orthophoto generation. Visibility is then handled implicitly by depth sorting and transmittance in the compositing term \(\prod_{j<n}(1-\alpha'_j)\), which the authors describe as avoiding explicit DSM construction and explicit occlusion detection [2411.19594].

## 3. Fully anisotropic Gaussian kernel

A second core component is the Fully Anisotropic Gaussian Kernel (FAGK). The paper describes FAGK as extending standard 3DGS by incorporating spherical harmonic coefficients not only for color, but also for opacity, rotation, and scaling. This is a marked departure from vanilla 3DGS, where color is usually view-dependent but geometry and opacity are typically view-independent. The explicit spherical harmonic basis used is
$$
A_l^m(\theta,\varphi)=
\begin{cases}
\sqrt{2}K_l^m\cos(m\varphi)P_l^m(\cos\theta), & m>0\\
\sqrt{2}K_l^m\sin(-m\varphi)P_l^{-m}(\cos\theta), & m<0\\
K_l^0 P_l^0(\cos\theta), & m=0,
\end{cases}
$$
with normalization
$$
K_l^m=\sqrt{\frac{(2l+1)(l-|m|)!}{4\pi(l+|m|)!}}.
$$
The paper states that coefficients up to third order are used, as in 3DGS, and that the SH order is increased every \(1000\) iterations [2411.19594].

The exact formulas mapping SH coefficients to color, opacity, rotation, and scale are not explicitly written in the visible text. What is explicit is the intended functional role. The paper attributes improvements in reflective surfaces and slender structures to this direction-adaptive kernel design. Weak-texture regions such as water, as well as thin objects such as power lines, cranes, towers, and tree structures, are presented as cases in which fixed kernels are prone to blurring, holes, dilation, or breakage. FAGK is described as behaving like an anti-aliasing and super-sampling mechanism by allowing transparency transitions and effective support to adapt more precisely.

The ablation discussion connects these claims to orthophoto rendering resolution. Without FAGK, lower spatial-resolution TDOMs show edge dilation artifacts because projected Gaussians become too broad, making building boundaries appear thicker. With FAGK, the paper reports refined transparency transitions and better sampling precision. Quantitatively, at full SH degree \(3\), the model without FAGK obtains SSIM \(0.868\), PSNR \(23.33\), and LPIPS \(0.218\), while the full model obtains SSIM \(0.870\), PSNR \(25.10\), and LPIPS \(0.224\). The discussion in the paper treats the PSNR improvement and the qualitative gains on reflections and slender structures as the main evidence for FAGK’s value [2411.19594].

## 4. Large-scale optimization and divide-and-conquer design

Scalability is addressed through a divide-and-conquer strategy inspired by VastGaussian. The paper motivates this design by noting that vanilla 3DGS is memory-limited: on a \(24\) GB RTX \(3090\), around \(10\) million Gaussians can be optimized, while even a small Mip-NeRF360 garden scene may require \(5.8\) million Gaussians. Large urban scenes are therefore treated as infeasible without partitioning [2411.19594].

The scene is partitioned into overlapping rectangular regions based on camera positions projected onto the ground plane. The camera set \(V\) is first divided into \(m\) sections, each containing about \(|V|/m\) views, and then each section is subdivided into \(n\) cells, each with approximately \(|V|/(m\times n)\) views. This is intended to balance training across blocks. To preserve continuity, each cell is expanded by \(20\%\) beyond its original boundary. Additional cameras are then selected by visibility, defined as the ratio of the projected area of the divided cell on an image to the full image area; cameras whose visibility exceeds a threshold \(t\) are added to the local training set. The threshold value itself is not numerically specified in the visible text.

The paper further states that all 3D points seen by these extra cameras are added to the point set \(\mathbf{P}_i\) for the block. This is intended to improve Gaussian initialization, reduce depth ambiguity, and avoid floating artifacts from underconstrained geometry outside the immediate cell. Training is run independently for each block using the original 3DGS framework, after which the local Gaussian fields are merged into one unified field. The overlap and shared support are presented as the mechanism that prevents seams without requiring tiled orthophoto blending or color balancing.

An ablation on Phantom3-ieu reports the following partitioning results: no partition gives SSIM \(0.831\), PSNR \(26.25\), LPIPS \(0.173\), time \(0\)h\(41\)m; \(2\times2\) gives SSIM \(0.921\), PSNR \(31.01\), LPIPS \(0.057\), time \(0\)h\(39\)m; \(3\times3\) gives \(0.919/30.77/0.053\) in \(1\)h\(35\)m; \(4\times4\) gives \(0.902/28.90/0.061\) in \(2\)h\(02\)m; and \(5\times5\) gives \(0.907/28.95/0.059\) in \(3\)h\(19\)m. The paper concludes that moderate partitioning stabilizes training and fits memory, while excessive partitioning increases runtime and introduces more boundary effects [2411.19594].

## 5. End-to-end workflow and orthophoto construction

The input to the method is a set of multi-view aerial images with camera poses and calibration estimated from Structure from Motion. The workflow is described as beginning after structure from motion, and the compared methods are stated to use ContextCapture poses for downstream tasks. Before Gaussian optimization, the sparse point cloud is preprocessed with Manhattan alignment so that the scene’s dominant horizontal directions match the \(x\)- and \(y\)-axes and the orthophoto view can be taken along the \(z\)-axis [2411.19594].

After training, an orthographic output grid is defined at spatial resolutions \(s_x,s_y\). The pixel-center coordinates of the map are given by
$$
X = \left\{ \bar{X} + s_x\cdot \left(i-\frac{W}{2}+\delta_x\right)\mid i=0,1,\dots,W \right\},
$$
$$
Y = \left\{ \bar{Y} + s_y\cdot \left(j-\frac{H}{2}+\delta_y\right)\mid j=0,1,\dots,H \right\},
$$
where \(W,H\) are the width and height of the TDOM, and \(\delta_x,\delta_y\) are half-pixel offsets so that each projection ray passes through the pixel center. The orthophoto center is chosen from the camera centroid,
$$
\bar{X} = \frac{1}{N}\sum_{i=1}^N x_i,\qquad
\bar{Y} = \frac{1}{N}\sum_{i=1}^N y_i.
$$

Rendering then proceeds by orthographically splatting the relevant Gaussians to the output plane, sorting them by depth along the viewing direction, and alpha-blending their contributions. The paper emphasizes that this produces a seamless scene-wide TDOM directly from the learned Gaussian field, thereby avoiding the image mosaicking stage characteristic of classical workflows.

The optimization schedule reported in the paper uses \(30{,}000\) iterations in all experiments and holds out one image as a test image every fixed number of images. The text does not provide a full explicit loss equation. It states that Gaussian kernels are optimized based on the discrepancy between the rendered image and the input image, and the use of SSIM, PSNR, and LPIPS in evaluation suggests a standard photometric 3DGS-style reconstruction objective, but that inference is not written as an explicit formula in the visible text [2411.19594].

## 6. Evaluation, practical implications, and limitations

The experimental study uses the NPU-DroneMap dataset and a self-collected WHU dataset. NPU-DroneMap includes phantom3-huangqi, centralPark, factory, freeway, village, ieu, and npu, with image counts from about \(221\) to \(835\) and typical resolution \(1920\times1080\). The WHU set includes shanty, stadium, mine, and chengzi, with image sizes around \(1600\times900\) or \(1228\times820\). The chengzi sequence includes a high-precision manually labeled CAD map for geometric overlay validation. All experiments are conducted on four NVIDIA RTX \(4090\) GPUs with \(30{,}000\) iterations. Baselines are ContextCapture, Metashape, Pix4DMapper, and Map2DFusion, with additional qualitative comparison to Ortho-NeRF [2411.19594].

The paper reports two forms of geometric assessment. Relative mapping precision is evaluated through line-segment length ratios. Against Metashape, the mean relative error is \(0.15486\%\) and the mean absolute error is \(0.001622\); against Pix4DMapper, the mean relative error is \(0.12591\%\) and the mean absolute error is \(0.001772\). Absolute plausibility is supported by overlay with a CAD vector map on the Chengzi dataset, where the generated TDOM is stated to align well with manually edited map boundaries.

Qualitatively, the paper emphasizes five recurring findings: cleaner and straighter building edges; better suppression of building facades; better reconstruction of slender structures such as triangular power towers, excavators, cranes, cables, and trees; more continuous weak-texture and reflective regions such as water bodies; and scalability to full-scene TDOM generation without final mosaicking. The authors also report Canny-based edge extraction and least-squares line fitting analyses in support of boundary quality. On Phantom-ieu, Phantom-factory, and Phantom-npu, vanilla 3DGS is reported to fail due to memory exhaustion at about \(21.6\)–\(22.8\) GB VRAM, whereas TOrtho-Gaussian succeeds with about \(10.2\)–\(11.2\) GB VRAM and training times of roughly \(39\)–\(43\) minutes.

The practical implication drawn in the paper is that orthophoto production can be reformulated as orthographic rendering of a learned 3D Gaussian field rather than as differential rectification over an explicit DSM. This suggests a simplified production workflow for GIS, digital twins, and urban reconstruction, especially in scenes where DSM errors, occlusion failures, and seamline artifacts dominate conventional outputs.

The limitations are also explicit. Large-scale reconstruction is improved but not fully solved at city scale; future work is said to require more reasonable partition strategies to minimize resource waiting among blocks. Excessive partitioning degrades quality and introduces boundary artifacts. Although FAGK improves challenging regions, its formulation is only partially specified in the visible text and the gains are mixed on some perceptual metrics. The method still depends on good SfM poses and sufficient image coverage. Finally, because the representation is a learned Gaussian field rather than surveyed control geometry, map-scale geometric guarantees remain empirical unless external control is incorporated. The paper identifies even larger scenes, improved partition strategies, and integration of semantic cues and monocular depth priors as future directions [2411.19594].

Source: https://www.emergentmind.com/topics/tortho-gaussian