InnerGS: Internal Scenes Rendering

• InnerGS is a volumetric rendering method that reconstructs object interiors as continuous density fields from sparse, depth-aligned 2D slices, such as MRI or CT.
• It employs a factorized anisotropic 3D Gaussian model to convert slice-based supervision into smooth interior reconstruction without requiring multi-view camera poses.
• A CUDA implementation using tile-based rasterization, conditional splatting, and parallel depth sorting enables rapid, high-quality rendering across various medical imaging modalities.

Searching arXiv for the specified paper to ground the article in the cited source. InnerGS is a volumetric rendering method for reconstructing an object’s interior as a continuous density field from a sparse set of depth-aligned 2D slices, such as MRI or CT cross-sections, without relying on multi-view camera poses. It is introduced in “InnerGS: Internal Scenes Rendering via Factorized 3D Gaussian Splatting” (Liang et al., 18 Aug 2025). The method adapts 3D Gaussian Splatting (3DGS) to internal-scene reconstruction by replacing the conventional emphasis on external surfaces with a density-based formulation in which anisotropic 3D Gaussians model the full interior volume. The resulting representation is described as plug-and-play, pose-free, and compatible with any data modalities, with a CUDA implementation reported for efficient rendering and optimization (Liang et al., 18 Aug 2025).

1. Problem setting and motivation

InnerGS addresses the problem of reconstructing internal scenes from sparse sliced data, with the stated goal of recovering smooth and detailed internal structures from depth-aligned 2D slices (Liang et al., 18 Aug 2025). The motivating examples include medical imaging settings such as MRI and CT, where observations are naturally organized as slices rather than as images with camera intrinsics and extrinsics.

The work is positioned against limitations of standard 3DGS. Two specific failure modes are identified. First, conventional 3DGS exhibits a surface bias: existing pipelines optimize Gaussians to match silhouettes and appearances from external viewpoints, so interior voxels remain empty or nearly zero density. Second, classical 3DGS is projection-dependent: it projects each 3D Gaussian through known camera extrinsics and intrinsics into 2D, whereas slices carry no camera parameters, making the standard 2D elliptical splatting formulation inapplicable (Liang et al., 18 Aug 2025).

Within that framing, InnerGS proposes a volumetric representation that directly models 3D density through the inner 3D Gaussian distribution. A central design choice is to treat each training slice as a “view” at fixed depth whose pixels must be explained by the underlying 3D density. This suggests a reinterpretation of slice-based supervision: instead of recovering only visible surfaces, optimization is driven to populate the entire interior volume continuously between observed cross-sections (Liang et al., 18 Aug 2025).

2. Representation and mathematical formulation

The basic primitive in InnerGS is the anisotropic 3D Gaussian. Each Gaussian $i$ is parameterized by a center $\boldsymbol{\mu}_i \in \mathbb{R}^3$, a symmetric positive-definite covariance matrix $\Sigma_i \in \mathbb{R}^{3 \times 3}$, a weight $w_i$, and an RGB color $\mathbf{c}_i$ (Liang et al., 18 Aug 2025). Its density at a 3D point $\mathbf{x}$ is given as

$$G_i(\mathbf{x}) = \frac{1}{(2\pi)^{3/2}\,\lvert\Sigma_i\rvert^{1/2}} \exp\Bigl(-\tfrac12(\mathbf{x}-\boldsymbol{\mu}_i)^T\Sigma_i^{-1}(\mathbf{x}-\boldsymbol{\mu}_i)\Bigr).$$

The covariance is factorized as

$\Sigma_i = R_i\,D_i\,R_i^T,$

where $R_i$ is a $3 \times 3$ rotation matrix and $\boldsymbol{\mu}_i \in \mathbb{R}^3$0 contains the axis-aligned scale parameters (Liang et al., 18 Aug 2025). The paper states two consequences of this factorization. One is parameter-count reduction: instead of six free entries in a symmetric $\boldsymbol{\mu}_i \in \mathbb{R}^3$1 matrix, the parameterization uses three scales plus three rotation parameters. The other is splatting acceleration: because the eigen-decomposition is known in closed form, projection to a slice or image plane reduces to rotating and scaling in a fixed order, thereby avoiding a full matrix inversion at runtime (Liang et al., 18 Aug 2025).

The total volume density $\boldsymbol{\mu}_i \in \mathbb{R}^3$2 and RGB color $\boldsymbol{\mu}_i \in \mathbb{R}^3$3 are defined by summing over all Gaussians:

$\boldsymbol{\mu}_i \in \mathbb{R}^3$4

For a camera ray $\boldsymbol{\mu}_i \in \mathbb{R}^3$5, the accumulated color is written as

$\boldsymbol{\mu}_i \in \mathbb{R}^3$6

In this formulation, the scene is not represented as a surface plus appearance field, but as a continuous volumetric density and color field assembled from Gaussian basis functions. The paper’s emphasis is that this supports smooth interior reconstruction from sparse slices (Liang et al., 18 Aug 2025).

3. Factorized internal-scene rendering

The principal methodological contribution of InnerGS is a factorized volumetric representation in which each Gaussian is decomposed into a conditional 2D slice footprint and a 1D depth marginal (Liang et al., 18 Aug 2025). This factorization is presented as the mechanism by which InnerGS can render arbitrary slices efficiently while maintaining a continuous interior density model.

The paper describes two candidate-sampling schemes for fast rasterization on arbitrary slices. The first is 3D ellipsoid projection, which projects each Gaussian’s $\boldsymbol{\mu}_i \in \mathbb{R}^3$7 bounding cube onto every slice. The second, designated as the preferred method, is slice-adaptive conditional splatting. In the latter, the Gaussian density is factorized as $\boldsymbol{\mu}_i \in \mathbb{R}^3$8, and the 2D bounding box is adapted to the particular slice depth $\boldsymbol{\mu}_i \in \mathbb{R}^3$9 (Liang et al., 18 Aug 2025). The stated motivation is to reduce wasted computation by tailoring each Gaussian’s 2D support to each slice.

This factorization is also central to the rendering equation used during optimization. For a slice at depth $\Sigma_i \in \mathbb{R}^{3 \times 3}$0, the predicted image is formed by front-to-back compositing:

$\Sigma_i \in \mathbb{R}^{3 \times 3}$1

The work states that $\Sigma_i \in \mathbb{R}^{3 \times 3}$2 is computed by differential rasterization, and training enforces per-slice photometric consistency against the observed slices (Liang et al., 18 Aug 2025). Because the Gaussians have continuous support, optimization across sparse slices naturally interpolates density and color between them. The paper explicitly attributes smooth filling of gaps between slices to this continuous Gaussian support.

The method is described as requiring no additional hand-tuned smoothness term. Instead, smoothness is treated as an implicit prior induced by the Gaussian basis itself. A plausible implication is that the regularity of the representation is expected to emerge from the basis geometry and adaptive refinement rather than from explicit variational penalties.

4. Optimization pipeline and CUDA implementation

The reported pipeline begins with data ingestion: a collection of 2D slices $\Sigma_i \in \mathbb{R}^{3 \times 3}$3 with known slice positions $\Sigma_i \in \mathbb{R}^{3 \times 3}$4 along each axis $\Sigma_i \in \mathbb{R}^{3 \times 3}$5 (Liang et al., 18 Aug 2025). Gaussian initialization then places Gaussians on a coarse 3D grid, assigns uniform weights and colors, and uses a small isotropic covariance. Optimization proceeds iteratively over slices.

During each iteration, candidate Gaussians are selected for each slice using either 3D ellipsoid projection or conditional splatting. The selected candidates are rasterized front-to-back to obtain $\Sigma_i \in \mathbb{R}^{3 \times 3}$6, after which the primary loss is computed:

$\Sigma_i \in \mathbb{R}^{3 \times 3}$7

Gradients are back-propagated to update $\Sigma_i \in \mathbb{R}^{3 \times 3}$8 (Liang et al., 18 Aug 2025). The optimization process also includes pruning, splitting, and cloning operations. Gaussians with very low opacity $\Sigma_i \in \mathbb{R}^{3 \times 3}$9 are pruned, and Gaussians whose gradients exceed thresholds are split or cloned. The paper characterizes this adaptive refinement as a structural regularizer that concentrates detail where needed and suppresses noise, while also noting that the rules are threshold-based (Liang et al., 18 Aug 2025).

The CUDA implementation is described as plug-and-play and built on top of 3DGS. Several implementation elements are specified:

Component	Description
Tile-based rasterization	Each slice is split into tiles, with overlapping Gaussians tracked via shared-memory bounding boxes
Conditional splatting	Uses the 1D marginal $w_i$0 and a conditional 2D ellipse $w_i$1
Parallel depth sorting	Gaussians are sorted by $w_i$2 using GPU parallel radix sort
Memory management	Uses a Structure-of-Arrays layout, with batched pruning and splitting

The stated purpose of tile-based rasterization is efficient spatial culling on the GPU. Conditional splatting yields tight per-slice bounding rectangles. Parallel depth sorting enables front-to-back compositing with early ray termination. Batched memory operations are used to minimize kernel launches (Liang et al., 18 Aug 2025). Taken together, these design choices explain the paper’s claim of real-time or near real-time rendering.

5. Experimental evaluation

InnerGS is evaluated on several datasets and modalities: Brain MRI (T1-weighted, BrainWeb phantom, downsampled), Cardiac MRI (HVSMR-2.0 static whole-heart volumes), Wrist motion MRI (38 frames, dynamic, 12 slice views), and fMRI (BOLD time series, 33 axial slices, 160 timepoints) (Liang et al., 18 Aug 2025). The paper presents these experiments as demonstrations of both volumetric fidelity and modality breadth.

For Brain MRI with 5% hold-out slices, the reported quantitative result is PSNR $w_i$3 dB, with axial values $w_i$4–$w_i$5, and SSIM $w_i$6. For Cardiac MRI, the reported average is PSNR $w_i$7 dB and SSIM $w_i$8. For Wrist motion MRI, compared to a naïve “previous frame” baseline, the paper reports average PSNR $w_i$9 dB versus $\mathbf{c}_i$0 dB and SSIM $\mathbf{c}_i$1 versus $\mathbf{c}_i$2. For fMRI, the description is qualitative at the summary level: consistent PSNR/SSIM improvements over a static baseline across 20 test frames, especially in central high-activity slices (Liang et al., 18 Aug 2025).

An ablation compares the two candidate-sampling schemes in a synthetic $\mathbf{c}_i$3 domain. Method 2, “Conditional Splatting,” is reported to reduce average 2D bounding-box area from $\mathbf{c}_i$4 to $\mathbf{c}_i$5, candidates per pixel from $\mathbf{c}_i$6 to $\mathbf{c}_i$7, render time from $\mathbf{c}_i$8 s to $\mathbf{c}_i$9 s, and false positives per pixel from $\mathbf{x}$0 to $\mathbf{x}$1. The paper also reports that