3D-GSRD: Advanced Gaussian Scene Representations

• 3D-GSRD is a research program that extends 3D Gaussian Splatting by enabling editable, super-resolved, and sensor-tailored scene reconstructions.
• It represents scenes as anisotropic Gaussians with parameters for position, covariance, color, and opacity, allowing precise geometric and appearance control.
• Key techniques include ARAP deformation, mask-guided refinement, distortion-aware projection, and specialized hardware rasterization to address efficiency and fidelity challenges.

“3D-GSRD” (Editor's term) denotes a research grouping centered on extending 3D Gaussian Splatting (3DGS) beyond baseline novel-view synthesis into deformation and editing, super-resolution, surface reconstruction, omnidirectional rendering, synthetic aperture radar reconstruction, and hardware acceleration. The common substrate across these lines is a scene represented as a set of anisotropic Gaussians whose parameters include position, covariance or scale–rotation structure, color, and opacity; the primary research variation lies in how these primitives are constrained, projected, rasterized, or optimized for a target task (Han et al., 17 Apr 2025, Ito et al., 26 May 2025, Feng et al., 27 Nov 2025, Huang et al., 25 Jul 2025, Li et al., 25 Jun 2025, Li et al., 20 Mar 2025).

1. Common representation and problem structure

A recurrent formalism in this literature models a scene as a Gaussian set. In ARAP-GS, the scene is represented as

$$G=\{g_1, g_2, \dots, g_N\}, \quad g_i=\{\mu, \Sigma, c, \alpha\},$$

where $\mu$ is the Gaussian center, $\Sigma$ is the covariance matrix, $c$ is the spherical-harmonic color, and $\alpha$ is opacity; the covariance is parameterized as

$\Sigma = RSS^{T}R^{T}.$

IE-SRGS uses the same primitive decomposition in terms of position $\bm{\mu}$, covariance $\bm{\Sigma}$, color $\bm{c}$, and opacity $\alpha$. ErpGS writes each Gaussian as $\mu$0, emphasizing per-axis scale and quaternion rotation. SAR-GS retains the mean–covariance structure but augments the primitive with scattering-related quantities and spherical harmonic coefficients, because the rendering target is SAR backscatter rather than optical radiance (Han et al., 17 Apr 2025, Feng et al., 27 Nov 2025, Ito et al., 26 May 2025, Li et al., 25 Jun 2025).

The shared motivation is equally consistent. Standard 3DGS is attractive because it provides high rendering quality and efficient rendering, but the surveyed works identify several systematic gaps: drag-driven editing is difficult because 3DGS is an explicit but highly redundant representation of many anisotropic Gaussians; geometry remains weakly constrained for accurate surface recovery; omnidirectional equirectangular images induce large distortions that generate extremely large 3D Gaussians; low-resolution supervision yields insufficient high-frequency detail; SAR reconstruction requires physically grounded scattering models rather than optical compositing alone; and edge deployment is bottlenecked by Gaussian rasterization, which accounts for over $\mu$1 of total runtime on the tested edge SoC (Han et al., 17 Apr 2025, Huang et al., 25 Jul 2025, Ito et al., 26 May 2025, Feng et al., 27 Nov 2025, Li et al., 25 Jun 2025, Li et al., 20 Mar 2025).

Taken together, these works suggest a task-specialized reinterpretation of 3DGS: the Gaussian primitive is preserved, but the optimization target shifts from generic photometric fitting toward geometric editability, resolution enhancement, sensor-specific fidelity, or systems efficiency.

2. Drag-driven deformation and appearance refinement

ARAP-GS introduces a two-stage framework for drag-driven 3DGS editing and is explicitly described as “the first to apply ARAP deformation directly to 3D Gaussians.” Its central move is to treat the center of each 3D Gaussian as the “vertex” in an As-Rigid-As-Possible deformation system, transferring the classical ARAP principle from meshes to Gaussian centers. The standard ARAP energy is written as

$\mu$2

with alternating updates of positions and local rotations via SVD. For positions, the update step is

$\mu$3

To make the method tractable on large scenes, ARAP is not solved over all Gaussians. A representative subset $\mu$4 is randomly sampled, a KNN graph is built on that subset, ARAP is solved there, and the remaining Gaussians are deformed by interpolation. For a non-sampled Gaussian indexed by $\mu$5,

$\mu$6

with weights

$\mu$7

The paper emphasizes that direct interpolation without ARAP is inferior because it can break geometry, cause unexpected deformation elsewhere, and struggle with rotations (Han et al., 17 Apr 2025).

The second stage addresses appearance degradation after geometric deformation. ARAP-GS states that after ARAP, “properties such as opacity and color remain unoptimized,” so it refines rendered views with the off-the-shelf StableSR diffusion-based image super-resolution model. To avoid cross-view inconsistency, it uses “Iterative Dataset Update” borrowed from Instruct-NeRF2NeRF and “Mask-Guided 3DGS Fine-tuning.” During fine-tuning, one viewpoint is updated every $\mu$8 iterations; the edited-region mask is derived from thresholded Gaussian displacement and projection to the camera plane, and supervision is merged as

$\mu$9

The user supplies a pre-trained 3DGS scene, handle points $\Sigma$0, and target points $\Sigma$1; the output is a final edited 3DGS after ARAP deformation, StableSR refinement, iterative dataset update, and mask-guided fine-tuning (Han et al., 17 Apr 2025).

Quantitatively, the paper evaluates on $\Sigma$2 scenes spanning real scenes and single objects and reports the best results on all metrics. The Dragging Accuracy Index is

$\Sigma$3

with lower being better. ARAP-GS achieves a DAI of $\Sigma$4, compared with $\Sigma$5 for Instruct-NeRF2NeRF, $\Sigma$6 for GaussianEditor, $\Sigma$7 for DragDiffusion+3DGS, and $\Sigma$8 for SDEDrag+3DGS. The user study over $\Sigma$9 participants reports $c$0 preference for ARAP-GS, versus $c$1, $c$2, $c$3, and $c$4 for the baselines, while GPT-4o evaluation assigns $c$5 to ARAP-GS against $c$6, $c$7, $c$8, and $c$9. Runtime is reported as $\alpha$0 to $\alpha$1 minutes on a single RTX 3090 GPU; for a scene of around $\alpha$2 Gaussians, the ARAP stage takes about $\alpha$3 minutes and $\alpha$4 fine-tuning iterations take about $\alpha$5 minutes. The representative subset is fixed at $\alpha$6, with KNN size $\alpha$7, interpolation using $\alpha$8 neighbors, and a maximum of $\alpha$9 ARAP iterations (Han et al., 17 Apr 2025).

3. Surface-oriented Gaussian reconstruction

Gaussian Set Surface Reconstruction (GSSR) reframes 3DGS as a surface-oriented Gaussian set rather than a rendering-oriented cloud. Its stated motivation is that standard 3DGS optimizes Gaussian splats mainly for novel-view rendering quality, so learned Gaussians can drift away from the latent surface, become unevenly distributed, form thick or redundant layers, and remain hard to edit. GSSR proposes to distribute Gaussians evenly along the latent surface while aligning their dominant normals with the surface normal, combining pixel-level and Gaussian-level single-view normal consistency with multi-view photometric consistency. The total loss is reported as

$\Sigma = RSS^{T}R^{T}.$0

with

$\Sigma = RSS^{T}R^{T}.$1

A scale loss

$\Sigma = RSS^{T}R^{T}.$2

compresses the smallest axis so Gaussians become thin surface-like splats (Huang et al., 25 Jul 2025).

The method’s distinctive contribution is direct per-Gaussian supervision. Pixel-level normal consistency enforces agreement between normals from rendered depth and normals rendered from Gaussians, while Gaussian-level normal consistency directly supervises each Gaussian: $\Sigma = RSS^{T}R^{T}.$3 Pixel-level and Gaussian-level multi-view photometric consistency further constrain placement through NCC-based homographic comparisons, and opacity regularization

$\Sigma = RSS^{T}R^{T}.$4

pushes Gaussians toward transparent-and-removable or opaque-and-meaningful states. Periodic depth- and normal-guided Gaussian reinitialization is performed at $\Sigma = RSS^{T}R^{T}.$5 and $\Sigma = RSS^{T}R^{T}.$6 iterations, while total training runs for $\Sigma = RSS^{T}R^{T}.$7 iterations (Huang et al., 25 Jul 2025).

The reported effect is improved Gaussian placement rather than only improved rendered geometry. On DTU, the paper reports a Chamfer distance mean of about $\Sigma = RSS^{T}R^{T}.$8, Gaussian centroid accuracy mean of about $\Sigma = RSS^{T}R^{T}.$9, and Gaussian completeness mean of about $\bm{\mu}$0, with centroid accuracy better than PGSR at $\bm{\mu}$1 and completeness much better than PGSR’s $\bm{\mu}$2. On Tanks and Temples, the mean values are about $\bm{\mu}$3 F1, $\bm{\mu}$4 precision, and $\bm{\mu}$5 completeness. On Mip-NeRF360, GSSR remains competitive in novel-view synthesis with average PSNR around $\bm{\mu}$6, SSIM around $\bm{\mu}$7, and LPIPS around $\bm{\mu}$8. The paper explicitly connects this cleaner, surface-aligned Gaussian set to downstream scene editing, object manipulation, animation or deformation, and generation of new Gaussian-based 3D environments (Huang et al., 25 Jul 2025).

4. Super-resolution and high-fidelity Gaussian recovery

IE-SRGS addresses 3DGS super-resolution from low-resolution multi-view input. The paper studies reconstruction of a high-resolution 3DGS model from low-resolution inputs and argues that prior methods overly depend on pre-trained 2D super-resolution models. The identified failure modes are “3D Gaussian ambiguity,” cross-view inconsistency, and domain gap: external 2D priors can recover sharp local details, but because they process views independently they provide conflicting signals for a shared 3D Gaussian, and they are trained on generic 2D datasets rather than novel-view 3D reconstruction (Feng et al., 27 Nov 2025).

The proposed framework is an internal-external knowledge fusion pipeline with three stages. External knowledge is generated by SwinIR for HR images and Depth Anything V2 for depth maps. Internal knowledge is generated by a multi-scale 3DGS model built on Mip-Splatting with MV-Regulation, producing cross-view consistent, domain-adaptive internal HR image and depth references via SR-Splatting. Fusion is mask-guided. A pixel-wise discrepancy map

$\bm{\mu}$9

is thresholded to form a binary mask $\bm{\Sigma}$0, so internally generated supervision is preferred in unreliable regions and external supervision is trusted elsewhere. Geometry is fused more globally through a weighted combination of internal and external depth losses: $\bm{\Sigma}$1 The internal model is trained for $\bm{\Sigma}$2 iterations with the same hyperparameters as Mip-Splatting; MV-Regulation trains on $\bm{\Sigma}$3 randomly sampled views; supervision weights are $\bm{\Sigma}$4 and $\bm{\Sigma}$5; the mask threshold is $\bm{\Sigma}$6 for real-world scenes and $\bm{\Sigma}$7 for synthetic scenes (Feng et al., 27 Nov 2025).

Evaluation covers $\bm{\Sigma}$8 scenes from Mip-NeRF 360, Deep Blending, Tanks and Temples, and NeRF Synthetic. On NeRF Synthetic, IE-SRGS reports $\bm{\Sigma}$9 PSNR, $\bm{c}$0 SSIM, and $\bm{c}$1 LPIPS, compared with $\bm{c}$2 for SRGS and $\bm{c}$3 for 3DGS. On real-world datasets, reported averages are $\bm{c}$4 on Mip-NeRF 360, $\bm{c}$5 on Deep Blending, and $\bm{c}$6 on Tanks and Temples. The ablation on Mip-NeRF 360 shows a monotonic progression from $\bm{c}$7 for Mip-Splatting baseline to $\bm{c}$8 after adding MV-Regulation, internal and external texture and geometry guidance, and finally mask-guided texture integration. Training overhead is moderate—$\bm{c}$9m$\alpha$0s versus $\alpha$1m$\alpha$2s on NeRF Synthetic and $\alpha$3m$\alpha$4s versus $\alpha$5m$\alpha$6s on Mip-NeRF360 relative to SRGS—while inference is faster at $\alpha$7 FPS versus $\alpha$8 FPS on NeRF Synthetic and $\alpha$9 FPS versus $\mu$00 FPS on Mip-NeRF360 (Feng et al., 27 Nov 2025).

A plausible implication is that 3DGS super-resolution is being reformulated from a pure external-prior problem into a consistency-constrained fusion problem: 2D priors supply detail, but the internal 3D model arbitrates view agreement.

5. Sensor- and projection-specific generalizations

ErpGS extends 3DGS to omnidirectional novel-view synthesis from equirectangular projection images. The paper identifies two ERP-specific difficulties: projection distortion and obstacle inconsistency. Because pixels near the poles represent smaller solid angles than pixels near the equator, standard 3DGS can generate extremely large 3D Gaussians, degrading rendering accuracy and leading to floaters or blurred geometry. ErpGS responds with three mechanisms: geometric regularization enforcing agreement between normals rendered from Gaussians and normals computed from rendered depth, scale regularization

$\mu$01

and a flattening regularizer

$\mu$02

It further introduces distortion-aware weighting based on spherical area and a viewpoint-dependent mask to suppress obstacles such as the robot body, camera stand, photographer, and other non-scene objects. ERP-specific rendering uses spherical coordinates and an affine-linearized covariance projection

$\mu$03

Training uses $\mu$04 iterations, Adam, PyTorch based on 3DGS, and an RTX 4090 24GB; $\mu$05 is used from the beginning, and after $\mu$06 iterations $\mu$07 and $\mu$08 are added, with $\mu$09, $\mu$10, and $\mu$11. On OmniBlender, Ricoh360, and OmniScenes, ErpGS is reported as best on all datasets and scenes in the table, with example results of $\mu$12 PSNR, $\mu$13 SSIM, and $\mu$14 LPIPS on OmniBlender/barbershop; $\mu$15 on Ricoh360/bricks; and $\mu$16 on OmniScenes/room. The ablation on OmniBlender shows $\mu$17 PSNR for the full model versus $\mu$18 without $\mu$19, $\mu$20 without $\mu$21, and $\mu$22 without $\mu$23. A documented failure case is the “lone-monk” scene, where the distant tower is hard to reconstruct, which the authors attribute to too few initial 3D points in that region (Ito et al., 26 May 2025).

SAR-GS adapts 3DGS to synthetic aperture radar target reconstruction through the SAR Differentiable Gaussian Splatting Rasterizer (SDGR). Here the Gaussian is not only a geometric primitive but also a scatterer, and forward rendering is based on the Mapping and Projection Algorithm rather than perspective alpha compositing alone. The projected covariance is written as

$\mu$24

and SAR image formation uses per-Gaussian backscattering intensity followed by Gaussian image blending: $\mu$25 The framework employs custom CUDA gradient flow instead of generic autograd, explicitly deriving gradients with respect to intensity, covariance, extinction coefficients, and related parameters. Experiments cover simplified architectural targets, synthetic vehicle targets, and real MSTAR data. On synthetic reconstruction, the reported metrics are CD $\mu$26, Precision $\mu$27, Recall $\mu$28, and F1 $\mu$29 for T72; $\mu$30 for BTR70; and $\mu$31 for KRAZ. In image-domain rendering, SAR-GS improves upon SAR-NeRF in SSIM, PSNR, and especially LPIPS for T72 and KRAZ, and improves SSIM and LPIPS for BTR70. On real MSTAR measurements, reported results are CD $\mu$32, Precision $\mu$33, Recall $\mu$34, and F1 $\mu$35 for T72; $\mu$36 for BTR70; and $\mu$37 for 2S1. The paper notes that background clutter can produce floating Gaussians and also reports an inconsistency in one experimental description, giving the depression angle as around $\mu$38 in one passage and $\mu$39 in another (Li et al., 25 Jun 2025).

These two directions illustrate the same general pattern under different measurement models: 3DGS is retained as the scene parameterization, but the projection, regularization, and even the meaning of “appearance” are made domain-specific.

6. Rendering systems and hardware acceleration

GauRast addresses a different bottleneck: the cost of rasterizing Gaussians on resource-constrained edge GPUs. The paper reports that on real-world NeRF-360 scenes, the baseline 3DGS pipeline achieves only $\mu$40–$\mu$41 FPS on the NVIDIA Jetson Orin NX under a $\mu$42 W power limit, and that the Gaussian rasterization stage accounts for over $\mu$43 of total runtime across all measured scenes. Rather than proposing a standalone accelerator, GauRast argues that 3DGS rasterization and triangle rasterization are structurally similar: both take primitive parameters, perform a projection-like transformation, evaluate primitive-to-pixel coverage, compute per-pixel weights, and accumulate outputs. Triangle rasterization uses inside/outside tests and minimum-depth selection; Gaussian rasterization uses Gaussian density evaluation and transparency-aware color accumulation. The Gaussian contribution and final compositing are written as

$\mu$44

with

$\mu$45

This mapping lets GauRast extend the existing triangle rasterizer rather than replacing it (Li et al., 20 Mar 2025).

Architecturally, GauRast modifies only the Graphics Processing Cluster rasterizer path and leaves the SMs untouched. The design uses Tile Buffers A and B as ping-pong buffers, a PE block for rasterization, and CUDA-collaborative scheduling in which preprocessing and sorting remain on CUDA cores while the enhanced rasterizer handles the dominant rasterization stage. Compared with a standard triangle rasterizer, each PE requires only $\mu$46 adders, $\mu$47 multiplier, and $\mu$48 exponentiation unit for Gaussian support while reusing $\mu$49 adders and $\mu$50 multipliers. Synthesized results report a $\mu$51 average runtime reduction for the rasterization operator and $\mu$52 average energy efficiency improvement for the original 3DGS pipeline, yielding a $\mu$53 reduction in total runtime and $\mu$54 FPS average end-to-end. For the latest efficient 3DGS variant, the paper reports $\mu$55 runtime reduction in Gaussian rasterization, $\mu$56 energy efficiency improvement, $\mu$57 end-to-end speedup, and $\mu$58 FPS average. The enhanced graphics portion incurs $\mu$59 overhead relative to the original graphics units, but only $\mu$60 of the entire SoC area. In FP16 form, the design matches GSCore’s performance while requiring $\mu$61 mm$\mu$62 versus $\mu$63 mm$\mu$64, which the paper describes as a $\mu$65 area-efficiency advantage. Typical power for the $\mu$66-PE prototype is $\mu$67 W in $\mu$68 nm CMOS at $\mu$69 GHz and $\mu$70 V (Li et al., 20 Mar 2025).

In the context of 3D-GSRD, GauRast indicates that Gaussian research is no longer only algorithmic. The representation is being co-designed with existing graphics hardware, especially where rasterization dominates the runtime budget.

7. Limitations and prospective directions

The surveyed works converge on a common limitation: 3DGS remains expressive but not unconstrained. ARAP-GS preserves the topological structure of the scene with minimal disruption, so it cannot create edits that require tearing or topology change; the paper’s failure case is asking a man to open his mouth, where the Gaussians in the mouth region do not separate to create a hole. The same paper notes that it does not yet directly modify Gaussian properties such as scaling and color during ARAP deformation and suggests stronger diffusion models such as ControlNet, along with handle-based and skeleton-based deformation, as future work (Han et al., 17 Apr 2025).

Other limitations are task-specific but structurally similar. ErpGS still depends on sparse initial geometry and view coverage, and its obstacle suppression is explicitly discussed for OmniScenes, suggesting dataset dependence in masking practice (Ito et al., 26 May 2025). IE-SRGS reduces but does not remove the tradeoff between detail and consistency; its own framing is that external priors are strong for detail but weak for 3D consistency, while internal priors are strong for consistency but weak for high-frequency detail (Feng et al., 27 Nov 2025). GSSR improves surface alignment, yet its reported rendering quality on Mip-NeRF360 is somewhat below the strongest rendering baseline in that table for some metrics, indicating the usual reconstruction-versus-rendering balance (Huang et al., 25 Jul 2025). GauRast accelerates the rasterization bottleneck rather than the entire 3DGS pipeline, and its gains depend on workloads where rasterization dominates (Li et al., 20 Mar 2025). SAR-GS requires high-quality multi-view training data, is sensitive to SAR noise and clutter, exhibits weak inter-point structural coupling, and shows that densification can help on synthetic data but hurt real SAR reconstruction by over-splitting background clutter (Li et al., 25 Jun 2025).

Taken together, these papers suggest that 3D-GSRD is evolving along three coupled axes. First, the Gaussian set is becoming more explicitly geometric, as seen in ARAP-driven deformation and per-Gaussian surface alignment. Second, the supervision model is becoming more heterogeneous, combining diffusion priors, internal multi-view consistency, distortion-aware losses, or physics-based scattering models according to task. Third, deployment concerns are moving closer to the core method, with specialized rasterization hardware turning Gaussian rendering itself into a systems design object. Under this reading, 3D-GSRD is less a single algorithm than a technical program for making Gaussian scene representations editable, reconstructible, super-resolvable, sensor-aware, and computationally practical.