2Xplat: Modular 3D Gaussian Splatting

• 2Xplat is a modular feed-forward 3D Gaussian Splatting framework that decouples camera pose estimation from conditioned 3D Gaussian synthesis.
• It employs a two-stage pipeline using a geometry expert (DA3) to predict camera intrinsics/extrinsics and an appearance expert (MVP) for pixel-aligned Gaussian generation.
• Rapid convergence in under 5,000 iterations and superior performance metrics highlight its state-of-the-art novel-view synthesis and robust pose recovery.

2Xplat is a modular, pose-free feed-forward 3D Gaussian Splatting (3DGS) framework that explicitly separates camera pose estimation from 3D Gaussian generation. By leveraging two specialized pretrained experts—a geometry expert for camera parameters and an appearance expert for conditioned 3DGS synthesis—2Xplat demonstrates state-of-the-art novel-view synthesis performance and rapid convergence, effectively challenging the prevailing unified design paradigm in feed-forward 3D modeling (Jeong et al., 22 Mar 2026).

1. Motivation and Conceptual Architecture

Traditional feed-forward 3DGS approaches employ a monolithic, "all-in-one" network that simultaneously regresses camera poses and synthesizes Gaussians from unposed input images. While streamlined, this architecture entangles geometric and appearance factors within a single shared representation. It restricts the direct integration of potent pose-conditioned modules such as epipolar attention mechanisms or explicit camera-relative positional encodings.

In contrast, 2Xplat adopts a two-stage, two-expert pipeline (referred to as the “geometry expert → appearance expert” paradigm):

1. Geometry expert (pretrained Depth Anything 3, DA3) predicts multi-view camera intrinsics and extrinsics from the unposed input images.
2. Appearance expert (pretrained Multi-View Pipeline, MVP) generates pixel-aligned 3D Gaussians conditioned on these predicted camera parameters.

This architectural decoupling enables the direct reuse of large-scale pretrained weights for both pose estimation and 3DGS, without introducing randomly initialized task heads. 2Xplat converges in fewer than 5,000 fine-tuning iterations (compared to over 150,000 for unified models like YoNoSplat) and achieves both higher accuracy and efficiency in pose-free 3DGS generation.

2. Geometry Expert: Architecture and Loss Formulation

The geometry expert is instantiated as DA3, a multi-view, vision-transformer-based geometry foundation model pretrained using image sets with dense depth, point maps, ray maps, and camera pose supervision. The architecture consists of:

• Multi-view transformer encoder fusing $N$ context and target images into a unified feature space.
• Task-specific heads: Depth regression, point-map and ray-map prediction, and final camera parameter regression.

To resolve ambiguities in the global reference frame, pose supervision is imposed on relative poses between every pair $(i,j)$ of the input images: $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$

The composite camera loss consists of the following terms:

• Rotation loss:

$$\mathcal{L}_R(i,j) = \arccos\left(\frac{\mathrm{tr}(R_{i\leftarrow j}^\top\hat R_{i\leftarrow j})-1}{2}\right)$$

• Translation loss (Huber):

$$\mathcal{L}_t(i,j) = H_\delta(\hat t_{i\leftarrow j} - t_{i\leftarrow j})$$

• Intrinsic loss:

$\mathcal{L}_K(j) = \|\hat K_j - K_j\|_2^2$

• Aggregate loss:

$$\mathcal{L}_\mathrm{cam} = \frac{1}{N(N-1)}\sum_{i\neq j}\left(\lambda_R\mathcal{L}_R(i,j)+\lambda_t\mathcal{L}_t(i,j)\right) + \frac{\lambda_K}{N}\sum_{j=1}^N\mathcal{L}_K(j)$$

with typical hyperparameters: $\lambda_R=0.1$, $\lambda_t=10$, $\lambda_K=0.5$.

This design enables high-accuracy, pose estimation essential for downstream 3DGS synthesis.

3. Appearance Expert and Pose-Conditioned 3DGS Generation

The appearance expert comprises a pretrained MVP network designed for pose-conditioned Gaussian synthesis. MVP integrates several architectural elements:

• Pose conditioning using camera-relative positional encodings (PRoPE)
• Alternating cross-view and spatial token attention
• Register tokens for stable inter-view feature alignment

Given predicted pose parameters $(i,j)$0, MVP outputs a set of pixel-aligned 3D Gaussian fields for each of the $(i,j)$1 context images: $(i,j)$2 where each pixel is decoded into a Gaussian primitive $(i,j)$3.

A differentiable 3DGS renderer $(i,j)$4 composites the Gaussians to synthesize novel target images: $(i,j)$5 where $(i,j)$6 balances pixel-wise and perceptual similarity.

4. Joint Training Scheme and Computational Efficiency

2Xplat is fine-tuned end-to-end, initialized from both experts' large-scale pretrained weights. The training pipeline follows:

1. Initialize DA3 (pose) and MVP (3DGS) with their respective pretrained weights.
2. Sample $(i,j)$7 context and $(i,j)$8 target views per minibatch.
3. Predict all camera parameters: $(i,j)$9.
4. Generate Gaussians for context views: $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$0.
5. Render target views: $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$1.
6. Compute total loss:

$$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$2

1. Update parameters using AdamW (learning rate $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$3, $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$4, $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$5, weight decay $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$6) with gradient clipping.
2. Iterate for 2K–5K steps using 8 GH200 GPUs.

At inference, a single forward pass predicts both pose and 3DGS representations in 0.15–0.35 seconds, varying by backbone size. The training schedule does not introduce any randomly initialized task heads, expediting convergence and mitigating overfitting.

5. Empirical Performance and Benchmark Evaluation

2Xplat consistently outperforms previous pose-free feed-forward 3DGS frameworks, such as YoNoSplat and NoPoSplat, across standard multi-view benchmarks. In the DL3DV low-resolution setting (6 context views with pose and intrinsics), 2Xplat attains PSNR 26.63, SSIM 0.856, and LPIPS 0.121, relative to YoNoSplat at 24.72/0.817/0.139. On higher-resolution DL3DV (960×540, 16–128 views), 2Xplat matches or surpasses pose-dependent feed-forward models (MVP, iLRM) and is within 1–2 dB of the much slower 3D-GS₃₀ₖ iterative optimizer.

In intrinsic-free regimes (where ground-truth intrinsics $$\hat T_{i\leftarrow j} = \hat T_i^{-1}\, \hat T_j = \begin{bmatrix} \hat R_{i\leftarrow j} & \hat t_{i\leftarrow j} \ 0 & 1 \end{bmatrix}.$$7 are unavailable), 2Xplat† still outperforms competitors by over 1.5 dB PSNR. On the RE10K indoor set, 2Xplat† achieves PSNR 27.11, SSIM 0.877, LPIPS 0.128, establishing new state-of-the-art results for both pose-free and pose-dependent modalities. Pose estimation performance is also robust, with 2Xplat’s pose AUC@5°=0.718 and AUC@10°=0.843 after only 2K iterations, nearly matching the specialized geometry model DA3 and the π³ baseline.

Ablation studies confirm (Tables 7, 8) that backbone parameterization and relative pose-loss choice are critical: larger models yield only modest additional gains, and relative pose regression achieves the best compromise between rendering and alignment accuracy.

6. Specialization, Robustness, and Theoretical Implications

The two-expert modular design introduces several benefits:

• Faster convergence: Both geometry and appearance experts are initialized from strong pretrained weights, requiring only light fine-tuning. There is no need for randomly initialized heads, as in monolithic models.
• Specialization: The geometry expert is restricted to multi-view alignment, while the appearance expert can fully exploit pose-dependent tokens, epipolar attention, and register token stabilization.
• Tolerance to noisy predictions: End-to-end updates enable the appearance module (MVP) to remain robust to imperfect geometry, mitigating pose drift effects that can degrade rendering.

A theoretical implication is that decoupling fundamentally distinct tasks—geometry from appearance—helps avoid representational interference, supports better reuse of foundation model pretraining, and can surpass “one generalist” architectures both in convergence rate and final synthesis fidelity (Jeong et al., 22 Mar 2026).

7. Summary and Impact

2Xplat establishes a new paradigm for feed-forward 3DGS by demonstrating that explicit modularization is superior to shared representations for joint geometry and appearance estimation. It delivers state-of-the-art novel-view synthesis and efficient pose recovery within a minimal number of training iterations. The results advocate broadly for modular design principles and the reuse of specialized foundation models in complex vision systems.