RoboArmGS: Hybrid Robotic Arm Modeling

• RoboArmGS is a hybrid digital asset and motion modeling framework that refines conventional URDF kinematics using Bézier curve residuals for smoother joint trajectories.
• It integrates Gaussian splatting with learnable motion refiners to enhance rendering quality and improve Real2Sim2Real transfer for robotic applications.
• End-to-end differentiable training with photometric, structural, and temporal losses yields state-of-the-art performance on both static and dynamic benchmarks.

RoboArmGS is a hybrid digital asset and motion modeling framework for robotic arms that refines conventional URDF-rigged kinematic representations with Bézier curve-based residuals, yielding substantially higher fidelity in real-world dynamic rendering and facilitating robust Real2Sim2Real transfer. It addresses key limitations of prior 3D Gaussian Splatting methods that rigidly bind Gaussians to mesh and URDF links, offering a learnable motion refiner for accurate kinematics and asset construction. This section presents an integrated technical overview, spanning physical modeling, neural rendering, algorithmic details, dataset context, and quantitative performance.

1. Real2Sim2Real Motivation and Rationale

The Real2Sim2Real (R2S2R) paradigm underlies robotic asset creation and policy learning by linking real-world robot motion capture with simulator-based modeling, which then informs Sim2Real deployments. Conventional methods rely on URDF-driven forward kinematics for mesh and Gaussian binding, resulting in degraded visual and physical fidelity due to non-ideal joint motion, transmission backlash, and link flex (Wang et al., 22 Nov 2025). RoboArmGS explicitly models the residual mismatch between nominal URDF kinematics and actual joint trajectories, providing photorealistic digital twins for downstream manipulation and expressive motion tasks.

A plausible implication is that as robot arms become more flexible and encounter unmodeled or dynamic disturbances, rigid kinematic binding will increasingly underperform for policy transfer, monitoring, and rendering, motivating the Bézier refinement approach.

2. Hybrid Representation: Gaussian Splatting and Bézier Refinement

RoboArmGS constructs digital assets as structured clouds of anisotropic 3D Gaussians, each bound to arm links defined by mesh geometry and URDF semantics. The innovation is a hybrid motion model:

• URDF-driven kinematics: Gaussians follow FK transforms derived from URDF joint angles for a geometric prior.
• Bézier Curve Residuals: For each joint $j$, a continuous-time residual $\Delta q^{(j)}(t)$ is parameterized by a degree-$n$ Bézier curve:

$$\Delta q^{(j)}(t) = \sum_{i=0}^{n} c_{i}^{(j)} B_{i,n}(t),$$

where $c_{i}^{(j)}$ are learned control points and $B_{i,n}(t)$ are Bernstein polynomials (Wang et al., 22 Nov 2025).

The refined trajectory is

$q^{(j)}(t) = q_{URDF}^{(j)}(t) + \Delta q^{(j)}(t)$

and each Gaussian's world coordinates are updated via the refined FK per time step.

This enables the 3DGS model to encode realistic joint deviations, greatly reducing rendering artifacts such as ghosting or blurring seen in earlier FK-only methods.

3. Learnable Binding, Training Losses, and End-to-End Differentiability

All parameters—including Gaussian locations, scales, Bézier control points, and static offsets—are trained end-to-end in a differentiable splatting pipeline. Binding of Gaussians to mesh topology is maintained coherently across dynamic sequences:

• Photometric Loss: Mean-square error between rendered and ground-truth color.
• Structural Regularization: SSIM, LPIPS, position and scale regularizers.
• Temporal Smoothness: Bézier velocity loss

$$L_{vel} = \int_{0}^{1} ||v(t)||_{2}^{2} dt \approx \sum_{k} ||(c_{k+1}-c_{k-1})/(2\Delta t)||_{2}^{2}$$

• Total Training Objective:

$$L_{total} = L_{photo} + \lambda_{SSIM} L_{SSIM} + \lambda_{pos} L_{pos} + \lambda_{scale} L_{scale} + \lambda_{vel} L_{vel}$$

Gradients propagate efficiently from pixel errors through Bézier curve basis functions back to each control point via the chain rule, allowing direct refinement of motion residuals for each joint (Wang et al., 22 Nov 2025).

4. RoboArm4D Dataset: Benchmarking Dynamic Asset Creation

The RoboArm4D dataset provides the first public benchmark for evaluating dynamic robotic arm asset creation methods:

Arm Model	Degrees of Freedom	Sequence Length (frames)	Sensors / Modalities
Franka Research 3	7	900–1800	RGB, joint angles, masks
UR5e	6	900–1800	RGB, joint angles, masks
ABB IRB 120	6	900–1800	RGB, joint angles, masks

Annotations include camera intrinsics/extrinsics, accurate joint logs, and hand-refined foreground masks. Splits are 8:1:1 for train/val/test, enabling both novel-view and novel-pose synthesis benchmarking (Wang et al., 22 Nov 2025).

5. Quantitative Evaluation and Ablation Studies

RoboArmGS demonstrates state-of-the-art performance in both static and dynamic digital asset evaluation:

Metric	RoboArmGS (static)	2DGS Baseline	RoboArmGS (dynamic)	3DGS+FK	4DGS
PSNR (dB)	28.45	26.44	25.74	19.89	20.58
SSIM	0.954	0.935	0.942	–	–
LPIPS	0.051	0.118	0.054	–	–

Ablations confirm the necessity of Structured Gaussian Binding and Bézier Motion Refinement—removing either causes severe drops in PSNR, especially on dynamic scene synthesis tasks. Both global Bézier corrections and per-joint static offsets are indispensable for rendering and motion fidelity (Wang et al., 22 Nov 2025).

6. Discussion: Limitations, Extensions, and Applications

RoboArmGS assumes that each joint’s residual motion is amenable to representation with a single, smooth Bézier curve; highly non-stationary events (e.g., collisions) may necessitate adaptive or piecewise modeling. Per-joint independence can overlook kinematic couplings such as belt drives. The framework is extensible to higher-degree or adaptive-knot splines, multi-arm and mobile-base platforms, and may benefit from integration with sensor or physics-based feedback.

Applications include photorealistic digital twins for Sim2Real policy transfer, remote monitoring, anomaly detection by analyzing Bézier residuals, and real-time virtual/augmented reality visualization controlling robot motion trajectories (Wang et al., 22 Nov 2025).

7. Integration with Broader Motion Generation and Neuro-Symbolic Control

RoboArmGS complements broader developments in robotic arm motion modeling and control, including neuro-symbolic architectures for real-time grasping and social interactions (Hanson et al., 2020), and expressive motion generation with self-collision avoidance (Li et al., 13 Mar 2025). Its asset creation pipeline is well-aligned with recent mesh-physics-Gaussian hybrid representation frameworks, further augmenting physical plausibility and transfer fidelity in manipulation policies (Lou et al., 27 Aug 2024).

The system’s contribution is distinguished by its learnable, continuous-time motion refinement within the differentiable rendering paradigm. This suggests growing relevance for high-fidelity neural asset modeling as robotic platforms advance in dexterity, compliance, and perceptual complexity.