GEAR: GEometry-motion Alternating Refinement for Articulated Object Modeling with Gaussian Splatting

Abstract: High-fidelity interactive digital assets are essential for embodied intelligence and robotic interaction, yet articulated objects remain challenging to reconstruct due to their complex structures and coupled geometry-motion relationships. Existing methods suffer from instability in geometry-motion joint optimization, while their generalization remains limited on complex multi-joint or out-of-distribution objects. To address these challenges, we propose GEAR, an EM-style alternating optimization framework that jointly models geometry and motion as interdependent components within a Gaussian Splatting representation. GEAR treats part segmentation as a latent variable and joint motion parameters as explicit variables, alternately refining them for improved convergence and geometric-motion consistency. To enhance part segmentation quality without sacrificing generalization, we leverage a vanilla 2D segmentation model to provide multi-view part priors, and employ a weakly supervised constraint to regularize the latent variable. Experiments on multiple benchmarks and our newly constructed dataset GEAR-Multi demonstrate that GEAR achieves state-of-the-art results in geometric reconstruction and motion parameters estimation, particularly on complex articulated objects with multiple movable parts.

• The paper introduces an EM-style alternating optimization framework that decouples geometry and motion to enhance articulated object modeling.
• It leverages Gaussian Splatting on multi-view RGB-D data and integrates weak supervision from SAM to achieve robust part segmentation.
• Experimental results demonstrate faster convergence and superior accuracy on multi-joint datasets, benefiting robotics and AR/VR applications.

GEAR: EM-Style Alternating Optimization for Articulated Object Modeling with Gaussian Splatting

Motivation and Problem Statement

Modeling articulated objects is central to embodied intelligence, robotics, and AR/VR, where high-fidelity, interactive digital assets are required for robust simulation and manipulation. Articulated objects introduce complexity through their multi-part structures and coupled geometry-motion relationships, with mutual dependencies in part segmentation and motion parameter estimation. Existing methods typically suffer from instability or poor generalization on complex multi-joint or out-of-distribution objects due to tightly coupled optimization or reliance on category-specific priors.

GEAR introduces an EM-style alternating optimization framework for articulated object modeling, leveraging Gaussian Splatting as the underlying representation. The method advances stability by decoupling geometry and motion into latent part segmentation and explicit joint motion parameters, alternately refined. Weakly supervised multi-view segmentation priors are provided via vanilla SAM, enhancing robustness while maintaining broad generalization.

Figure 1: Articulated object modeling involves coupled optimization of geometry and motion; GEAR employs alternating refinement to decouple and stabilize this process.

Methodological Framework

Gaussian Splatting Representation

GEAR employs 2D Gaussian Splatting (2DGS) as the base representation, reconstructing articulated objects from multi-view RGB-D data in both canonical and target states. Each Gaussian is assigned a learnable mask vector for part segmentation and undergoes rigid transformation via SE(3) matrices corresponding to joints.

EM-Style Alternating Optimization

The optimization strategy draws a strict analogy to the Expectation-Maximization algorithm. Segmentation masks $\mathcal{M}$ are treated as latent variables, and motion parameters $\mathcal{T}$ as explicit variables, with alternating E-step and M-step:

• Initialization: A voxel-based coarse module identifies dynamic regions and provides preliminary segmentation and motion parameters.
• E-Step (Geometry Modeling): Fixes current motion parameters, optimizes part segmentation using rendering loss and weak supervision from SAM mask aggregation (see Figure 2). Multi-view segment consistency and KNN-based clustering regularize spatial coherence.
• M-Step (Motion Modeling): Updates rigid motion parameters using hard assignment of Gaussians to parts, enforcing strict physical constraints with dual quaternion modeling for rotations/translations.

Alternating between these steps ensures both geometric and motion consistency, mitigating ambiguity in error attribution and preventing non-physical local minima plaguing traditional joint optimization.

Figure 3: GEAR’s EM-style pipeline alternates between geometry and motion modeling, beginning from a robust initialization and driving convergence via multi-phase refinement.

Figure 2: SAM Mask Aggregation module: fine-grained image regions from vanilla SAM are aligned and aggregated into coherent part-level masks, serving as weak supervision for geometric modeling.

Experimental Analysis

Quantitative Results

GEAR was evaluated on the PARIS, ArtGS-Multi, and GEAR-Multi datasets, comprising both synthetic and real-world objects of varying articulation complexity. Across all benchmarks, GEAR demonstrates strong numerical results in geometric reconstruction (Chamfer Distance CD-w, CD-m, CD-s) and motion estimation (Axis Angle, Axis Position, Geo Dist):

• On PARIS, GEAR achieves top or second-best results in most metrics; lowest errors on simulated objects for motion estimation and high geometric fidelity.
• On challenging multi-joint datasets (ArtGS-Multi, GEAR-Multi), GEAR consistently outperforms baselines, with robust handling of thin structures, small movable parts, and diverse topological arrangements.
• The method maintains practical runtime and memory efficiency, converging faster and with lower final errors than joint optimization or staged baselines.

  Figure 4: Qualitative results on complex multi-joint objects; GEAR attains clean geometric separation and accurate articulation modeling compared to ArtGS.

  

  Figure 5: Qualitative comparison on ArtGS-Multi; GEAR disentangles spatially-close components and estimates motion axes with higher fidelity than baselines.

Ablation and Convergence

Ablation studies validate that alternating optimization is crucial for preventing local minima and error propagation. Removal of regularization or loss components leads to segmentation failures and geometric distortions, particularly in highly articulated, multi-part objects. The EM interval is robust, with degeneration only under excessively large intervals.

Figure 6: Initialization and convergence visualizations demonstrate that coarse spatial priors effectively guide the alternating optimization to photorealistic, accurate reconstructions.

Practical Applications and Limitations

GEAR assets can be exported into URDF files and deployed in robotic simulation environments (e.g., Isaac Sim), bridging the Sim2Real gap for embodied AI applications. The EM alternate methodology improves plugin compatibility with prior frameworks (e.g., ArtGS), refining poor local minima and boosting multi-joint performance.

Figure 7: GEAR-generated assets support direct robotic manipulation in Isaac Sim, facilitating embodied AI research.

Despite robustness, GEAR exhibits sensitivity in the coarse initialization under spatial ambiguity (adjacent movable parts with minimal gaps) and fails under extreme articulations (e.g., 180-degree rotations misclassified as prismatic joints) owing to non-convex optimization landscapes. Transparent or highly reflective materials introduce rendering artifacts, propagating noise into segmentation and motion recovery.

Figure 8: Spatial ambiguity during initialization can cause erroneous grouping of closely positioned movable parts.

Figure 9: Extreme articulation leads to motion misclassification, with the model incorrectly identifying revolute actions as prismatic.

Implications and Future Directions

GEAR’s alternating geometry-motion refinement yields improved convergence, fidelity, and generalizability for articulated object modeling. Theoretically, decoupling coupled objectives with alternating optimization substantially lowers error attribution ambiguity, offering a general paradigm for complex multi-object inference. Practically, its robustness across diverse object categories supports applications in digital-twin simulation, robotics, and virtual environments.

Future work should investigate integration with generative priors (e.g., diffusion models, neural deformation fields) for improved reconstruction under sparse observations and non-rigid dynamics. Explicit temporal modeling or trajectory constraints could address failures in extreme articulation scenarios. Enhancing representation robustness for transparent/reflection materials remains an open challenge.

Conclusion

GEAR establishes a rigorous EM-style alternating optimization framework for articulated object reconstruction with Gaussian Splatting. It achieves superior stability and accuracy in both geometry and motion estimation, particularly on multi-joint objects, and demonstrates strong generalization without category-specific fine-tuning. The framework is extensible, robust, and practical for embodied AI; its limitations identify promising avenues for future theoretical and practical advancements.