JODA: Composable Joint Dynamics for Articulated Objects

Abstract: Articulated objects used in simulation and embodied AI are typically specified by geometry and kinematic structure, but lack the fine-grained dynamical effects that govern realistic mechanical behavior, such as frictional holding, detents, soft closing, and snap latching. Existing approaches either ignore the detailed structure of dynamics entirely, or use simple models with limited expressiveness. We introduce JODA, a framework for generating joint-level dynamics as a structured three-channel field over the joint degree of freedom, capturing conservative forces, dry friction, and damping. Instantiated using shape-constrained piecewise cubic interpolation (PCHIP), this formulation defines a compact and expressive function space that is both interpretable and compatible with differentiable simulation. Building on this representation, we develop methods for inferring and refining joint dynamics from multimodal inputs. Given visual observations and joint context, a vision-LLM proposes structured dynamical primitives, which are composed into a unified dynamics field. The resulting representation supports both direct manipulation and gradient-based refinement. We demonstrate that JODA enables plausible and controllable modeling of diverse joint behaviors, providing a unified interface for inference, editing, and optimization. Code and example assets with their generated profiles will be released upon publication.

• The paper presents a unified composable framework that explicitly models and synthesizes joint-level dynamics to bridge the sim-to-real gap in articulated object simulation.
• It leverages vision-language models to propose and iteratively refine parameterized dynamics fields for realistic friction, damping, and conservative forces.
• Empirical results show lower RMSE compared to baseline spring models, validating the approach on assets like doors, cabinets, and buttons.

Composable Joint Dynamics for Articulated Asset Simulation: An Analysis of JODA

Introduction and Motivation

JODA ("Composable Joint Dynamics for Articulated Objects") (2605.09954) addresses a central limitation in current simulation-ready articulated object assets: the absence of expressive, physically plausible joint-level dynamics. While modern pipelines accurately reconstruct object geometry and kinematics from multimodal inputs, the nuanced mechanical behaviors—such as frictional holding, latching, soft-close, and complex resistive forces—that shape real-world interaction are typically absent or modeled with crude heuristics. The resulting sim-to-real gap restricts embodied AI, robotics, and digital twin applications where precise interaction behavior is critical.

JODA introduces a unified, composable framework that explicitly represents, synthesizes, and refines such joint-level dynamics. It leverages vision-LLMs (VLMs) to ground semantic priors about physical behavior into structured, parameterized dynamical fields, providing a practical and interpretable interface for simulation, editing, and gradient-based optimization.

Figure 1: Overview of the JODA pipeline, with a VLM proposing and refining structured joint effects into a composable dynamics profile for simulation and editing.

Structured Joint Dynamics Representation

Central to JODA is its three-channel function space defined along the joint DOF:

• Conservative force, $F_\text{cons}(s)$: Captures position-dependent forces such as spring returns, attraction wells, and bistable barriers.
• Dry friction, $F_\text{fric,max}(s)$: Models the maximum resistive force opposing motion, encoding effects like detents and holding friction.
• Damping, $C_\text{damp}(s)$: Encodes the velocity-dependent dissipation.

Each channel decomposes into localized effect components, parameterized by active joint intervals and shape-constrained piecewise cubic interpolants (PCHIP). This structure allows for compositionality, locality, interpretability, and direct editability—addressing engineering requirements for both rapid authoring and downstream differentiable simulation.

A curated library of effect templates abstracts recurring families of mechanical structure (e.g., snap detents, magnetic closures, bistable mechanisms), promoting semantic transfer and reliable numerical instantiation without requiring explicit mechanical reconstruction.

VLM-Guided Inference and Iterative Refinement

JODA's inference pipeline employs VLMs—prompted with rendered images of the asset across motion ranges and joint context metadata—to generate structured joint effect proposals. These proposals specify effect identities, intervals, and qualitative strengths for each channel, remaining agnostic to exact numerical values. The proposals are then compiled into physically grounded, numerically parameterized dynamics fields using joint context to anchor the units and scales.

This separation of semantic reasoning from numeric instantiation reduces the burden on the VLM and supports iterative closed-loop refinement. Diagnostic profile visualizations provide feedback to the VLM, enabling multi-round improvement of effect selection and parameterization. The representation enables flexible refinement: direct parameter editing, language-guided revision (via further VLM interaction), and gradient-based optimization via differentiable simulation rollouts.

Figure 2: Lighter-button use case—simulated and real visuals, JODA-generated force/friction profiles, and corresponding force/trajectory plots.

Empirical Analysis and Numerical Results

JODA is evaluated across diverse articulated categories—from doors and cabinets to buttons and switches—in both simulated and real data regimes.

Qualitative Evaluation

Case studies demonstrate JODA's ability to generate complex, nonlinear interactive behaviors that align with real-world object dynamics:

• Lighter Button: JODA's composite field produces characteristic "push-through" and "pop-back" transients during both depression and release, matching tactile experiences and empirical trajectories, in contrast to baseline spring-only models that fail to capture these dynamics (see Figure 2).
• Refrigerator and Microwave Doors: Accurate modeling of endpoint attraction, magnetic latching, and rebound effects enables simulated trajectories that faithfully reproduce real-world testing, including partial closures and the need for targeted pushes to achieve latching.

Figure 4: Microwave door results—baseline results in unrealistic automatic closure, JODA yields a rebound and requires an explicit latching force, aligning with observed real interactions.

Quantitative Comparison

Force profiles derived from JODA's generated fields are compared with real-world quasi-static measurements (cf. [Jain et al., 2010]). On both refrigerator and cabinet doors, JODA achieves lower RMSE than linear-spring baselines (0.121 vs 0.240 for fridge, 0.180 vs 0.386 for cabinet, normalized), validating its expressive capability for capturing highly nonlinear force behaviors inherent to everyday articulated mechanisms.

Ablation and Optimization

Template ablation reveals the necessity of engineering-guided effect vocabularies—direct VLM-to-numeric mapping without templates frequently results in unrealistic, irreproducible, and physically unfaithful profiles. Iterative VLM refinement is particularly beneficial in gravity-affected scenes, facilitating convergence toward intended dynamic behaviors. Differentiable optimization allows JODA to align simulation rollouts with empirical trajectory supervision, further improving sim-to-real fidelity.

Practical and Theoretical Implications

JODA's composable framework transforms joint-level dynamics from an implicit, manually tuned artifact into a first-class, interpretable, and revisable asset property. Practically, this underpins:

• Improved simulation-to-reality transfer: Richer dynamics bridge the behavioral gap for embodied agents and training policies.
• Scalable asset authoring: Automation of dynamics generation via VLM-guided proposals reduces the cost and expertise required for large interactive object sets.
• Engineering auditability and refinement: Structured effect fields support both direct and programmatic editing, facilitating design iteration and simulation integrity.
• Differentiable dynamics: Gradient access supports precise optimization under physical, task, or perceptual objectives, further closing the reality gap.

Theoretically, JODA demonstrates an effective route for integrating high-level semantic priors (from VLMs) with low-level physical instantiation via compositional representations, providing a template for future work on multimodal asset synthesis and physically-aware AI systems.

Figure 3: Visual reference of fridge used in force profile quantitative comparison.

Future Directions

JODA currently assumes dominant single-DOF internal dynamics and restricts its compositional vocabulary to three channels. Future work should address:

• High-dimensional and coupled joint effects.
• History-dependent and nonlinear friction/damping (e.g., hysteresis).
• More sophisticated, perceptually aligned effect templates.
• Automatic diagnostics and human-alignment metrics for predicted object feel.
• Integration of direct trajectory or tactile feedback for in-the-wild asset refinement.

Conclusion

JODA (2605.09954) advances the field of articulated asset simulation by introducing a compact, composable, and interpretable representation of joint-level dynamics, powered by multimodal VLM-guided effect synthesis and refinement. Its interface and empirical results substantiate the necessity—and practical tractability—of structured dynamic modeling for bridging the sim-to-real gap, with direct implications for scalable simulation, embodied AI training, and interactive robotic systems. The approach lays foundation for future advances in data-driven, physically grounded asset authoring and simulation-based intelligence.