Differentiable Object Pose Connectivity Metrics for Regrasp Sequence Optimization

Abstract: Regrasp planning is often required when one pick-and-place cannot transfer an object from an initial pose to a goal pose while maintaining grasp feasibility. The main challenge is to reason about shared-grasp connectivity across intermediate poses, where discrete search becomes brittle. We propose an implicit multi-step regrasp planning framework based on differentiable pose sequence connectivity metrics. We model grasp feasibility under an object pose using an Energy-Based Model (EBM) and leverage energy additivity to construct a continuous energy landscape that measures pose-pair connectivity, enabling gradient-based optimization of intermediate object poses. An adaptive iterative deepening strategy is introduced to determine the minimum number of intermediate steps automatically. Experiments show that the proposed cost formulation provides smooth and informative gradients, improving planning robustness over other alternatives. They also demonstrate generalization to unseen grasp poses and cross-end-effector transfer, where a model trained with suction constraints can guide parallel gripper grasp manipulation. The multi-step planning results further highlight the effectiveness of adaptive deepening and minimum-step search.

• The paper introduces a differentiable framework that leverages energy-based models to optimize regrasp sequences, reducing brittle connectivity in grasp planning.
• It formulates a smooth pose connectivity metric by marginalizing grasp energies, enabling gradient-based intermediate pose optimization and adaptive sequence length selection.
• Empirical studies confirm improved success rates across diverse object geometries and hardware, demonstrating robustness and effective cross-end-effector transfer.

Differentiable Pose Connectivity Metrics for Regrasp Sequence Optimization

Problem Formulation and Motivation

Regrasp planning in robotic manipulation is vital when direct pick-and-place actions cannot bridge initial and target object poses due to grasp infeasibility under task, kinematic, or environmental constraints. Traditional methods discretize the pose and grasp spaces, constructing regrasp graphs whose nodes encode object placements and available grasps (2604.14733). However, these approaches exhibit resolution-dependent brittleness, especially under constrained grasp sets or complex geometries.

This paper proposes an implicit, continuous reformulation by developing differentiable object pose connectivity metrics derived from Energy-Based Models (EBMs). Instead of discrete search, the method enables gradient-based optimization of intermediate object poses that maximize shared-grasp connectivity along a sequence, with adaptive selection of minimum required regrasp steps.

Energy-Based Model Formulation for Grasp Feasibility

The grasp feasibility function $E_{\phi_f}(\mathbf{T}, \boldsymbol{g})$, parametrized via an MLP, maps object pose and canonical grasp configuration to an energy value—lower energies indicate greater feasibility with respect to inverse kinematics and collision constraints. Grasp representations are pose-invariant, encoded in the object frame, and evaluated for feasibility in the workspace.

Shared grasp feasibility for pose pairs $(\mathbf{T}_a, \mathbf{T}_b)$ is analytically derived via additive energy composition: the joint probability of a grasp being feasible for both poses obeys a Boltzmann form, yielding an additive energy structure. Summing individual energies and thresholding allows efficient identification of shared feasible grasps across the pose pair.

Figure 1: Visualization of bottle object, canonical grasp, sampled grasp candidates, and workspace feasible grasps.

Figure 2: Energy composition validates additive property—summation of individual energy landscapes aligns minima with shared feasible grasps.

Differentiable Pose Connectivity Metrics

The pose connectivity score $Q_\mathrm{pair}(\mathbf{T}_a, \mathbf{T}_b)$ marginalizes composed grasp energies over the grasp set, producing a smooth scalar field quantifying shared-grasp abundance. This metric supports differentiable optimization, circumventing the combinatorial explosion and brittle connectivity of discrete graphs.

For a sequence of object poses $$(\mathbf{T}_0, \mathbf{T}_1, ..., \mathbf{T}_{N+1})$$, the differentiable sequence cost $J_\mathrm{seq}$ aggregates pairwise connectivity scores, including a regularization term that penalizes variance across link transitions. This prevents bottlenecks where individual transitions exhibit degraded connectivity, thus promoting more uniform feasibility throughout the sequence.

Adaptive Iterative Deepening Search

The optimal sequence length $N$—the minimum steps required for a feasible regrasp sequence—is adaptively determined via iterative deepening. For each $N$, batches of candidate pose sequences are initialized around analytically computed stable placements and optimized using Langevin dynamics, leveraging the smooth gradients from $J_\mathrm{seq}$. Candidate solutions are verified by thresholding shared-grasp abundance for each transition.

Figure 3: Illustration of stable placements and intermediate pose constraints via planar perturbations and rotations.

This not only automates sequence length selection but efficiently guides search within the pose space, avoiding the limitations of fixed-length discrete sampling or exhaustive enumeration.

Empirical Evaluation

Extensive simulation and real-world experiments on objects with diverse geometries (Bottle, Bunny, Pentagon, Mug) and multiple end-effectors (parallel grippers, suction cups) validate the proposed framework:

• Ablation studies demonstrate the necessity of regularization in the sequence cost. Naive summation costs produce spurious minima, while truncated costs create flat energy plateaus; only the proposed $J_\mathrm{seq}$ provides convex energy landscapes and meaningful gradients facilitating optimization convergence.

  Figure 4: Changes of grasps as intermediate pose shifts, showing effectiveness of cost formulations and importance of gradient structure.

• One-step intermediate pose generation achieves high success rates on objects with more stable placements and less geometric complexity; complex objects (e.g., Mug) pose greater challenges due to narrow connectivity passages. The success rate increases notably with $J_\mathrm{seq}$ as $(\mathbf{T}_a, \mathbf{T}_b)$0 (number of candidates) increases, underscoring the robustness provided by continuous gradient guidance.

  Figure 5: Canonical placements, optimization evolution, and pose distributions—proposed cost aligns generated intermediate poses with ground truth.

• Generalization to unseen grasps reveals non-trivial performance on test sets not used during training, partly limited by sampling coverage but indicating the learned energy landscape captures significant object-level geometry.
• Cross-End-Effector transfer shows asymmetric transferability: EBMs trained on suction constraints can guide parallel gripper manipulation with moderate success, but not vice versa. This validates the hypothesis that gripper grasp centers are a subset of suction contact distributions.

  Figure 6: Objects used in experiments, highlighting geometry diversity for benchmark evaluation.

  

  Figure 7: Six end-effectors and their candidate grasps—demonstrates geometric constraint diversity and transfer scenario.

• Multi-step sequence optimization via adaptive deepening restores solvability for unsolvable pose pairs, efficiently increasing sequence length as necessary. Success rates for Bunny and Pentagon are notably higher, correlated with greater numbers of stable placements, supporting better connectivity in sequence planning.

  Figure 8: Example of initial pose, goal pose, generated intermediate sequences, and realized multi-step planning and deployment.

Practical and Theoretical Implications

The differentiable connectivity metrics fundamentally improve regrasp planning robustness and scalability, especially under constrained environments and diverse object/end-effector geometries. Gradient-based optimization enables real-time refinement of intermediate poses, and adaptive sequence length selection prevents unnecessary complexity. The approach generalizes across unseen grasp sets and end-effector hardware, presenting a modular, extensible solution compatible with data-driven and analytical modeling.

The theory underlying additive energy composition and marginalization informs future research in manipulation planning, including online grasp generation, direct integration of IK and collision constraints, and the use of EBMs for other combinatorial planning tasks.

Conclusion

This work establishes a smooth, differentiable framework for regrasp sequence optimization via energy-based modeling of shared-grasp connectivity. The methodology outperforms discrete search baselines, demonstrates robust gradient optimization, and generalizes across object geometries and hardware configurations. Future directions include tighter integration of grasp synthesis constraints and expanding online planning capabilities. The framework advances the state-of-the-art in scalable, adaptive manipulation planning with practical implications for autonomous robotic systems.