Can I Pour into It? Robot Imagining Open Containability Affordance of Previously Unseen Objects via Physical Simulations (2008.02321v2)

Abstract: Open containers, i.e., containers without covers, are an important and ubiquitous class of objects in human life. In this letter, we propose a novel method for robots to "imagine" the open containability affordance of a previously unseen object via physical simulations. We implement our imagination method on a UR5 manipulator. The robot autonomously scans the object with an RGB-D camera. The scanned 3D model is used for open containability imagination which quantifies the open containability affordance by physically simulating dropping particles onto the object and counting how many particles are retained in it. This quantification is used for open-container vs. non-open-container binary classification (hereafter referred to as open container classification). If the object is classified as an open container, the robot further imagines pouring into the object, again using physical simulations, to obtain the pouring position and orientation for real robot autonomous pouring. We evaluate our method on open container classification and autonomous pouring of granular material on a dataset containing 130 previously unseen objects with 57 object categories. Although our proposed method uses only 11 objects for simulation calibration (training), its open container classification aligns well with human judgements. In addition, our method endows the robot with the capability to autonomously pour into the 55 containers in the dataset with a very high success rate. We also compare to a deep learning method. Results show that our method achieves the same performance as the deep learning method on open container classification and outperforms it on autonomous pouring. Moreover, our method is fully explainable.

• The paper introduces a dual-stage simulation method that evaluates open containability affordances and optimizes pouring strategies with over 96% classification accuracy.
• It combines 3D scanning with particle-based physical simulations to quantify container capacity through particle retention ratios, achieving 98.18% pouring success.
• The approach advances robotic autonomy by enabling robots to interpret and interact with novel objects, paving the way for adaptive, real-world manipulation tasks.

Imagination of Open Containability Affordances in Robotics: A Physical Simulation Approach

In contemporary robotic manipulation and interaction tasks, the necessity for robots to understand and interact with unseen objects dynamically is of paramount importance. The work by Wu and Chirikjian delineates an innovative approach leveraging physical simulations to impart robots with the ability to "imagine" open containability affordances of previously unseen objects. This capability is essential for distinguishing open containers from non-open containers and facilitating autonomous pouring tasks.

The proposed method strategically combines 3D scanning for object perception with a physical simulation-based framework to predict affordances. The method's novelty lies in its interaction-centered definition of object characteristics, rather than reliance on appearance-based classification, which can be limited by intra-class variations and inter-class generalization challenges. By simulating the interaction of particles with the object's surface, the robot quantifies the container's ability through a metric called open containability, defined as the ratio of particles retained in the object to those initially dropped onto it.

A notable aspect of Wu and Chirikjian's algorithm is its dual simulation process: open containability imagination and pouring imagination. This dual-stage simulation initially assesses the object's potential to serve as a container and subsequently predicts the most promising orientation and position for executing a pouring task. The former identifies an object's affordance to hold granular material by simulating particle retention in the object under perturbations. The latter optimizes the pouring strategy by assessing the particle-in-object retention ratio across various simulated pouring positions and orientations.

The empirical evaluation conducted on a dataset comprising 130 unseen objects across 57 categories demonstrates the robustness of the proposed method. An accuracy of 96.15% against human judgements in open container classification is noteworthy, particularly given the limited calibration data used. Furthermore, the method showcased a high success rate of 98.18% in autonomous pouring tasks on identified open containers, outperforming a deep learning-based alternative, AffordanceNet, which was also examined.

The implications of this work are twofold. Practically, it underscores a significant advancement in robotic autonomy, allowing manipulation systems to adaptively interpret and interact with new and potentially unconventional objects, which is pivotal in settings like home robotics or industrial applications where pre-known or consistently formatted objects are rare. Theoretically, it paves the way for further exploration into physical simulation methods for affordance reasoning, encouraging the integration of physics-based models with perception systems to enhance functionality understanding.

For future developments, considerations include the incorporation of more sophisticated scanning techniques to improve object model fidelity and extend affordance reasoning to more complex scenes. Additionally, integrating methods to dynamically adapt simulation parameters based on real-world feedback could enhance the robustness and reliability of this approach in varied contexts.

This paper's contribution lies in seamlessly integrating perception and interaction, thereby providing a scalable and teachable model for robots to comprehend complex affordances, with potential trajectories aligning towards greater autonomy and adaptiveness.