Pour me a drink: Robotic Precision Pouring Carbonated Beverages into Transparent Containers (2309.08892v2)
Abstract: With the growing emphasis on the development and integration of service robots within household environments, we will need to endow robots with the ability to reliably pour a variety of liquids. However, liquid handling and pouring is a challenging task due to the complex dynamics and varying properties of different liquids, the exacting precision required to prevent spills and ensure accurate pouring, and the necessity for robots to adapt seamlessly to a multitude of containers in real-world scenarios. In response to these challenges, we propose a novel autonomous robotics pipeline that empowers robots to execute precision pouring tasks, encompassing both carbonated and non-carbonated liquids, as well as opaque and transparent liquids, into a variety of transparent containers. Our proposed approach maximizes the potential of RGB input alone, achieving zero-shot capability by harnessing existing pre-trained vision segmentation models. This eliminates the need for additional data collection, manual image annotations, or extensive training. Furthermore, our work integrates ChatGPT, facilitating seamless interaction between individuals without prior expertise in robotics and our pouring pipeline, this integration enables users to effortlessly request and execute pouring actions. Our experiments demonstrate the pipeline's capability to successfully pour a diverse range of carbonated and non-carbonated beverages into containers of varying sizes, relying solely on visual input.
- “Visual closed-loop control for pouring liquids” In 2017 IEEE International Conference on Robotics and Automation (ICRA) Singapore, Singapore: IEEE, 2017, pp. 2629–2636 DOI: 10.1109/ICRA.2017.7989307
- “Autonomous Precision Pouring From Unknown Containers” In IEEE Robotics and Automation Letters 4.3, 2019, pp. 2317–2324 DOI: 10.1109/LRA.2019.2902075
- “Look and Listen: A Multi-Sensory Pouring Network and Dataset for Granular Media from Human Demonstrations” In 2022 International Conference on Robotics and Automation (ICRA) Philadelphia, PA, USA: IEEE, 2022, pp. 2519–2524 DOI: 10.1109/ICRA46639.2022.9812125
- “Self-supervised Transparent Liquid Segmentation for Robotic Pouring” arXiv, 2022 arXiv:2203.01538 [cs]
- “Precision Pouring into Unknown Containers by Service Robots” In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China: IEEE, 2019, pp. 5875–5882 DOI: 10.1109/IROS40897.2019.8967911
- “Accurate Pouring with an Autonomous Robot Using an RGB-D Camera” arXiv, 2018 arXiv:1810.03303 [cs]
- “Robust Robotic Pouring using Audition and Haptics” In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Las Vegas, NV, USA: IEEE, 2020, pp. 10880–10887 DOI: 10.1109/IROS45743.2020.9340859
- Carolyn Matl, Robert Matthew and Ruzena Bajcsy “Haptic Perception of Liquids Enclosed in Containers” In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China: IEEE, 2019, pp. 7142–7149 DOI: 10.1109/IROS40897.2019.8968528
- Zijie Li and Amir Barati Farimani “Graph neural network-accelerated Lagrangian fluid simulation” In Computers & Graphics 103, 2022, pp. 201–211 DOI: 10.1016/j.cag.2022.02.004
- “PourNet: Robust Robotic Pouring Through Curriculum and Curiosity-based Reinforcement Learning” In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Kyoto, Japan: IEEE, 2022, pp. 9332–9339 DOI: 10.1109/IROS47612.2022.9981195
- “FluidLab: A Differentiable Environment for Benchmarking Complex Fluid Manipulation” arXiv, 2023 arXiv:2303.02346 [cs]
- “You Only Look Once: Unified, Real-Time Object Detection” arXiv, 2016 arXiv:1506.02640 [cs]
- Glenn Jocher, Ayush Chaurasia and Jing Qiu “YOLO by Ultralytics”, 2023
- Jonathan Long, Evan Shelhamer and Trevor Darrell “Fully Convolutional Networks for Semantic Segmentation” arXiv, 2015 arXiv:1411.4038 [cs]
- Sagi Eppel, Haoping Xu and Alan Aspuru-Guzik “Computer vision for liquid samples in hospitals and medical labs using hierarchical image segmentation and relations prediction” arXiv, 2021 arXiv:2105.01456 [cs]
- Olaf Ronneberger, Philipp Fischer and Thomas Brox “U-Net: Convolutional Networks for Biomedical Image Segmentation” arXiv, 2015 arXiv:1505.04597 [cs]
- “Rethinking Atrous Convolution for Semantic Image Segmentation” arXiv, 2017 arXiv:1706.05587 [cs]
- “Segmenting Transparent Objects in the Wild” arXiv, 2020 arXiv:2003.13948 [cs]
- “Segmenting Transparent Object in the Wild with Transformer” arXiv, 2021 arXiv:2101.08461 [cs]
- “RGB-D Local Implicit Function for Depth Completion of Transparent Objects” arXiv, 2021 arXiv:2104.00622 [cs]
- “ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation” arXiv, 2019 arXiv: http://arxiv.org/abs/1910.02550
- “KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects” arXiv, 2020 arXiv:1912.02805 [cs]
- “RoboChop: Autonomous Framework for Fruit and Vegetable Chopping Leveraging Foundational Models” arXiv, 2023 arXiv:2307.13159 [cs]
- “Towards Learning to Perceive and Reason About Liquids” arXiv, 2016 arXiv:1608.00887 [cs]
- Pedro Piacenza, Daewon Lee and Volkan Isler “Pouring by Feel: An Analysis of Tactile and Proprioceptive Sensing for Accurate Pouring” In 2022 International Conference on Robotics and Automation (ICRA) Philadelphia, PA, USA: IEEE, 2022, pp. 10248–10254 DOI: 10.1109/ICRA46639.2022.9811898
- Jeeangh Reyes-Montiel, Antonio Marin-Hernandez and Sergio Hernandez-Mendez “A Geometric Approach for Partial Liquids’ Pouring from a Regular Container by a Robotic Manipulator:” In Proceedings of the 19th International Conference on Informatics in Control, Automation and Robotics Lisbon, Portugal: SCITEPRESS - ScienceTechnology Publications, 2022, pp. 688–694 DOI: 10.5220/0011321600003271
- “Attention Is All You Need” arXiv, 2023 arXiv:1706.03762 [cs]
- “AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles” arXiv, 2017 arXiv:1705.05065 [cs]
- “ChatGPT for Robotics: Design Principles and Model Abilities” arXiv, 2023 arXiv:2306.17582 [cs]
- “Microsoft COCO: Common Objects in Context” arXiv, 2015 arXiv:1405.0312 [cs]
- “Computer Vision for Recognition of Materials and Vessels in Chemistry Lab Settings and the Vector-LabPics Data Set” In ACS Central Science 6.10, 2020, pp. 1743–1752 DOI: 10.1021/acscentsci.0c00460
- “A Modular Robotic Arm Control Stack for Research: Franka-Interface and FrankaPy” arXiv, 2020 arXiv:2011.02398 [cs]