Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

DNAct: Diffusion Guided Multi-Task 3D Policy Learning (2403.04115v2)

Published 7 Mar 2024 in cs.RO, cs.AI, and cs.CV

Abstract: This paper presents DNAct, a language-conditioned multi-task policy framework that integrates neural rendering pre-training and diffusion training to enforce multi-modality learning in action sequence spaces. To learn a generalizable multi-task policy with few demonstrations, the pre-training phase of DNAct leverages neural rendering to distill 2D semantic features from foundation models such as Stable Diffusion to a 3D space, which provides a comprehensive semantic understanding regarding the scene. Consequently, it allows various applications to challenging robotic tasks requiring rich 3D semantics and accurate geometry. Furthermore, we introduce a novel approach utilizing diffusion training to learn a vision and language feature that encapsulates the inherent multi-modality in the multi-task demonstrations. By reconstructing the action sequences from different tasks via the diffusion process, the model is capable of distinguishing different modalities and thus improving the robustness and the generalizability of the learned representation. DNAct significantly surpasses SOTA NeRF-based multi-task manipulation approaches with over 30% improvement in success rate. Project website: dnact.github.io.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Is conditional generative modeling all you need for decision-making? arXiv preprint arXiv:2211.15657, 2022.
  2. Model-based offline planning. arXiv preprint arXiv:2008.05556, 2020.
  3. Rt-1: Robotics transformer for real-world control at scale. arXiv, 2022.
  4. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023.
  5. Offline reinforcement learning via high-fidelity generative behavior modeling. arXiv preprint arXiv:2209.14548, 2022.
  6. Transformers as meta-learners for implicit neural representations. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVII, pages 170–187. Springer, 2022.
  7. Learning continuous image representation with local implicit image function. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8628–8638, 2021.
  8. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023.
  9. Reinforcement learning with neural radiance fields. NeurIPS, 2022.
  10. Idql: Implicit q-learning as an actor-critic method with diffusion policies. arXiv preprint arXiv:2304.10573, 2023.
  11. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.
  12. Diffusion-based generation, optimization, and planning in 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16750–16761, 2023.
  13. Perceiver: General perception with iterative attention. In ICML, 2021.
  14. Rlbench: The robot learning benchmark & learning environment. RA-L, 2020.
  15. Coarse-to-fine q-attention: Efficient learning for visual robotic manipulation via discretisation. In CVPR, 2022.
  16. Bc-z: Zero-shot task generalization with robotic imitation learning. In CoRL, 2022.
  17. Codenerf: Disentangled neural radiance fields for object categories. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12949–12958, 2021.
  18. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205.09991, 2022.
  19. Synergies between affordance and geometry: 6-dof grasp detection via implicit representations. arXiv preprint arXiv:2104.01542, 2021.
  20. Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation. arXiv, 2018.
  21. Rrt-connect: An efficient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 2, pages 995–1001. IEEE, 2000.
  22. Scene editing as teleoperation: A case study in 6dof kit assembly. In IROS, 2022a.
  23. 3d neural scene representations for visuomotor control. In Conference on Robot Learning, pages 112–123. PMLR, 2022b.
  24. Adaptdiffuser: Diffusion models as adaptive self-evolving planners. arXiv preprint arXiv:2302.01877, 2023.
  25. Vision transformer for nerf-based view synthesis from a single input image. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 806–815, 2023a.
  26. Mira: Mental imagery for robotic affordances. In Conference on Robot Learning, pages 1916–1927. PMLR, 2023b.
  27. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019.
  28. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
  29. 6-dof graspnet: Variational grasp generation for object manipulation. In ICCV, 2019.
  30. 6-dof grasping for target-driven object manipulation in clutter. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 6232–6238. IEEE, 2020.
  31. Occupancy flow: 4d reconstruction by learning particle dynamics. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5379–5389, 2019.
  32. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
  33. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019.
  34. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, 2018.
  35. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
  36. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Information Processing Systems, 35:23192–23204, 2022.
  37. Learning transferable visual models from natural language supervision. In ICML, 2021.
  38. Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration. In 2018 IEEE international conference on robotics and automation (ICRA), pages 3758–3765. IEEE, 2018.
  39. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10901–10911, 2021.
  40. Sharf: Shape-conditioned radiance fields from a single view. arXiv preprint arXiv:2102.08860, 2021.
  41. Snerl: Semantic-aware neural radiance fields for reinforcement learning. ICML, 2023.
  42. Cliport: What and where pathways for robotic manipulation. In CoRL, 2022.
  43. Perceiver-actor: A multi-task transformer for robotic manipulation. In CoRL, 2023a.
  44. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pages 785–799. PMLR, 2023b.
  45. Scene representation networks: Continuous 3d-structure-aware neural scene representations. Advances in Neural Information Processing Systems, 32, 2019.
  46. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256–2265. PMLR, 2015.
  47. Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations. IEEE Robotics and Automation Letters, 5(3):4978–4985, 2020.
  48. Alex Trevithick and Bo Yang. Grf: Learning a general radiance field for 3d representation and rendering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15182–15192, 2021.
  49. Se (3)-diffusionfields: Learning cost functions for joint grasp and motion optimization through diffusion. arXiv preprint arXiv:2209.03855, 2022.
  50. Ibrnet: Learning multi-view image-based rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4690–4699, 2021.
  51. Diffusion policies as an expressive policy class for offline reinforcement learning. arXiv preprint arXiv:2208.06193, 2022.
  52. Sun3d: A database of big spaces reconstructed using sfm and object labels. In Proceedings of the IEEE international conference on computer vision, pages 1625–1632, 2013.
  53. Universal manipulation policy network for articulated objects. RA-L, 2022.
  54. Multi-task reinforcement learning with soft modularization. Advances in Neural Information Processing Systems, 33:4767–4777, 2020.
  55. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pages 1094–1100. PMLR, 2020.
  56. Visual reinforcement learning with self-supervised 3d representations. RA-L, 2023a.
  57. Gnfactor: Multi-task real robot learning with generalizable neural feature fields. arXiv preprint arXiv:2308.16891, 2023b.
Citations (12)
View on Semantic Scholar

Summary

  • The paper introduces a novel integration of neural rendering pre-training with diffusion training, which boosts multi-task policy learning efficiency by over 30%.
  • It demonstrates that distilling 2D semantic features into a unified 3D representation enables robust generalization from limited demonstrations.
  • DNAct outperforms state-of-the-art methods with fewer parameters, excelling in both simulated environments and real-world robotic tasks.

DNAct: Enhancing Robotic Manipulation with Diffusion Guided Multi-Task 3D Policy Learning

Introduction to DNAct

In the domain of robotic manipulation, achieving a harmonious blend of semantic understanding and action decision-making continues to be a paramount challenge. Recently, a novel approach named DNAct, standing for Diffusion Guided Multi-Task 3D Policy Learning, has emerged, addressing the intricacies of learning generalized policies across diverse robotic tasks. This methodology has shown promising results, significantly surpassing state-of-the-art (SOTA) NeRF-based multi-task manipulation approaches by over 30\% in success rates. Notably, DNAct achieves this with a reduced parameter count, offering a more efficient alternative for robotic manipulation tasks.

Key Contributions

DNAct's primary contribution lies in its unique integration of neural rendering pre-training with diffusion training, facilitating the learning of a generalized multi-task policy from a limited number of demonstrations. The approach demonstrates exceptional proficiency in handling challenging robotic tasks necessitating rich 3D semantics and accurate geometry comprehension. The paper showcases significant advancements in three main areas:

  • Unified 3D Representation Learning: Through distilling 2D semantic features from foundation models into a 3D space via neural rendering, DNAct acquires a potent 3D semantic representation. This process equips the policy with an impressive out-of-distribution generalization capacity, setting it apart from existing NeRF-based methodologies.
  • Diffusion Training for Multi-Modality: By employing diffusion training, DNAct enhances its ability to discern the inherent multi-modality present within multi-task demonstrations. This approach allows DNAct to successfully capture and reconstitute action sequences from varied tasks, leading to an improved robustness and generalizability of the learned representation.
  • Efficiency and Performance: DNAct not only surpasses baseline methods in terms of success rates but does so with a significantly lower parameter count. This efficiency, combined with its demonstrated capability to excel even when pre-trained on tasks orthogonal to the training and assessment phases, underscores DNAct's potential for broad applicability in real-world robotic tasks.

Theoretical and Practical Implications

From a theoretical perspective, DNAct's innovative integration of neural rendering with diffusion training presents a significant shift in how robots can learn to interpret and interact with their environment. It opens new avenues for the exploration of foundational model distillation into 3D spaces, potentially transforming the landscape of robotic manipulation.

Practically, DNAct's ability to generalize from limited demonstrations and its success in both simulated and real-world tasks indicate a substantial step forward in the deployment of robots capable of performing complex multi-task manipulations. Robots endowed with DNAct's policy learning framework could adapt more seamlessly to the dynamic and unstructured environments typical of real-world scenarios, such as households or industrial settings.

Future Directions and Speculation

Looking ahead, DNAct offers a fertile ground for further exploration and development. One potential direction could involve investigating the integration of larger, more diverse foundation models to enhance the pre-training phase's effectiveness. Additionally, future research might focus on optimizing the diffusion training process, potentially uncovering more efficient or effective ways to capture the multi-modality of task demonstrations.

Another intriguing prospect lies in exploring DNAct's applicability beyond robotic manipulation, perhaps extending its methodology to other domains within AI that benefit from a nuanced understanding of 3D space and semantics. As robotic technologies continue to evolve, DNAct's framework might inspire innovative solutions across a broad spectrum of applications, from autonomous navigation to interactive human-robot collaboration.

Conclusion

In conclusion, DNAct marks a notable advancement in the field of robotic manipulation, showcasing a novel approach to learning generalizable multi-task policies. Its integration of neural rendering and diffusion training not only enhances semantic understanding and action decision-making but also opens new pathways for future research. As we move forward, DNAct's contributions promise to significantly influence the development and deployment of more adaptive, efficient, and capable robotic systems.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets