UniSG^GA: A 3D scenegraph powered by Geometric Algebra unifying geometry, behavior and GNNs towards generative AI (2306.10621v1)
Abstract: This work presents the introduction of UniSGGA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relationships between objects in a simplified way, within the graph representation, we propose UniSGGA, that seamlessly integrates Geometric Algebra (GA) forms. This novel approach enhances the overall performance and capability of GNNs in handling generative and predictive tasks, opening up new possibilities and aiming to lay the foundation for further exploration and development of graph-based generative AI models that can effectively incorporate behavior data for enhanced scene generation and synthesis.
- E.J. Bayro-Corrochano. 2001. Geometric Neural Computing. IEEE Transactions on Neural Networks 12, 5 (Sept. 2001), 968–986.
- Geometric deep learning: going beyond euclidean data. IEEE Signal Processing Magazine 34, 4 (2017), 18–42.
- Graph-to-3d: End-to-end generation and manipulation of 3d scenes using scene graphs. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 16352–16361.
- A Papier-Mâché Approach to Learning 3D Surface Generation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
- pygae/clifford: v1.4.0. https://doi.org/10.5281/zenodo.5116900
- Strategies for Pre-training Graph Neural Networks. In International Conference on Learning Representations.
- Recording and Replaying Psychomotor User Actions in VR. In ACM SIGGRAPH 2022 Posters. Association for Computing Machinery, New York, NY, USA, Article 30, 2 pages.
- Less Is More: Efficient Networked VR Transformation Handling Using Geometric Algebra. Advances in Applied Clifford Algebras 33, 1 (Feb. 2023), 6.
- Learning Shape-Motion Representations from Geometric Algebra Spatio-Temporal Model for Skeleton-Based Action Recognition. In 2019 IEEE International Conference on Multimedia and Expo (ICME). 1066–1071.
- Qifan Liu and Wenming Cao. 2022. Geometric algebra graph neural network for cross-domain few-shot classification. Applied Intelligence 52, 11 (2022), 12422–12435.
- Long-Term Traffic Speed Prediction Based on Geometric Algebra ConvLSTM and Graph Attention. 2022 IEEE International Conference on Smart Internet of Things 00 (2022), 108–115.
- Yigit Oktar and Mehmet Turkan. 2022. Preserving Spatio-Temporal Information in Machine Learning: A Shift-Invariant k𝑘kitalic_k-Means Perspective. Journal of Signal Processing Systems 94, 12 (2022), 1471–1483.
- Project Elements: A computational entity-component-system in a scene-graph pythonic framework, for a neural, geometric computer graphics curriculum. In Eurographics 2023 - Education Papers. The Eurographics Association.
- Alberto Pepe and Joan Lasenby. 2023. Modeling Orientational Features via Geometric Algebra for 3D Protein Coordinates Prediction. Preprint. Authorea.
- Learning rotations. Mathematical Methods in the Applied Sciences (2022).
- A multi‐channel geometric algebra residual network for traffic data prediction. IET Intelligent Transport Systems 16, 11 (2022), 1549–1560.
- UniSG: Unifying Entity-Component-Systems, 3D & Learning Scenegraphs with GNNs for Generative AI. Preprint. Authorea.
- Immersive visual scripting based on VR software design patterns for experiential training. The Visual Computer 36 (2020), 1965–1977.
- MAGES 4.0: Accelerating the world’s transition to VR training and democratizing the authoring of the medical metaverse. IEEE Computer Graphics and Applications 43, 2 (2023), 43–56.