Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Vision-based Discovery of Nonlinear Dynamics for 3D Moving Target (2404.17865v1)

Published 27 Apr 2024 in cs.CV, cs.AI, and nlin.CD

Abstract: Data-driven discovery of governing equations has kindled significant interests in many science and engineering areas. Existing studies primarily focus on uncovering equations that govern nonlinear dynamics based on direct measurement of the system states (e.g., trajectories). Limited efforts have been placed on distilling governing laws of dynamics directly from videos for moving targets in a 3D space. To this end, we propose a vision-based approach to automatically uncover governing equations of nonlinear dynamics for 3D moving targets via raw videos recorded by a set of cameras. The approach is composed of three key blocks: (1) a target tracking module that extracts plane pixel motions of the moving target in each video, (2) a Rodrigues' rotation formula-based coordinate transformation learning module that reconstructs the 3D coordinates with respect to a predefined reference point, and (3) a spline-enhanced library-based sparse regressor that uncovers the underlying governing law of dynamics. This framework is capable of effectively handling the challenges associated with measurement data, e.g., noise in the video, imprecise tracking of the target that causes data missing, etc. The efficacy of our method has been demonstrated through multiple sets of synthetic videos considering different nonlinear dynamics.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Automated reverse engineering of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 104(24):9943–9948, 2007.
  2. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15):3932–3937, 2016.
  3. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy of Sciences, 116(45):22445–22451, 2019.
  4. Grounding physical concepts of objects and events through dynamic visual reasoning. In International Conference on Learning Representations, 2021.
  5. Physics-informed learning of governing equations from scarce data. Nature Communications, 12(1):6136, 2021.
  6. Automated discovery of fundamental variables hidden in experimental data. Nature Computational Science, 2(7):433–442, 2022.
  7. Deep learning in video multi-object tracking: A survey. Neurocomputing, 381:61–88, 2020.
  8. Data-driven discovery of intrinsic dynamics. Nature Machine Intelligence, 4(12):1113–1120, 2022.
  9. Deepstereo: Learning to predict new views from the world’s imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5515–5524, 2016.
  10. Gipuma: Massively parallel multi-view stereo reconstruction. Publikationen der Deutschen Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation e. V, 25(361-369):2, 2016.
  11. William Gilpin. Chaos as an interpretable benchmark for forecasting and data-driven modelling. arXiv preprint arXiv:2110.05266, 2021.
  12. Deepmvs: Learning multi-view stereopsis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2821–2830, 2018.
  13. Physics-as-inverse-graphics: Unsupervised physical parameter estimation from video. In International Conference on Learning Representations, 2020.
  14. John R Koza. Genetic programming as a means for programming computers by natural selection. Statistics and computing, 4:87–112, 1994.
  15. A robot hand-eye calibration method of line laser sensor based on 3d reconstruction. Robotics and Computer-Integrated Manufacturing, 71:102136, 2021.
  16. Incorporating actor-critic in monte carlo tree search for symbolic regression. Neural Computing and Applications, pages 1–17, 2021.
  17. Distilling governing laws and source input for dynamical systems from videos. Proceedings of the 31st International Joint Conference on Artificial Intelligence, pages 3873–3879, 2022.
  18. Accurate monocular 3d object detection via color-embedded 3d reconstruction for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6851–6860, 2019.
  19. Deep learning for visual tracking: A comprehensive survey. IEEE Transactions on Intelligent Transportation Systems, 23(5):3943–3968, 2021.
  20. Context-aware correlation filter tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1396–1404, 2017.
  21. Symbolic regression via neural-guided genetic programming population seeding. arXiv preprint arXiv:2111.00053, 2021.
  22. Stereovision-based fuzzy obstacle avoidance method. International Journal of Humanoid Robotics, 8(01):169–183, 2011.
  23. Ida-3d: Instance-depth-aware 3d object detection from stereo vision for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13015–13024, 2020.
  24. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. In International Conference on Learning Representations, 2021.
  25. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
  26. Discovering nonlinear pdes from scarce data with physics-encoded learning. In The Tenth International Conference on Learning Representations, 2022.
  27. Encoding physics to learn reaction–diffusion processes. Nature Machine Intelligence, 5:765–779, 2023.
  28. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 779–788, 2016.
  29. Data-driven discovery of partial differential equations. Science Advances, 3(4):e1602614, 2017.
  30. Learning equations for extrapolation and control. In International Conference on Machine Learning, pages 4442–4450, 2018.
  31. Distilling free-form natural laws from experimental data. Science, 324(5923):81–85, 2009.
  32. Pixelwise view selection for unstructured multi-view stereo. In Computer Vision–ECCV 2016: 14th European Conference, pages 501–518. Springer, 2016.
  33. Physics-informed spline learning for nonlinear dynamics discovery. Proceedings of the 30th International Joint Conference on Artificial Intelligence, pages 2054–2061, 2021.
  34. Bayesian spline learning for equation discovery of nonlinear dynamics with quantified uncertainty. In Advances in Neural Information Processing Systems, 2022.
  35. Symbolic physics learner: Discovering governing equations via monte carlo tree search. In The Eleventh International Conference on Learning Representations, 2023.
  36. Symbolic pregression: Discovering physical laws from distorted video. Physical Review E, 103(4):043307, 2021.
  37. Cot-amflow: Adaptive modulation network with co-teaching strategy for unsupervised optical flow estimation. In Conference on Robot Learning, pages 143–155, 2021.
  38. Simple online and realtime tracking with a deep association metric. In 2017 IEEE International Conference on Image Processing (ICIP), pages 3645–3649, 2017.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets