Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
120 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CasCalib: Cascaded Calibration for Motion Capture from Sparse Unsynchronized Cameras (2405.06845v1)

Published 10 May 2024 in cs.CV

Abstract: It is now possible to estimate 3D human pose from monocular images with off-the-shelf 3D pose estimators. However, many practical applications require fine-grained absolute pose information for which multi-view cues and camera calibration are necessary. Such multi-view recordings are laborious because they require manual calibration, and are expensive when using dedicated hardware. Our goal is full automation, which includes temporal synchronization, as well as intrinsic and extrinsic camera calibration. This is done by using persons in the scene as the calibration objects. Existing methods either address only synchronization or calibration, assume one of the former as input, or have significant limitations. A common limitation is that they only consider single persons, which eases correspondence finding. We attain this generality by partitioning the high-dimensional time and calibration space into a cascade of subspaces and introduce tailored algorithms to optimize each efficiently and robustly. The outcome is an easy-to-use, flexible, and robust motion capture toolbox that we release to enable scientific applications, which we demonstrate on diverse multi-view benchmarks. Project website: https://github.com/jamestang1998/CasCalib.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. P. Besl and N. McKay. A method for registration of 3-d shapes, ieee t. pattern anal., 14, 239–256, 1992.
  2. Recovering non-rigid 3d shape from image streams. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), volume 2, pages 690–696. IEEE, 2000.
  3. Signature verification using a" siamese" time delay neural network. Advances in neural information processing systems, 6, 1993.
  4. Automatic calibration of stationary surveillance cameras in the wild. In Computer Vision – ECCV 2016 Workshops, volume 9914, 10 2016.
  5. Openpose: Realtime multi-person 2d pose estimation using part affinity fields, 2019.
  6. Wildtrack: A multi-camera hd dataset for dense unscripted pedestrian detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5030–5039, 2018.
  7. M. Contributors. Openmmlab pose estimation toolbox and benchmark. https://github.com/open-mmlab/mmpose, 2020.
  8. Superpoint: Self-supervised interest point detection and description, 2018.
  9. Spatio-temporal calibration of multiple kinect cameras using 3d human pose. Sensors, 22(22), 2022.
  10. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, page 226–231. AAAI Press, 1996.
  11. Single view physical distance estimation using human pose, 2021.
  12. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24(6):381–395, jun 1981.
  13. Multicamera people tracking with a probabilistic occupancy map. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2):267–282, 2008.
  14. Gp2c: Geometric projection parameter consensus for joint 3d pose and focal length estimation in the wild, 2019.
  15. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  16. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7):1325–1339, jul 2014.
  17. A. Jakubovic and J. Velagic. Image feature matching and object detection using brute-force matchers. pages 83–86, 09 2018.
  18. Camera handoff: tracking in multiple uncalibrated stationary cameras. In Proceedings Workshop on Human Motion, pages 113–118, 2000.
  19. H. W. Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83–97, 1955.
  20. Extrinsic camera calibration from a moving person. IEEE Robotics and Automation Letters, 7(4):10344–10351, 2022.
  21. Surveillance camera autocalibration based on pedestrian height distributions. In BMVC 2011 - Proceedings of the British Machine Vision Conference 2011, pages 117.1–117.11, 01 2011.
  22. Robust autocalibration for a surveillance camera network. In Proceedings of IEEE Workshop on Applications of Computer Vision, pages 433–440, 01 2013.
  23. Auto calibration of multi-camera system for human pose estimation. IET Computer Vision, 16(7):607–618, 2022.
  24. D. G. Lowe. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision, 60(2):91–110, nov 2004.
  25. Wxbs: Wide baseline stereo generalizations, 2015.
  26. Unsupervised Calibration of Camera Networks and Virtual PTZ Cameras. In Proc. Computer Vision Winter Workshop (CVWW), 2012.
  27. SuperGlue: Learning feature matching with graph neural networks. In CVPR, 2020.
  28. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4104–4113, 2016.
  29. Deep high-resolution representation learning for human pose estimation, 2019.
  30. I. E. Sutherland. Three-dimensional data input by tablet. Proceedings of the IEEE, 62(4):453–461, 1974.
  31. Human pose as calibration pattern: 3d human pose estimation with multiple unsynchronized and uncalibrated cameras. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1856–18567, 2018.
  32. Esther: Joint camera self-calibration and automatic radial distortion correction from tracking of walking humans. IEEE Access, 7:10754–10766, 2019.
  33. Bundle adjustment - a modern synthesis. In Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, ICCV ’99, page 298–372, Berlin, Heidelberg, 1999. Springer-Verlag.
  34. Automatic multi-camera extrinsic parameter calibration based on pedestrian torsors †. Sensors, 19(22), 2019.
  35. R. Tsai. A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal on Robotics and Automation, 3(4):323–344, 1987.
  36. Wide-baseline multi-camera calibration using person re-identification, 2021.
  37. Deepptz: Deep self-calibration for ptz cameras. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), March 2020.
  38. Q. Zhang and A. B. Chan. Single-frame based deep view synchronization for unsynchronized multi-camera surveillance, 2022.
  39. Z. Zhang. Iterative point matching for registration of free-form curves and surfaces. International journal of computer vision, 13(2):119–152, 1994.
  40. Semantically synchronizing multiple-camera systems with human pose estimation. Sensors, 21(7), 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets