Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Planar Parallax for Monocular Depth Estimation (2301.03178v2)

Published 9 Jan 2023 in cs.CV

Abstract: Recent research has highlighted the utility of Planar Parallax Geometry in monocular depth estimation. However, its potential has yet to be fully realized because networks rely heavily on appearance for depth prediction. Our in-depth analysis reveals that utilizing flow-pretrain can optimize the network's usage of consecutive frame modeling, leading to substantial performance enhancement. Additionally, we propose Planar Position Embedding (PPE) to handle dynamic objects that defy static scene assumptions and to tackle slope variations that are challenging to differentiate. Comprehensive experiments on autonomous driving datasets, namely KITTI and the Waymo Open Dataset (WOD), prove that our Planar Parallax Network (PPNet) significantly surpasses existing learning-based methods in performance.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Attention attention everywhere: Monocular depth prediction with skip attention. In WACV, 2023.
  2. PointNetLK: Robust & efficient point cloud registration using PointNet. In CVPR, 2019.
  3. Multi-view depth estimation by fusing single-view depth probability with multi-view geometry. In CVPR, 2022.
  4. Detection of close cut-in and overtaking vehicles for driver assistance based on planar parallax. In IV, 2005.
  5. AdaBins: Depth estimation using adaptive bins. In CVPR, 2021.
  6. Unsupervised scale-consistent depth and ego-motion learning from monocular video. NeurIPS, 2019.
  7. Unsupervised scale-consistent depth learning from video. IJCV, 2021.
  8. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012.
  9. Object modelling by registration of multiple range images. IVC, 1992.
  10. Parallax geometry of smooth surfaces in multiple views. In ICCV, 1999.
  11. Deep geometric functional maps: Robust feature learning for shape correspondence. In CVPR, 2020.
  12. FlowNet: Learning optical flow with convolutional networks. In ICCV, 2015.
  13. Depth map prediction from a single image using a multi-scale deep network. NeurIPS, 2014.
  14. Deep ordinal regression network for monocular depth estimation. In CVPR, 2018.
  15. Are we ready for autonomous driving? the KITTI vision benchmark suite. In CVPR, 2012.
  16. Cascade cost volume for high-resolution multi-view stereo and stereo matching. In CVPR, 2020.
  17. Deep residual learning for image recognition. In CVPR, 2016.
  18. FlowNet 2.0: Evolution of optical flow estimation with deep networks. In CVPR, 2017.
  19. Parallax geometry of pairs of points for 3D scene analysis. In ECCV, 1996.
  20. Direct recovery of planar-parallax from multiple frames. PAMI, 2002.
  21. Unsupervised learning of multi-frame optical flow with occlusions. In ECCV, 2018.
  22. Quantitative assessment method of image stitching performance based on estimation of planar parallax. IA, 2021.
  23. Robust consistent video depth estimation. In CVPR, 2021.
  24. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv, 2019.
  25. DeepI2P: Image-to-point cloud registration via deep classification. In CVPR, 2021.
  26. Learning to fuse monocular and multi-view cues for multi-frame depth estimation in dynamic scenes. In CVPR, 2023.
  27. Va-depthnet: A variational approach to single image depth prediction. In ICLR, 2022.
  28. Learning by analogy: Reliable supervision from transformations for unsupervised optical flow estimation. In CVPR, 2020.
  29. Swin Transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021.
  30. Multi-view depth estimation using epipolar spatio-temporal networks. In CVPR, 2021.
  31. Decoupled weight decay regularization. arXiv, 2017.
  32. Using planar parallax to estimate the time-to-contact. In CVPR, 1999.
  33. Consistent video depth estimation. ToG, 2020.
  34. Inverse perspective mapping simplifies optical flow computation and obstacle detection. Biological cybernetics, 1991.
  35. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In CVPR, 2016.
  36. UnFlow: Unsupervised learning of optical flow with a bidirectional census loss. In AAAI, 2018.
  37. Object scene flow for autonomous vehicles. In CVPR, 2015.
  38. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 2019.
  39. idisc: Internal discretization for monocular depth estimation. In CVPR, 2023.
  40. Vision transformers for dense prediction. In ICCV, 2021.
  41. Harpreet S Sawhney. 3D geometry from planar parallax. In CVPR, 1994a.
  42. Harpreet S Sawhney. Motion video analysis using planar parallax. In Storage and Retrieval for Image and Video Databases II, 1994b.
  43. Harpreet S Sawhney. Simplifying motion and structure analysis using planar parallax and image warping. In ICPR, 1994c.
  44. Shashua and Navab. Relative affine structure: theory and application to 3D reconstruction from perspective views. In CVPR, 1994.
  45. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In CVPR, 2018.
  46. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, 2020.
  47. RAFT: Recurrent all-pairs field transforms for optical flow. In ECCV, 2020a.
  48. Deepv2d: Video to depth with differentiable structure from motion. In ICLR, 2020b.
  49. Using plane+parallax for calibrating dense camera arrays. In CVPR, 2004.
  50. Monocular 3D object detection with depth from motion. In ECCV, 2022.
  51. Occlusion aware unsupervised learning of optical flow. In CVPR, 2018.
  52. Joint prediction of monocular depth and structure using planar and parallax geometry. PR, 2022.
  53. GMFlow: Learning optical flow via global matching. In CVPR, 2022.
  54. Gedepth: Ground embedding for monocular depth estimation. In ICCV, 2023.
  55. Enforcing geometric constraints of virtual normal for depth prediction. In ICCV, 2019.
  56. Virtual normal: Enforcing geometric constraints for accurate and robust depth prediction. PAMI, 2021.
  57. Detecting motion regions in the presence of a strong parallax from a moving camera by multiview geometric constraints. PAMI, 2007.
  58. Monocular road planar parallax estimation. arXiv, 2021.
  59. NeWCRFs: Neural window fully-connected CRFs for monocular depth estimation. In CVPR, 2022.
  60. MaskFlowNet: Asymmetric feature matching with learnable occlusion mask. In CVPR, 2020.
  61. Unsupervised deep epipolar flow for stationary or dynamic scenes. In CVPR, 2019.
  62. Lighteddepth: Video depth estimation in light of limited inference view angles. In CVPR, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now