Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Keypoint-based Stereophotoclinometry for Characterizing and Navigating Small Bodies: A Factor Graph Approach (2312.06865v1)

Published 11 Dec 2023 in cs.CV

Abstract: This paper proposes the incorporation of techniques from stereophotoclinometry (SPC) into a keypoint-based structure-from-motion (SfM) system to estimate the surface normal and albedo at detected landmarks to improve autonomous surface and shape characterization of small celestial bodies from in-situ imagery. In contrast to the current state-of-the-practice method for small body shape reconstruction, i.e., SPC, which relies on human-in-the-loop verification and high-fidelity a priori information to achieve accurate results, we forego the expensive maplet estimation step and instead leverage dense keypoint measurements and correspondences from an autonomous keypoint detection and matching method based on deep learning to provide the necessary photogrammetric constraints. Moreover, we develop a factor graph-based approach allowing for simultaneous optimization of the spacecraft's pose, landmark positions, Sun-relative direction, and surface normals and albedos via fusion of Sun sensor measurements and image keypoint measurements. The proposed framework is validated on real imagery of the Cornelia crater on Asteroid 4 Vesta, along with pose estimation and mapping comparison against an SPC reconstruction, where we demonstrate precise alignment to the SPC solution without relying on any a priori camera pose and topography information or humans-in-the-loop

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Cheng, A. F., Rivkin, A. S., Michel, P., Atchison, J., Barnouin, O., Benner, L., Chabot, N. L., Ernst, C., Fahnestock, E. G., Kueppers, M., Pravec, P., Rainey, E., Richardson, D. C., Stickle, A. M., and Thomas, C., “AIDA DART asteroid deflection test: Planetary defense and science objectives,” Planetary and Space Science, Vol. 157, 2018, pp. 104–115.
  2. Mazanek, D. D., Merrill, R. G., Brophy, J. R., and Mueller, R. P., “Asteroid Redirect Mission concept: A bold approach for utilizing space resources,” Acta Astronautica, Vol. 117, 2015, pp. 163–171.
  3. Rivkin, A. S., and DeMeo, F. E., “How Many Hydrated NEOs Are There?” J. of Geophysical Research: Planets, Vol. 124, No. 1, 2019, pp. 128–142.
  4. Barucci, M. A., Dotto, E., and Levasseur-Regourd, A. C., “Space missions to small bodies: asteroids and cometary nuclei,” Astronomy and Astrophysics Rev., Vol. 19, No. 48, 2011, pp. 1–29.
  5. Bhaskaran, S., Nandi, S., Broschart, S., Wallace, M., Cangahuala, L. A., and 0lson, C., “Small body landings using autonomous onboard optical navigation,” J. of the Astronautical Sciences, Vol. 58, No. 3, 2011, pp. 1365–1378.
  6. Gaskell, R. W., Barnouin-Jha, O. S., Scheeres, D. J., Konopliv, A. S., Mukai, T., Abe, S., Saito, J., Ishiguro, M., Kubota, T., Hashimoto, T., Kawaguchi, J., Yoshikawa, M., Shirakawa, K., Kominato, T., Hirata, N., and Demura, H., “Characterizing and navigating small bodies with imaging data,” Meteoritics & Planetary Science, Vol. 43, No. 6, 2008, pp. 1049–1061.
  7. Raymond, C. A., Jaumann, R., Nathues, A., Sierks, H., Roatsch, T., Preusker, F., Scholten, F., Gaskell, R. W., Jorda, L., Keller, H.-U., Zuber, M. T., Smith, D. E., Mastrodemos, N., and Mottola, S., “The Dawn Topography Investigation,” Space Science Rev., Vol. 163, No. 1, 2011, pp. 487–510.
  8. Mastrodemos, N., Rush, B. P., Vaughan, A. T., and Owen, W. M., “Optical Navigation for the Dawn Mission at Vesta,” Int. Symp. on Space Flight Dynamics (ISSFD), Pasadena, CA, USA, 2012, pp. 1–26.
  9. Lorenz, D. A., Olds, R., May, A., Mario, C., Perry, M. E., Palmer, E. E., and Daly, M., “Lessons learned from OSIRIS-REx autonomous navigation using natural feature tracking,” IEEE Aerospace Conf., Big Sky, MT, USA, 2017, pp. 1–12.
  10. Barnouin, O. S., Daly, M. G., Palmer, E. E., Johnson, C. L., Gaskell, R. W., Al Asad, M., Bierhaus, E. B., Craft, K. L., Ernst, C. M., Espiritu, R. C., Nair, H., Neumann, G. A., Nguyen, L., Nolan, M. C., Mazarico, E., Perry, M. E., Philpott, L. C., Roberts, J. H., Steele, R. J., Seabrook, J., Susorney, H. C. M., Weirich, J. R., and Lauretta, D. S., “Digital terrain mapping by the OSIRIS-REx mission,” Planetary and Space Science, Vol. 180, No. 104764, 2020, pp. 1–16.
  11. Gaskell, R., Barnouin, O., Daly, M., Palmer, E., Weirich, J., Ernst, C., Daly, R., and Lauretta, D., “Stereophotoclinometry on the OSIRIS-REx Mission: Mathematics and Methods,” The Planetary Science J., Vol. 4, No. 4, 2023.
  12. Quadrelli, M., Wood, L., Riedel, J., McHenry, M., Aung, M., Cangahuala, L., Volpe, R., Beauchamp, P., and Cutts, J., “Guidance, Navigation, and Control Technology Assessment for Future Planetary Science Missions,” J. of Guidance, Control, and Dynamics, Vol. 38, No. 7, 2015, pp. 1165–1186.
  13. Nesnas, I., Hockman, B. J., Bandopadhyay, S., Morrell, B. J., Lubey, D. P., Villa, J., Bayard, D. S., Osmundson, A., Jarvis, B., Bersani, M., and Bhaskaran, S., “Autonomous Exploration of Small Bodies Toward Greater Autonomy for Deep Space Missions,” Frontiers in Robotics and AI, Vol. 8, No. 650885, 2021, pp. 1–26.
  14. Getzandanner, K. M., Antreasian, P. G., Moreau, M. C., Leonard, J. M., Adam, C. D., Wibben, D., Berry, K., Highsmith, D., and Lauretta, D., “Small Body Proximity Operations & TAG: Navigation Experiences & Lessons Learned from the OSIRIS-REx Mission,” AIAA SciTech Forum, San Diego, CA, USA, 2022, pp. 1–23.
  15. Dor, M., Driver, T., Getzandanner, K., and Tsiotras, P., “Visual SLAM for Asteroid Relative Navigation,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) Workshops, Virtual, 2021, pp. 2066–2075.
  16. Dor, M., Skinner, K., Driver, T., and Tsiotras, P., “AstroSLAM: Autonomous Monocular Navigation in the Vicinity of a Celestial Small Body–Theory and Experiments,” arXiv:2212.00350, 2022.
  17. Dellaert, F., and Kaess, M., “Factor Graphs for Robot Perception,” Found. Trends Robotics, Vol. 6, No. 1-2, 2017, pp. 1–139.
  18. Driver, T., Skinner, K., Dor, M., and Tsiotras, P., “AstroVision: Towards Autonomous Feature Detection and Description for Missions to Small Bodies Using Deep Learning,” Special Issue on AI for Space, Acta Astronautica, Vol. 210, 2023.
  19. Kaess, M., Johannsson, H., Roberts, R., Ila, V., Leonard, J., and Dellaert, F., “iSAM2: Incremental Smoothing and Mapping Using the Bayes Tree,” Intl. J. of Robotics Research (IJRR), Vol. 31, No. 2, 2012, pp. 217–236.
  20. Edstedt, J., Athanasiadis, I., Wadenbäck, M., and Felsberg, M., “DKM: Dense kernelized feature matching for geometry estimation,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, 2023, pp. 17765–17775.
  21. Forster, C., Carlone, L., Dellaert, F., and Scaramuzza, D., “On-manifold preintegration for real-time visual–inertial odometry,” IEEE Trans. on Robotics (T-RO), Vol. 33, No. 1, 2016, pp. 1–21.
  22. Absil, P.-A., Baker, C. G., and Gallivan, K. A., “Trust-region methods on Riemannian manifolds,” Foundations of Computational Mathematics, Vol. 7, 2007, pp. 303–330.
  23. Dellaert, F., “Factor graphs and GTSAM: A hands-on introduction,” Tech. Rep. GT-RIM-CP&R-2012-002, Georgia Institute of Technology, 2012.
  24. Bercovici, B., and McMahon, J. W., “Robust Autonomous Small-Body Shape Reconstruction and Relative Navigation Using Range Images,” J. of Guidance, Control, and Dynamics, Vol. 42, No. 7, 2019, pp. 1473–1488.
  25. Nakath, D., Clemens, J., and Rachuy, C., “Active Asteroid-SLAM,” J. of Intelligent and Robotic Systems (JINT), Vol. 99, 2020, pp. 303–333.
  26. Church, E., Bourbeau, T., Curriden, J., Deguzman, A., Jaen, F., Ma, H., Mahoney, K., Miller, C., Short, B., Waldorff, K., et al., “Flash Lidar On-Orbit Performance at Asteroid Bennu,” AAS Guidance, Navigation and Control (GN&C) Conf., Virtual, 2020, pp. 1349–1360.
  27. Leonard, J. M., Moreau, M. C., Antreasian, P. G., Getzandanner, K. M., Church, E., Miller, C., Daly, M. G., Barnouin, O. S., and Lauretta, D. S., “Cross-Calibration of GNC and OLA LIDAR Systems Onboard OSIRIS-REx,” AAS Guidance, Navigation and Control (GN&C) Conf., Breckenridge, CO, USA, 2022, pp. 1–22.
  28. Daly, M. G., Barnouin, O. S., Dickinson, C., Seabrook, J., Johnson, C. L., Cunningham, G., Haltigin, T., Gaudreau, D., Brunet, C., Aslam, I., Taylor, A., Bierhaus, E. B., Boynton, W., Nolan, M., and Lauretta, D. S., “The OSIRIS-REx Laser Altimeter (OLA) Investigation and Instrument,” Space Science Rev., Vol. 212, No. 924, 2017, pp. 899–924.
  29. Rizk, B., d’Aubigny, C. D., Golish, D., Fellows, C., Merrill, C., Smith, P., Walker, M., Hendershot, J., Hancock, J., Bailey, S., DellaGiustina, D., Lauretta, D., Tanner, R., Williams, M., Harshman, K., Fitzgibbon, M., Verts, W., Chen, J., Connors, T., Hamara, D., Dowd, A., Lowman, A., Dubin, M., Burt, R., Whiteley, M., Watson, M., McMahon, T., Ward, M., Booher, D., Read, M., Williams, B., Hunten, M., Little, E., Saltzman, T., Alfred, D., O’Dougherty, S., Walthall, M., Kenagy, K., Peterson, S., Crowther, B., Perry, M., See, C., Selznick, S., Sauve, C., Beiser, M., Black, W., Pfisterer1, R., Lancaster, A., Oliver, S., Oquest, C., Crowley, D., Morgan, C., Castle, C., Dominguez, R., and Sullivan, M., “OCAMS: The OSIRIS-REx Camera Suite,” Space Science Rev., Vol. 214, No. 26, 2018, pp. 1–55.
  30. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G., “ORB: An efficient alternative to SIFT or SURF,” IEEE Int. Conf. on Computer Vision (ICCV), Barcelona, Spain, 2011, pp. 2564–2571.
  31. Rosten, E., and Drummond, T., “Machine learning for high-speed corner detection,” European Conf. on Computer Vision (ECCV), Graz, Austria, 2006, pp. 430–443.
  32. Calonder, M., Lepetit, V., Strecha, C., and Fua, P., “BRIEF: Binary Robust Independent Elementary Features,” European Conf. on Computer Vision (ECCV), Heraklion, Crete, 2010, pp. 778–792.
  33. Woodham, R. J., “Photometric method for determining surface orientation from multiple images,” Optical Engineering, Vol. 19, No. 1, 1980, p. 1309–1332.
  34. Hayakawa, H., “Photometric stereo under a light source with arbitrary motion,” J. of the Optical Society of America (JOSA), Vol. 11, No. 11, 1994, pp. 3079–3089.
  35. Logothetis, F., Mecca, R., and Cipolla, R., “A differential volumetric approach to multi-view photometric stereo,” IEEE Int. Conf. on Computer Vision (ICCV), Seoul, Korea, 2019, pp. 1052–1061.
  36. Shi, B., Tan, P., Matsushita, Y., and Ikeuchi, K., “Bi-polynomial modeling of low-frequency reflectances,” IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 36, No. 6, 2013, pp. 1078–1091.
  37. Ikehata, S., and Aizawa, K., “Photometric stereo using constrained bivariate regression for general isotropic surfaces,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 2179–2186.
  38. Cho, D., Matsushita, Y., Tai, Y.-W., and Kweon, I. S., “Semi-calibrated photometric stereo,” IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 42, No. 1, 2018, pp. 232–245.
  39. Santo, H., Samejima, M., Sugano, Y., Shi, B., and Matsushita, Y., “Deep photometric stereo network,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, HI, USA, 2017, pp. 501–509.
  40. Chen, G., Han, K., and Wong, K.-Y. K., “PS-FCN: A flexible learning framework for photometric stereo,” European Conf. on Computer Vision (ECCV), Munich, Germany, 2018, pp. 3–18.
  41. Bi, S., Xu, Z., Sunkavalli, K., Kriegman, D., and Ramamoorthi, R., “Deep 3D capture: Geometry and reflectance from sparse multi-view images,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Virtual, 2020, pp. 5960–5969.
  42. Kaya, B., Kumar, S., Oliveira, C., Ferrari, V., and Van Gool, L., “Multi-View Photometric Stereo Revisited,” IEEE/CVF Winter Conf. on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2023, pp. 3126–3135.
  43. Ackermann, J., Goesele, M., et al., “A survey of photometric stereo techniques,” Foundations and Trends in Computer Graphics and Vision, Vol. 9, No. 3-4, 2015, pp. 149–254.
  44. Ju, Y., Lam, K.-M., Xie, W., Zhou, H., Dong, J., and Shi, B., “Deep learning methods for calibrated photometric stereo and beyond: A survey,” arXiv:2212.08414, 2022.
  45. Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R., “NeRF: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, Vol. 65, No. 1, 2021, pp. 99–106.
  46. Cadena, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J., Reid, I., and Leonard, J. J., “Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age,” IEEE Trans. on Robotics (T-RO), Vol. 32, No. 6, 2016, pp. 1309–1332.
  47. Baid*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, A., Lambert*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, J., Driver*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, T., Krishnan*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, A., Stepanyan, H., and Dellaert, F., “Distributed Global Structure-from-Motion with a Deep Front-End,” arXiv:2311.18801, 2023. *{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPTAuthors contributed equally to this work.
  48. He, K., Zhang, X., Ren, S., and Sun, J., “Deep residual learning for image recognition,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770–778.
  49. Nistér, D., “An efficient solution to the five-point relative pose problem,” IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 26, No. 6, 2004, pp. 756–770.
  50. Barath, D., and Matas, J., “Graph-Cut RANSAC,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 2018, pp. 6733–6741.
  51. Dellaert, F., Rosen, D. M., Wu, J., Mahony, R. E., and Carlone, L., “Shonan Rotation Averaging: Global Optimality by Surfing S⁢O⁢(p)n𝑆𝑂superscript𝑝n{SO(p)}^{\mbox{n}}italic_S italic_O ( italic_p ) start_POSTSUPERSCRIPT n end_POSTSUPERSCRIPT,” European Conf. on Computer Vision (ECCV), Glasgow, UK, 2020, pp. 292–308.
  52. Wilson, K., and Snavely, N., “Robust Global Translations with 1DSfM,” European Conf. on Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 61–75.
  53. Sampson, P. D., “Fitting conic sections to “very scattered” data: An iterative refinement of the Bookstein algorithm,” Computer Graphics and Image Processing, Vol. 18, No. 1, 1982, pp. 97–108.
  54. Schröder, S., Mottola, S., Keller, H., Raymond, C., and Russell, C., “Resolved photometry of Vesta reveals physical properties of crater regolith,” Planetary and Space Science, Vol. 85, 2013a, pp. 198–213.
  55. Mildenhall*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, B., Srinivasan*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, P. P., Tancik*{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT, M., Barron, J. T., Ramamoorthi, R., and Ng, R., “NeRF: Representing scenes as neural radiance fields for view synthesis,” European Conf. on Computer Vision (ECCV), Virtual, 2020, pp. 1–17. *{}^{*}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPTAuthors contributed equally to this work.
  56. Li, J.-Y., Helfenstein, P., Buratti, B. J., Takir, D., and Clark, B. E., “Asteroid Photometry,” Asteroids IV, University of Arizona Press, 2015, pp. 129–150.
  57. Sierks, H., Keller, H., Jaumann, R., Michalik, H., Behnke, T., Bubenhagen, F., Büttner, I., Carsenty, U., Christensen, U., Enge, R., et al., “The Dawn framing camera,” Space Science Rev., Vol. 163, 2011, pp. 263–327.
  58. Schröder, S., Maue, T., Marqués, P. G., Mottola, S., Aye, K., Sierks, H., Keller, H., and Nathues, A., “In-flight calibration of the Dawn Framing Camera,” Icarus, Vol. 226, No. 2, 2013b, pp. 1304–1317.
  59. Schröder, S., Mottola, S., Matz, K.-D., and Roatsch, T., “In-flight calibration of the Dawn Framing Camera II: Flat fields and stray light correction,” Icarus, Vol. 234, 2014, pp. 99–108.
  60. McEwen, A. S., “Photometric functions for photoclinometry and other applications,” Icarus, Vol. 92, No. 2, 1991, pp. 298 – 311.
  61. McEwen, A. S., “A precise Lunar photometric function,” Lunar and Planetary Science, Vol. 27, 1996, pp. 841–842.
  62. Alexandrov, O., and Beyer, R. A., “Multiview shape-from-shading for planetary images,” Earth and Space Science, Vol. 5, No. 10, 2018, pp. 652–666.
  63. Bierhaus, E. B., Clark, B. C., Harris, J. W., Payne1, K. S., Dubisher, R. D., Wurts, D. W., Hund, R. A., Kuhns, R. M., Linn, T. M., Wood, J. L., May, A. J., Dworkin, J. P., Beshore, E., and Lauretta, D. S., “The OSIRIS-REx Spacecraft and the Touch-and-Go Sample Acquisition Mechanism (TAGSAM),” Space Science Rev., Vol. 214, No. 107, 2018, pp. 1–46.
  64. Horn, B. K., “Height and gradient from shading,” Int. J. of Computer Vision (IJCV), Vol. 5, No. 1, 1990, pp. 37–75.
  65. Capanna, C., Gesquière, G., Jorda, L., Lamy, P., and Vibert, D., “Three-dimensional reconstruction using multiresolution photoclinometry by deformation,” The Visual Computer, Vol. 29, 2013, pp. 825–835.
  66. NASA, “Planetary Data System (PDS),” https://pds.nasa.gov/, 2022.
  67. Li, J.-Y., “Body-Fixed Coordinate Systems for Asteroid (4) Vesta,” https://sbnarchive.psi.edu/pds3/dawn/vir/DWNVVIR_I1B_v2/DOCUMENT/VESTA_COORDINATES/VESTA_COORDINATES_131018.HTM, 2012.
  68. Denevi, B. W., Blewett, D., Buczkowski, D., Capaccioni, F., Capria, M., De Sanctis, M., Garry, W., Gaskell, R., Le Corre, L., Li, J.-Y., et al., “Pitted terrain on Vesta and implications for the presence of volatiles,” Science, Vol. 338, No. 6104, 2012, pp. 246–249.
  69. Williams, D. A., O’Brien, D. P., Schenk, P. M., Denevi, B. W., Carsenty, U., Marchi, S., Scully, J. E., Jaumann, R., De Sanctis, M. C., Palomba, E., et al., “Lobate and flow-like features on asteroid Vesta,” Planetary and Space Science, Vol. 103, 2014, pp. 24–35.
  70. Briechle, K., and Hanebeck, U. D., “Template matching using fast normalized cross correlation,” Optical Pattern Recognition XII, Vol. 4387, SPIE, 2001, pp. 95–102.
  71. Xiong, Y., Olson, C. F., and Matthies, L. H., “Computing depth maps from descent images,” Machine Vision and Applications, Vol. 16, 2005, pp. 139–147.
  72. Zinßer, T., Schmidt, J., and Niemann, H., “Point set registration with integrated scale estimation,” Int. Conf. on Pattern Recognition and Information Processing (PRIP), Minsk, Belarus, 2005, pp. 116–119.
  73. Golish, D., Li, J.-Y., Clark, B., DellaGiustina, D., Zou, X.-D., Rizos, J., Hasselmann, P., Bennett, C., Fornasier, S., Drouet d’Aubigny, C., et al., “Regional Photometric Modeling of Asteroid (101955) Bennu,” The Planetary Science J., Vol. 124, No. 2, 2021, pp. 298 – 311.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now