Designing for Human Operations on the Moon: Challenges and Opportunities of Navigational HUD Interfaces
Abstract: Future crewed missions to the Moon will face significant environmental and operational challenges, posing risks to the safety and performance of astronauts navigating its inhospitable surface. Whilst head-up displays (HUDs) have proven effective in providing intuitive navigational support on Earth, the design of novel human-spaceflight solutions typically relies on costly and time-consuming analogue deployments, leaving the potential use of lunar HUDs largely under-explored. This paper explores an alternative approach by simulating navigational HUD concepts in a high-fidelity Virtual Reality (VR) representation of the lunar environment. In evaluating these concepts with astronauts and other aerospace experts (n=25), our mixed methods study demonstrates the efficacy of simulated analogues in facilitating rapid design assessments of early-stage HUD solutions. We illustrate this by elaborating key design challenges and guidelines for future lunar HUDs. In reflecting on the limitations of our approach, we propose directions for future design exploration of human-machine interfaces for the Moon.
- 1969. Apollo 11 Preliminary Science Report. https://history.nasa.gov/alsj/a11/a11psr.html. Accessed: 2023-11-18.
- Developing an Augmented Reality Lunar Surface Navigation System. In 2023 IEEE Aerospace Conference. IEEE, 1–8.
- Navigation solution concept and its practical approach development for lunar missions. Russian Engineering Research 40 (2020), 605–607.
- Apollo 11 Preliminary Science Report. NASA SP-214.
- Holo-SEXTANT: an augmented reality planetary EVA navigation interface. 48th International Conference on Environmental Systems.
- Nicholas Joseph Anastas. 2020. Augmented reality navigation system for human traversal of rough terrain. Ph. D. Dissertation. Massachusetts Institute of Technology.
- Field evaluation of T-NASA: Taxi navigation and situation awareness system. In 17th DASC. AIAA/IEEE/SAE. Digital Avionics Systems Conference. Proceedings (Cat. No. 98CH36267), Vol. 2. IEEE, F23–1.
- Graphical user interface concepts for tactical augmented reality. In Head-and Helmet-Mounted Displays XV: Design and Applications, Vol. 7688. SPIE, 144–155.
- Jeffrey Bardzell and Shaowen Bardzell. 2013. What is” critical” about critical design?. In Proceedings of the SIGCHI conference on human factors in computing systems. 3297–3306.
- Situation awareness as a function of frame of reference, computer-graphics eyepoint elevation, and geometric field of view. The International journal of aviation psychology 5, 3 (1995), 233–256.
- Personal navi: Benefits of an augmented reality navigational aid using a see-thru 3d volumetric hud. In Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 1–8.
- When does the driver benefit from AR-information in a navigation task compared to a Head-Up Display? Results of a driving simulator study. Proceedings of the Human Factors and Ergonomics Society Europe (2019), 219–230.
- Using science-driven analog research to investigate extravehicular activity science operations concepts and capabilities for human planetary exploration. Astrobiology 19, 3 (2019), 300–320.
- Planetary extravehicular activity (EVA) risk mitigation strategies for long-duration space missions. npj Microgravity 7, 1 (2021), 16.
- Dennis B Beringer. 1999. Flight command-guidance indicators or pathway displays- My way or the highway?. In International Symposium on Aviation Psychology, 10 th, Columbus, OH. 57–62.
- Training safe and effective spaceflight operations using terrestrial analogues. In Space Safety is No Accident: The 7th IAASS Conference. Springer, 313–318.
- Enhancing Reality: Adaptation Strategies for AR in the Field. In Augmented Cognition. Human Cognition and Behavior: 14th International Conference, AC 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22. Springer, 304–315.
- TJ Blucker and GL Stimmel. 1971. A method for lunar roving vehicle position determination from three landmark observations with a sun compass. Technical Report.
- MoonBuddy: A Voice-based Augmented Reality User Interface That Supports Astronauts During Extravehicular Activities. In Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–4.
- Harold R Booher. 2003. Handbook of human systems integration. Vol. 23. John Wiley & Sons.
- Investigation of EVA Information Interface Technology in a Mars Analog Arctic Field Science Setting. Technical Report. SAE Technical Paper.
- Guy A Boy. 2021. Human systems integration and design. Handbook of Human Factors and Ergonomics (2021), 38–54.
- Augmented reality improves procedural work on an international space station science instrument. Human factors 61, 6 (2019), 866–878. Publisher: SAGE Publications Sage CA: Los Angeles, CA.
- John Brooke. 1996. Sus: a “quick and dirty’usability. Usability evaluation in industry 189, 3 (1996), 189–194.
- Nicola Bruno and James E Cutting. 1988. Minimodularity and the perception of layout. Journal of Experimental Psychology: General 117, 2 (1988), 161.
- Prototyping of Lunar surface geological sampling tools for Moon spacewalk simulations by ESA. 69th International Astronautical Congress (IAC), Bremen, Germany.
- AARON: assistive augmented reality operations and navigation system for NASA’s exploration extravehicular mobility unit (xEMU). In International Conference on Intelligent Human Computer Interaction. Springer, 406–422.
- Geologic traverse planning for planetary EVA. Technical Report. SAE Technical Paper.
- A wearable computer for support of astronaut extravehicular activity. In Proceedings. Sixth International Symposium on Wearable Computers,. IEEE, 23–30.
- Yu-Shan Chang. 2022. Influence of virtual reality on engineering design creativity. Educational Studies 48, 3 (2022), 341–357.
- Evidence report: risk of injury and compromised performance due to EVA operations. Technical Report.
- Interface development for early notification warning system: Full windshield head-up display case study. In Human-Computer Interaction. Interacting in Various Application Domains: 13th International Conference, HCI International 2009, San Diego, CA, USA, July 19-24, 2009, Proceedings, Part IV 13. Springer, 683–692.
- David Coan. 2020. Exploration EVA System Concept of Operations Summary for Artemis Phase 1 Lunar Surface Mission. In EVA Exploration Workshop 2020. Issue: JSC-E-DAA-TN77584.
- Augmented reality for battlefield awareness. DRDC Toronto (2005).
- Kristine Davis and Ian Meginnis. 2019. Testing of the NASA Exploration extravehicular mobility unit demonstration (xemu demo) architecture at the neutral buoyancy laboratory. 49th International Conference on Environmental Systems.
- Categorisation of future applications for Augmented Reality in human lunar exploration. In Proceedings of the International Astronautical Congress, IAC. Paris, Frankreich. https://elib.dlr.de/191497/ ISSN: 0074-1795.
- Human systems integration in the military. (2015).
- Mica R Endsley. 2017. Direct measurement of situation awareness: Validity and use of SAGAT. In Situational awareness. Routledge, 129–156.
- Mica R Endsley and Esin O Kiris. 1995. The out-of-the-loop performance problem and level of control in automation. Human factors 37, 2 (1995), 381–394.
- Dean B. Eppler. 1991. Lighting constraints on lunar surface operations. Technical Report NAS 1.15:4271. https://ntrs.nasa.gov/citations/19910013701 NTRS Author Affiliations: NASA Lyndon B. Johnson Space Center NTRS Document ID: 19910013701 NTRS Research Center: Legacy CDMS (CDMS).
- Pathway HUDs: are they viable? Human Factors 43, 2 (2001), 173–193.
- C. I. Fassett and M. Zanetti. 2020. Effective Crewed Surface Science near the Lunar South Pole: Some Illumination Considerations. Virtual. https://ntrs.nasa.gov/citations/20200003145 NTRS Author Affiliations: Marshall Space Flight Center NTRS Report/Patent Number: MSFC-E-DAA-TN78862 NTRS Document ID: 20200003145 NTRS Research Center: Marshall Space Flight Center (MSFC).
- Edith Fischer and Richard F Haines. 1980. Cognitive issues in head-up displays. Technical Report.
- Making HSI more intelligent: human systems exploration versus experiment for the integration of humans and artificial cognitive systems. In Intelligent Human Systems Integration 2019: Proceedings of the 2nd International Conference on Intelligent Human Systems Integration (IHSI 2019): Integrating People and Intelligent Systems, February 7-10, 2019, San Diego, California, USA. Springer, 563–569.
- Towards a balanced analysis for a more intelligent human systems integration. In Intelligent Human Systems Integration 2021: Proceedings of the 4th International Conference on Intelligent Human Systems Integration (IHSI 2021): Integrating People and Intelligent Systems, February 22-24, 2021, Palermo, Italy. Springer, 31–37.
- Human systems exploration for ideation and innovation in potentially disruptive defense and security systems. In Disruption, Ideation and Innovation for Defence and Security. Springer, 79–117.
- Using augmented reality to cue obstacles for people with low vision. Optics Express 31, 4 (2023), 6827–6848.
- A helmet mounted display demonstration unit for a Space Station application. Technical Report. SAE Technical Paper.
- The Lunar Pathfinder PNT Experiment and Moonlight Navigation Service: The Future of Lunar Position, Navigation and Timing. In Proceedings of the 2022 International Technical Meeting of The Institute of Navigation. 632–642.
- AMADEE-18 and the analog mission performance metrics analysis: a benchmarking tool for mission planning and evaluation. Astrobiology 20, 11 (2020), 1295–1302.
- NC Haney and TG Graff. 2020. Apollo 17 EVA-1 and EVA-2 Task Decomposition: Planning for Artemis and Future Mars Missions. In Lunar and Planetary Science Conference.
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
- Shaojun He. 2011. Integration of multiple sensors for astronaut navigation on the lunar surface. The Ohio State University.
- Matthew H Hersch. 2009. Checklist: The secret life of Apollo’s ‘fourth crewmember’. The Sociological Review 57, 1_suppl (2009), 6–24.
- Lawrence J Hettinger and Gary E Riccio. 1992. Visually induced motion sickness in virtual environments. Presence: Teleoperators & Virtual Environments 1, 3 (1992), 306–310.
- Requirements and potential for enhanced eva information interfaces. Technical Report. SAE Technical Paper.
- HUMEX, a study on the survivability and adaptation of humans to long-duration exploratory missions, part I: lunar missions. Advances in Space Research 31, 11 (2003), 2389–2401.
- NASA habitat demonstration unit (HDU) deep space habitat analog. In AIAA Space 2013 conference and Exposition. 5436.
- ReLive: Bridging In-Situ and Ex-Situ Visual Analytics for Analyzing Mixed Reality User Studies. In CHI Conference on Human Factors in Computing Systems. 1–20.
- ILC Dover. 2023. Planetary Exploration Suits - Astro™. https://www.ilcdover.com/products/planetary-suits/. Accessed: 2023-12-13.
- Project Moonwalk : lessons learnt from testing human robot collaboration scenarios in a lunar and Martian simulation. https://api.semanticscholar.org/CorpusID:197641501
- Project Moonwalk: lessons learnt from testing human robot collaboration scenarios in a lunar and Martian simulation. In 69th International Astronautical Congress (IAC). IAC-17-F1. 2.3, Adelaide, SA.
- ISO13407 ISO. 1999. 13407: Human-centred design processes for interactive systems. Geneva: ISO (1999).
- Riitta Jääskeläinen. 2010. Think-aloud protocol. Handbook of translation studies 1 (2010), 371–374.
- ” in vr, everything is possible!”: Sketching and simulating spatially-aware interactive spaces in virtual reality. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–16.
- The Use of 2D and 3D Displays for Shape-Understanding versus Relative-Position Tasks. Human Factors: The Journal of Human Factors and Ergonomics Society 43 (2001), 79 – 98. https://api.semanticscholar.org/CorpusID:16140880
- Jaclyn L Kagey. 2022. EVA Operations Design Considerations. In Unique Science from the Moon in the Artemis Era Workshop.
- Oliver Beren Kaul and Michael Rohs. 2017. Haptichead: A spherical vibrotactile grid around the head for 3d guidance in virtual and augmented reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 3729–3740.
- SeungJun Kim and Anind K Dey. 2009. Simulated augmented reality windshield display as a cognitive mapping aid for elder driver navigation. In Proceedings of the SIGCHI conference on human factors in computing systems. 133–142.
- Virtual reality as an empirical research tool—Exploring user experience in a real building and a corresponding virtual model. Computers, environment and urban systems 54 (2015), 363–375.
- Autonomous Access to the Moon for Europe: The European Large Logistic Lander. 73rd International Astronautical Congress 2022 (2022), 1820–1822.
- Lunar surface concept of operations for the global exploration roadmap lunar surface exploration scenario. In 72nd International Astronautical Congress 2021.
- Lunar Surface Concept of Operations for the Global Exploration Roadmap Lunar Surface Exploration Scenario. https://ntrs.nasa.gov/citations/20210022596 NTRS Author Affiliations: European Space Agency, Japan Aerospace Exploration Agency, Langley Research Center, Johnson Space Center, Canadian Space Agency NTRS Meeting Information: 72nd International Astronautical Congress 2021; 2021-10-25 to 2021-10-29; undefined NTRS Document ID: 20210022596 NTRS Research Center: Headquarters (HQ).
- Design cognitive actions stimulating creativity in the VR design environment. In Proceedings of the 2019 on Creativity and Cognition. 604–611.
- Astronaut Smart Glove: A Human-Machine Interface For the Exploration of the Moon, Mars, and Beyond. 2020 International Conference on Environmental Systems.
- James R Lewis and Jeff Sauro. 2018. Item benchmarks for the system usability scale. Journal of Usability Studies 13, 3 (2018).
- Sketching in HCI: Hands-on Course of Sketching Techniques. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–5.
- Explainable ai: A review of machine learning interpretability methods. Entropy 23, 1 (2020), 18.
- Brett Lindberg and Jan Kallberg. 2020. Augmented reality: Seeing the benefits is believing. (2020).
- Military Applications of Augmented Reality. In Handbook of Augmented Reality, Borko Furht (Ed.). Springer, New York, NY, 671–706. https://doi.org/10.1007/978-1-4614-0064-6_31
- Jeffry Long and Christopher D Wickens. 1994. Conformal versus nonconformal symbology and the head-up display. In Helmet-and Head-Mounted Displays and Symbology Design Requirements, Vol. 2218. SPIE, 361–369.
- testing of an augmented reality tool for geological fieldwork during two analogue missions. https://dl.iafastro.directory/event/IAC-2022/paper/69201/
- Does augmented-reality head-up display help? A preliminary study on driving performance through a vr-simulated eye movement analysis. Ieee Access 9 (2021), 129951–129964.
- Chia-Chi Mao and Fei-Yi Chen. 2019. Augmented reality and 3-D visualization effects to enhance battlefield situational awareness. In International Conference on Human Interaction and Emerging Technologies. Springer, 303–309.
- Jose A Marmolejo. 1994. Helmet-mounted display and associated research activities recently conducted by the NASA Johnson Space Center. In Helmet-and Head-Mounted Displays and Symbology Design Requirements, Vol. 2218. SPIE, 281–291.
- Jose A Marmolejo. 1996. An Electronic Cuff Checklist Information Display for Extravehicular Activity. Technical Report. SAE Technical Paper.
- Jessica J Marquez and Dava J Newman. 2006. Mission Planning and Re-planning for Planetary Extravehicular Activities: Analysis of Excursions in a Mars-Analog Environment and Apollo Program. Technical Report. SAE Technical Paper.
- An independent human factors analysis and evaluation of the emergency medical protocol checklist for the International Space Station. Technical Report.
- Evaluation of a tunnel-in-the-sky head-up display design for curved approaches using eye-tracking. Aviation Psychology and Applied Human Factors (2020).
- Robert S McCann and David C Foyle. 1996. Scene-linked symbology to improve situation awareness. In AGARD CONFERENCE PROCEEDINGS AGARD CP. Citeseer, 16–16.
- Advanced navigation aids in the flight deck: Effects on ground taxi performance under low visibility conditions. SAE transactions (1996), 1419–1430.
- Attentional limitations with head-up displays. In Proceedings of the 7th international Symposium on Aviation Psychology. 70–75.
- Design of an AR visor display system for extravehicular activity operations. In 2020 IEEE Aerospace Conference. IEEE, 1–11.
- The influence of avatar (self and character) animations on distance estimation, object interaction and locomotion in immersive virtual environments. In Proceedings of the ACM SIGGRAPH Symposium on applied perception in graphics and visualization. 37–44.
- Increasing Agility: A Tangible XR for Rapid Prototype Development, Design Exploration and Usability Assessment. In Advances in Manufacturing, Production Management and Process Control: Proceedings of the AHFE 2021 Virtual Conferences on Human Aspects of Advanced Manufacturing, Advanced Production Management and Process Control, and Additive Manufacturing, Modeling Systems and 3D Prototyping, July 25-29, 2021, USA. Springer, 85–91.
- Paul Milgram and Herman Colquhoun. 1999. A taxonomy of real and virtual world display integration. Mixed reality: Merging real and virtual worlds 1, 1999 (1999), 1–26. Publisher: Springer New York.
- Operational assessment of Apollo lunar surface extravehicular activity. Technical Report.
- David A Mindell. 2011. Digital Apollo: Human and machine in spaceflight.
- Haptic fidelity framework: Defining the factors of realistic haptic feedback for virtual reality. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–17.
- Charles S Mullin Jr. 1960. Some psychological aspects of isolated Antarctic living. American journal of psychiatry 117, 4 (1960), 323–325.
- Augmented reality navigation systems. Universal Access in the Information Society 4 (2006), 177–187.
- NASA. 2019. Spacewalk Spacesuit Basics - NASA. https://www.nasa.gov/centers-and-facilities/johnson/spacewalk-spacesuit-basics/
- Alex Newson et al. 2015. Fifty Watches That Changed the World: Design Museum Fifty. Hachette UK.
- Out of this World Design: Bridging the Gap between Space Systems Engineering and Participatory Design Practices. SpaceCHI 2023 (2023).
- Using Virtual Reality to Shape Humanity’s Return to the Moon: Key Takeaways from a Design Study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
- Using Virtual Reality to Design and Evaluate a Lunar Lander: The EL3 Case Study. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22). Association for Computing Machinery, New York, NY, USA, 1–7. https://doi.org/10.1145/3491101.3519775
- Jonathan O’Callaghan. YearOfPublication. Astronauts Will Wear These Spacesuits on the Moon–And Maybe Mars, Too. https://www.scientificamerican.com/article/astronauts-will-wear-these-spacesuits-on-the-moon-mdash-and-maybe-mars-too/
- Ron T Ogan. 2016. Navigational Displays. Aerospace Navigation Systems (2016), 299–320.
- Road obstacle detection method based on an autoencoder with semantic segmentation. In Proceedings of the Asian Conference on Computer Vision.
- Jeffrey Osterlund and Brad Lawrence. 2012. Virtual reality: Avatars in human spaceflight training. Acta Astronautica 71 (2012), 139–150.
- SpaceCHI: Designing Human-Computer Interaction Systems for Space Exploration. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1–6.
- SpaceCHI 2.0: Advancing Human-Computer Interaction Systems for Space Exploration. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–7.
- Local sleep-like events during wakefulness and their relationship to decreased alertness in astronauts on ISS. npj Microgravity 5, 1 (2019), 10.
- The efficacy of head-down and head-up synthetic vision display concepts for retro-and forward-fit of commercial aircraft. The International Journal of Aviation Psychology 14, 1 (2004), 53–77.
- Lawrence J Prinzel III and Matthew Risser. 2004. Head-up displays and attention capture. Technical Report.
- Design and evaluation of an Augmented Reality tool for future human space exploration aided by an Internet of Things architecture. Advances in Space Research 70, 8 (2022), 2145–2166.
- Fast data-driven computation and intuitive visualization of fiber orientation uncertainty in 3D-polarized light imaging. Frontiers in Physics (2022), 893.
- Matthias Schneider. 2021. Navigation in the Augmented Reality Head-up Display: Guidelines for the Development of AR HUD Navigation Concepts. Ph. D. Dissertation. University of Regensburg.
- Erik Seedhouse. 2022. Starship. In SpaceX: Starship to Mars–The First 20 Years. Springer, 171–188.
- Mel Slater and Martin Usoh. 1993. Presence in immersive virtual environments. In Proceedings of IEEE virtual reality annual international symposium. IEEE, 90–96.
- Summary of the results from the lunar orbiter laser altimeter after seven years in lunar orbit. Icarus 283 (2017), 70–91.
- The Artemis Program: An Overview of NASA’s Activities to Return Humans to the Moon. 2020 IEEE Aerospace Conference, 1–10. https://doi.org/10.1109/AERO47225.2020.9172323
- The Artemis Program: An Overview of NASA’s Activities to Return Humans to the Moon. In 2020 IEEE Aerospace Conference. 1–10. https://doi.org/10.1109/AERO47225.2020.9172323 ISSN: 1095-323X.
- Head-up displays and visual attention: Integrating data and theory. Human factors and aerospace safety (2001).
- Support systems on the industrial shop-floors of the future–operators’ perspective on augmented reality. Procedia Cirp 44 (2016), 108–113.
- Comparative effectiveness of augmented reality in object assembly. In Proceedings of the SIGCHI conference on Human factors in computing systems. 73–80.
- Experimental evaluation of an augmented reality visualization for directing a car driver’s attention. In Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05). IEEE, 56–59.
- A system for supporting distributed field science operations during astronaut training and planetary exploration: The electronic fieldbook. In 52nd Lunar and Planetary Science Conference. 1118.
- An analysis of illumination and communication conditions near lunar south pole based on Kaguya Data. In Proceedings of International Planetary Probe Workshop, Barcelona. 1–7.
- Patricia May Ververs and Christopher D Wickens. 1998. Head-up displays: effect of clutter, display intensity, and display location on pilot performance. The International Journal of Aviation Psychology 8, 4 (1998), 377–403.
- The Artemis III science definition team report. In 52nd Lunar and Planetary Science Conference. 1261.
- DJ Weintraub and MJ Ensing. 1992. The book of HUD: A head-up display state of the art report (CSERIAC State of the Art Report). Wright-Patterson Air Force Base, OH: Armstrong Aerospace Medical Research Lab (1992).
- Christopher D Wickens. 2003. Aviation displays. Principles and practices of aviation psychology (2003), 147–199.
- Christopher D Wickens and Jeffry Long. 1995. Object versus space-based models of visual attention: Implications for the design of head-up displays. Journal of Experimental Psychology: Applied 1, 3 (1995), 179.
- CubeSat lunar positioning system enabled by novel on-board electric propulsion. IEEE Transactions on Plasma Science 46, 2 (2018), 319–329.
- Comparing pilots’ taxi performance, situation awareness and workload using command-guidance, situation-guidance and hybrid head-up display symbologies. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 46. SAGE Publications Sage CA: Los Angeles, CA, 16–20.
- XAIR: A Framework of Explainable AI in Augmented Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–30.
- Head up versus head down: The costs of imprecision, unreliability, and visual clutter on cue effectiveness for display signaling. Human factors 45, 3 (2003), 390–407.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.