Investigating the Correlation Between Presence and Reaction Time in Mixed Reality (2309.11662v1)
Abstract: Measuring presence is critical to improving user involvement and performance in Mixed Reality (MR). \emph{Presence}, a crucial aspect of MR, is traditionally gauged using subjective questionnaires, leading to a lack of time-varying responses and susceptibility to user bias. Inspired by the existing literature on the relationship between presence and human performance, the proposed methodology systematically measures a user's reaction time to a visual stimulus as they interact within a manipulated MR environment. We explore the user reaction time as a quantity that can be easily measured using the systemic tools available in modern MR devices. We conducted an exploratory study (N=40) with two experiments designed to alter the users' sense of presence by manipulating \emph{place illusion} and \emph{plausibility illusion}. We found a significant correlation between presence scores and reaction times with a correlation coefficient -0.65, suggesting that users with a higher sense of presence responded more swiftly to stimuli. We develop a model that estimates a user's presence level using the reaction time values with high accuracy of up to 80\%. While our study suggests that reaction time can be used as a measure of presence, further investigation is needed to improve the accuracy of the model.
- Air tap gesture. https://docs.microsoft.com/en-us/dynamics365/mixed-reality/guides/operator-gestures-hl2.
- Coffee Stack Game. https://rollicgames.com/games.
- Fruit ninja game. https://www.halfbrick.com/games/fruit-ninja.
- Hololens 2. https://www.microsoft.com/en-us/hololens.
- Intractable objects. https://docs.microsoft.com/en-us/windows/mixed-reality/design/interactable-object.
- Mixed reality capture. https://docs.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/mixed-reality-capture-overview.
- Windows Device Portal. https://learn.microsoft.com/en-us/windows/uwp/debug-test-perf/device-portal.
- Presence and reality judgment in virtual environments: a unitary construct? CyberPsychology & Behavior, 3, 2000.
- W. Barfield. Musings on presence twenty-five years after" being there". Presence Teleoperators Virtual Environ., 25, 2016.
- W. Barfield and S. Weghorst. The sense of presence within virtual environments: A conceptual framework. Advances in Human Factors Ergonomics, 19, 1993.
- An experimental study on the role of touch in shared virtual environments. ACM Transactions on Computer-Human Interaction, 7, 2000.
- Neural correlate of spatial presence in an arousing and noninteractive virtual reality: An eeg and psychophysiology study. Cyberpsychology & Behavior : the Impact of the Internet, Multimedia and Virtual Reality on Behavior and Society, 9, 2006.
- Breaking plausibility without breaking presence - evidence for the multi-layer nature of plausibility. IEEE Transactions on Visualization and Computer Graphics, 28, 2022.
- A conceptual model of the sense of presence in virtual environments. Presence: Teleoperators & Virtual Environments, 8, 1999.
- The dichotomy of presence elements: The where and what. In Proceedings IEEE Virtual Reality, 2003.
- The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. PloS one, 13, 2018.
- Quantitative measures of presence in virtual environments: the roles of attention and spatial comprehension. CyberPsychology & Behavior, 2, 1999.
- P. De Boeck and M. Jeon. An overview of models for response times and processes in cognitive tests. Frontiers in psychology, 10, 2019.
- A free, easy-to-use, computer-based simple and four-choice reaction time programme: the deary-liewald reaction time task. Behaviour Research, 43, 2011.
- Exploring the limitations of environment lighting on optical see-through head-mounted displays. In Symposium on Spatial User Interaction, 2020.
- Presence: A review. International Journal of Human Computer Interaction, 38, 2022.
- Effects of sensory information and prior experience on direct subjective ratings of presence. Presence, 8, 1999.
- J. Freeman and S. E. Avons. Focus group exploration of presence through advanced broadcast services. In Human Vision and Electronic Imaging V, vol. 3959. SPIE, 2000.
- Using behavioral realism to estimate presence: A study of the utility of postural responses to motion stimuli. Presence: Teleoperators & Virtual Environments, 9, 2000.
- Experiences with an ar evaluation test bed: Presence, performance, and physiological measurement. In IEEE International Symposium on Mixed and Augmented Reality, 2010.
- Perceptual uncertainty and action consequences independently affect hand movements in a virtual environment. Scientific Reports, 10: 22307, 12 2020.
- F. Graham. Attention: The heartbeat, the blink and the brain. Attention and Information Processing in Infants and Adults: Perspectives from Human and Animal Research (Eds: B.A. Campell, H. Hayne, R. Richardson), 1992.
- S. Grassini and K. Laumann. Questionnaire measures and physiological correlates of presence: A systematic review. Frontiers in Psychology, 11, 2020.
- Augmented feedback influences upper limb reaching movement times but does not explain violations of fitts’ law. Frontiers in psychology, 6, 2015.
- C. Heeter. Reflections on real presence by a virtual person. Presence: Teleoperators & Virtual Environments, 12, 2003.
- The illusion of presence in immersive virtual reality during an fmri brain scan. CyberPsychology & Behavior, 6, 2003.
- Presence as an emotional experience. Studies in health technology and informatics, 1999.
- Duration estimation and presence. In Presence Conference, 2001.
- Perceived depth and the feeling of presence in 3dtv. Displays, 18, 1998.
- W. A. IJsselsteijn. Presence in depth. Technische Universiteit Eindhoven Eindhoven, The Netherlands, 2004.
- Analysis of physiological response to two virtual environments: Driving and flying simulation. Cyberpsychology & Behavior : the Impact of the Internet, Multimedia and Virtual Reality on Behavior and Society, 5, 2002.
- T. F. I. J.D. Prothero, D.E. Parker. Towards a robust, quantitative measure for presence. Proceedings of the Conference on Experimental Analysis and Measurement of Situation Awareness, 1995.
- Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3, 1993.
- Spatial presence, performance, and behavior between real, remote, and virtual immersive environments. IEEE Transactions on Visualization and Computer Graphics, 26, 2020.
- T. Kim and F. Biocca. Telepresence via television: Two dimensions of telepresence may have different connections to memory and persuasion. Journal of computer-mediated communication, 3, 1997.
- Using eye tracking and psychophysiological methods to study spatial presence. In Annual International Workshop on Presence, 2003.
- Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology, 30, 1993.
- The actor-observer effect in virtual reality presentations. CyberPsychology & Behavior, 4, 2001.
- A cross-media presence questionnaire: The itc-sense of presence inventory. Presence: Teleoperators & Virtual Environments, 10, 2001.
- M. Lombard and T. Ditton. At the heart of it all: The concept of presence. Journal of Computer-Mediated Communication, 3, 1997.
- Presence promotes performance on a virtual spatial cognition task: impact of human factors on virtual reality assessment. Frontiers in Virtual Reality, 1, 2020.
- The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence. Nature Publishing Group Scientific Reports, 11, 2021.
- Physiological measures of presence in stressful virtual environments. ACM Transactions on Graphics, 21, 2002.
- Microsoft. Hololens 2 gestures for authoring and navigating in dynamics 365 guides. https://learn.microsoft.com/en-us/dynamics365/mixed-reality/guides/authoring-gestures-hl2, 2021.
- Microsoft. Improve visual quality and comfort. https://learn.microsoft.com/en-us/hololens/hololens-calibration#calibrating-your-hololens-2, 2022.
- Augmented Reality: A Class of displays on the Reality-Virtuality Continuum. In Telemanipulator and Telepresence Technologies. SPIE, 1995.
- A review of presence and performance in virtual environments. International Journal of Human–Computer Interaction, 12, 2000.
- Designing immersive virtual environments for human behavior research. Frontiers in Virtual Reality, 2, 2021.
- Correlating reaction time and nausea measures with traditional measures of cybersickness. Displays, 48, 2017.
- Measurement of presence and its consequences in virtual environments. International Journal of Human-Computer Studies, 52, 2000.
- D. Nunez. How is presence in non-immersive, non-realistic virtual environments possible? In Proceedings of the 3rd International Conference on Computer Graphics, Virtual Reality, Visualisation and Interaction in Africa, 2004.
- Breaking the experience: Effects of questionnaires in vr user studies. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020.
- H. Regenbrecht and T. W. Schubert. Measuring presence in augmented reality environments: Design and a first test of a questionnaire. ArXiv, abs/2103.02831, 2021.
- X. Retaux. Presence in the environment: Theories, methodologies and applications to video games. PsychNology J., 1, 2003.
- Measuring presence: Subjective, behavioral and physiological methods. Being there: Concepts, effects and measurement of user presence in synthetic environments, 5, 2003.
- The influence of in-vr questionnaire design on the user experience. In Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology, 2021.
- M. Sanchez-Vives and M. Slater. From presence to consciousness through virtual reality. Nature Reviews Neuroscience, 6, 2005.
- T. Schubert. The sense of presence in virtual environments: A three-component scale measuring spatial presence, involvement, and realness. Zeitschrift für Medienpsychologie, 15, 2003.
- The experience of presence: Factor analytic insights. Presence: Teleoperators & Virtual Environments, 10, 2001.
- "where’s pinky?": The effects of a reduced number of fingers in virtual reality. In ACM CHI Conference on Human Factors in Computing System in Play, 2017.
- Using presence questionnaires in virtual reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019.
- "these are not my hands!": Effect of gender on the perception of avatar hands in virtual reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017.
- Scikit-Learn. Gradient Boosting for Classification. https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html, Accessed August 2023.
- Scikit-Learn. A Random Forest Classifier. https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html, Accessed August 2023.
- Scikit-Learn. Soft Voting/Majority Rule Classifier for Unfitted Estimators. https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.VotingClassifier.html, Accessed August 2023.
- T. B. Sheridan. Musings on telepresence and virtual presence. Presence Teleoperators & Virtual Environment, 1, 1992.
- M. Slater. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364, 2009.
- Immersion, presence and performance in virtual environments: An experiment with tri-dimensional chess. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, 1996.
- M. Slater and A. Steed. A virtual presence counter. Presence, 9, 2000.
- The influence of dynamic shadows on presence in immersive virtual environments. Eurographics Workshops on Virtual Environments, 1999.
- Taking steps: the influence of a walking technique on presence in virtual reality. ACM Transactions on Computer-Human Interaction, 2, 1995.
- M. Slater and S. Wilbur. A framework for immersive virtual environments (five): Speculations on the role of presence in virtual environments. Presence: Teleoperators & Virtual Environments, 6, 1997.
- An ethnographic, action-based approach to human experience in virtual environments. International Journal of Human-Computer Studies, 59, 2003.
- What is mixed reality? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, p. 1–15. Association for Computing Machinery, New York, NY, USA, 2019. doi: 10 . 1145/3290605 . 3300767
- A. Steed and J. McDonnell. Experiences with repertory grid analysis for investigating effectiveness of virtual environments. The Online Proceedings of Presence, 2003.
- J. Steuer. Defining virtual reality: Dimensions determining telepresence. Journal of communication, 42, 1992.
- B. Stevens and J. Jerrams-Smith. The sense of object-presence with projection-augmented models. In International Workshop on Haptic Human-Computer Interaction, 2000.
- Putting the virtual into reality: Assessing object-presence with projection-augmented models. Presence: Teleoperators & Virtual Environments, 11, 2002.
- K. Szczurowski and M. Smith. Measuring presence: hypothetical quantitative framework. In 23rd International Conference on Virtual System & Multimedia, 2017.
- Re-creating the botanics: towards a sense of place in virtual environments. In The 3rd UK Environmental Psychology Conference, 2003.
- Using presence questionnaires in reality. Presence, 9, 2000.
- On the role of presence in mixed reality. Presence, 18, 2009.
- The effects of immersiveness on physiology. Studies in Health Technology and Informatics, 1998.
- Accuracy and precision of stimulus timing and reaction times with unreal engine and steamvr. PLoS ONE, 15, 2020.
- Presence and side effects: Complementary or contradictory? In HCI (2). Elsevier, 1997.
- Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators & Virtual Environments, 7, 1998.