Real-Time Viewport-Aware Optical Flow Estimation in 360-degree Videos for Visually-Induced Motion Sickness Mitigation

Abstract: Visually-induced motion sickness (VIMS), a side effect of perceived motion caused by visual stimulation, is a major obstacle to the widespread use of Virtual Reality (VR). Along with scene object information, visual stimulation can be primarily indicated by optical flow, which characterizes the motion pattern, such as the intensity and direction of the moving image. We estimated the real time optical flow in 360-degree videos targeted at immersive user interactive visualization based on the user's current viewport. The proposed method allows the estimation of customized visual flow for each experience of dynamic 360-degree videos and is an improvement over previous methods that consider a single optical flow value for the entire equirectangular frame. We applied our method to modulate the opacity of granulated rest frames (GRFs), a technique consisting of visual noise-like randomly distributed visual references that are stable to the user's body during immersive pre-recorded 360-degree video experience. We report the results of a pilot one-session between-subject study with 18 participants, where users watched a 2-minute high-intensity 360-degree video. The results show that our proposed method successfully estimates optical flow, with pilot data showing that GRFs combined with real-time optical flow estimation may improve user comfort when watching 360-degree videos. However, more data are needed for statistically significant results.