Can rhythm be touched? An evaluation of rhythmic sketch performance with augmented multimodal feedback

Abstract: Although it has been shown that augmented multimodal feedback has a facilitatory effect on motor performance for motor learning and music training, the functionality of haptic feedback combined with other modalities in rhythmic movement tasks has rarely been explored and analysed. In this paper, we evaluate the functionality of visual-haptic feedback in a rhythmic sketch task by comparing it with other multimodal conditions. Further, we examine the possibility of accessing the quality of task execution through kinematic analysis. Based on participants' speed profiles, we investigate the quality of motor control and movement smoothness under different feedback conditions. Results revealed better motor control ability with auditory feedback and improved movement smoothness with haptic feedback. Finally, we propose that haptic feedback can be integrated with other modal stimuli for different interaction purposes, and that kinematic analysis can be a complementary approach to gesture analysis as well as providing subjective evaluation of interaction performance.