Automated Training and Maintenance through Kinect

Abstract: In this paper, we have worked on reducing burden on mechanic involving complex automobile maintenance activities that are performed in centralised workshops. We have presented a system prototype that combines Augmented Reality with Kinect. With the use of Kinect, very high quality sensors are available at considerably low costs, thus reducing overall expenditure for system design. The system can be operated either in Speech mode or in Gesture mode. The system can be controlled by various audio commands if user opts for Speech mode. The same controlling can also be done by using a set of Gestures in Gesture mode. Gesture recognition is the task performed by Kinect system. This system, bundled with RGB and Depth camera, processes the skeletal data by keeping track of 20 different body joints. Recognizing Gestures is done by verifying user movements and checking them against predefined condition. Augmented Reality module captures real-time image data streams from high resolution camera. This module then generates 3D model that is superimposed on real time data.