Automatic sign-language recognition and translation

Establish automatic recognition and translation of natural sign language into spoken-language representations, including the simultaneous interpretation of handshapes, arm movements, facial expressions, and gaze direction.

Background

The proposed automated interpretation chain requires recognition of the signer’s manual and nonmanual features, followed by translation from sign-language grammar into an equivalent spoken-language structure. The report distinguishes this direction from speech-to-sign systems, for which restricted applications were considered feasible.

The authors explicitly leave natural sign-language recognition and translation unresolved and therefore do not pursue them in the remainder of the chapter.