Sign language recognition group
Sign language is a "Visual language" in which a sentence is expressed by
facial expression and motion and shape of hands and fingers.
It plays an important role as a way of communication of persons
with hearing impairment when such a person supports himself/herself and
has a social life.
Background and subject
5 percent of the population of Japan
It is said 6 that there exist 6 million people with hearing moderate impairment (where audiometric ISO value is greater than 40dB) in Japan. Since many peaple without hearing impairment cannot use sign language, peaple with hearing impairment may have to put up with discomfort.
 Recognition of sign language words
We are developing an automatic recognition system of a sign language word motion by Hidden Markov Models (HMMs), which are trained by motion and shape of hands extracted by image processing. Since hands and a face often overlap together in a sign language motion, it is difficult to extract shapes of hands by simple image processing. We utilize template matching and detector trained with appearance of a hand to estimate position and shape of a hand.
We use a HMM with topology customized for each sign language word to reflect variety of motions for the word, though previously proposed methods often use a single topology for all words. We also research automatic training data synthesis, which synthesizes a training data from other samples that have partially similar motion or shape of a hand.
We use neural network to recognize facial expressions from feature points extracted from a face on a movie. We have developed a program that can recognize happy, angry, sad, doubtful and impassive facial expressions. However, extraction of feature points on a face may be failed because hands may overlap a face in a movie of practical sign language. We try to improve the program so that it works on such a case.
Assistant professor Tadashi Matsuo ( matsuo [at] i.ci.ritsumei.ac.jp )
Bachelor student Yuka Nakajima ( ynakajima [at] i.ci.ritsumei.ac.jp )