ERIC Number: EJ757080
Record Type: Journal
Publication Date: 2007
Pages: 27
Abstractor: Author
ISBN: N/A
ISSN: ISSN-0047-2395
EISSN: N/A
Sign Language Subtitling by Highly Comprehensible "Semantroids"
Adamo-Villani, Nicolleta; Beni, Gerardo
Journal of Educational Technology Systems, v35 n1 p61-87 2006-2007
We introduce a new method of sign language subtitling aimed at young deaf children who have not acquired reading skills yet, and can communicate only via signs. The method is based on: 1) the recently developed concept of "semantroid[TM]" (an animated 3D avatar limited to head and hands); 2) the design, development, and psychophysical evaluation of a highly comprehensible model of the semantroid; and 3) the implementation of a new multi-window, scrolling captioning technique. Based on "semantic intensity" estimates, we have enhanced the comprehensibility of the semantroid by: i) the use of non-photorealistic rendering (NPR); and ii) the creation of a 3D face model with distinctive features. We have then validated the comprehensibility of the semantroid through a series of tests on human subjects which assessed accuracy and speed of recognition of facial stimuli and hand gestures as a function of mode of representation and facial geometry. Test results show that, in the context of sign language subtitling (i.e., in limited space), the most comprehensible semantroid model is a toon-rendered model with distinctive facial features. Because of its enhanced comprehensibility, this type of semantroid can be scaled to fit in a very small area, and thus it is possible to display multiple captioning windows simultaneously. The concurrent display of several progressive animated signed sentences allows for review of information, a feature not present in any sign language subtitling method presented so far. As an example of application, we have applied the multi-window, scrolling captioning technique to a children's video of a chemistry experiment. (Contains 7 figures and 1 table.)
Descriptors: Test Results, Semantics, Sign Language, Sentences, Reading Skills, Deafness, Young Children, Models, Validity, Technology Uses in Education, Nonverbal Communication, American Sign Language, Computer Mediated Communication, Language Processing
Baywood Publishing Company, Inc. 26 Austin Avenue, P.O. Box 337, Amityville, NY 11701. Tel: 800-638-7819; Tel: 631-691-1270; Fax: 631-691-1770; e-mail: info@baywood.com; Web site: http://baywood.com
Publication Type: Journal Articles; Reports - Research
Education Level: Early Childhood Education; Elementary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A