SPONSORING OFFICE:
Office of Special Education and Rehabilitative Services

Abstract:
We propose to integrate speech recognition and natural language processing software with Seamless Solutions, Inc.'s PC-based Signing Avatars? 3D character animations of sign language communication to provide a prototype PC-based speech-to-sign language and text-to-sign language interpreter. The key new development for this Phase II SBIR will be the addition of Natural Language Processing (NLP), using a sign lexicon and translation rules that analyze input English sentences to provide not only correct translation of each sign, but also correct and realistic facial expression, timing, and emphasis. The result will be the synthesis of high quality, realistic sign language translation of spoken or textual English language input.

The completed prototype Speech to Sign Language Interpreter will be evaluated in two applications: 1) as a tool for partially automating the process of adding sign language animations to video and multimedia educational materials, and 2) as a real-time, automated Interpreter for the Deaf.

Summary of Anticipated Results and Implications:
The prototype Speech to Sign Language Interpreter will demonstrate a cost-effective alternative for providing accessibility to video and multimedia for deaf individuals who use sign as a first language. It will allow signed content to be authored and added to existing media at least as readily as closed captioning is added now. As speech recognition and NLP technology progress, the Speech to Sign Language Interpreter will provided a direct, real-time means for communication between the hearing and the Deaf.

Commercialization is being pursued for each of these applications. Both partnering and licensing relationships are being discussed with several companies.

[TTY Emulater]

[Test Success]