Is Automatic Speech Recognition Ready for Direct Use by Classroom Teachers? PEPNet 2004 - Presentation Pittsburgh, PA, Sheraton Station Square, April 24, 2004, 10:15 – 11:30 AM Presenter/Author: Kathleen Eilers Crandall, Ph.D. Contributors: Donna E. Gustina, and Stephen S. Campbell National Technical Institute for the Deaf Rochester Institute of Technology
27
Embed
Is Automatic Speech Recognition Ready for Direct Use by Classroom Teachers?
Is Automatic Speech Recognition Ready for Direct Use by Classroom Teachers?. PEPNet 2004 - Presentation Pittsburgh, PA, Sheraton Station Square, April 24, 2004, 10:15 – 11:30 AM Presenter/Author: Kathleen Eilers Crandall, Ph.D. Contributors : Donna E. Gustina, and Stephen S. Campbell - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Is Automatic Speech Recognition Ready for Direct Use by Classroom Teachers?
PEPNet 2004 - PresentationPittsburgh, PA, Sheraton Station Square, April 24, 2004, 10:15 –
11:30 AM
Presenter/Author: Kathleen Eilers Crandall, Ph.D.Contributors: Donna E. Gustina, and Stephen S. Campbell
National Technical Institute for the DeafRochester Institute of Technology
The Glossograph
• Fay wrote about an experimental mechanical device used to transcribe human speech, and said,
• “… it is not unreasonable to hope that some instrument will yet be contrived …“
Fay, E.A. (1883). The glossograph. American Annals of the Deaf, 28, 67-69.
Sci-Fi or Reality?
"The pen was an archaic instrument, seldom used even for signatures...Apart from very short notes, it was usual to dictate everything into the speak-write…” (Nineteen eighty-four. Orwell, 1949)
Project
• Direct teacher use of Continuous Automatic Speech Recognition:– English Classroom/Lab
Funded by a grant from the Parsons Foundation of California
English Classroom/Lab Project
English Classroom/Lab Project
Purpose
Investigate direct use of ASR by classroom teacher to learn:
• Is acceptable recognition level attained?
• Under what conditions?– Style of speaking– Communication mode– Language complexity
Related Work
Use of ASR by an intermediary • Intermediary, a ‘captionist,’ re-speaks
professor’s words into a computer• Intermediary summarizes professor’s
words into a computer (‘interpreted speech’)
• Intermediary may use C-print (a shorthand typing system) in combination with ASR http://cprint.rit.edu/
• Vary by population and message predictability– New vs. Known information– Fluent readers vs.
Language learners– Reading for pleasure vs. Reading to master new
information
• CLOZE research and prediction of missing information
English Classroom/Lab Project
Results: ASR Software
75%
80%
85%
90%
95%
100%
Dragon ViaVoice XP
Conversation
Dictation
English Classroom/Lab Project
Results: Communication Mode
80%
82%
84%
86%
88%
90%
92%
94%
96%
98%
Simultaneous Commmunication Speech Only
Conversation
Dictation
English Classroom/Lab Project
Results: Language Complexity
82%
84%
86%
88%
90%
92%
94%
96%
98%
< 7th Grade > 7th Grade
Conversation
Dictation
English Classroom/Lab Project
Correcting Text
• Error correction– What to correct – When to correct– How to correct
Multitasking Demands
• Normal tasks for speaker/teacher– Formulating ideas relevant to topic– Attending to learning needs of students – Meeting lipreading and sign language needs
• Added tasks for speaker/teacher – Speaking to produce readable ASR text– Monitoring text– Making corrections
RecommendationsDiscussionQuestions
Grammatical Correctness
• Is ASR accuracy affected by the grammatical correctness of the user’s speech?
• Student written responses spoken as written: Accuracy – 93.8%
• Student written responses spoken after corrected: Accuracy - 94.3%
Style of Speaking
1. Style of speaking that more closely resembles dictation approaches a usable accuracy rate.
2. Lowering the complexity does not improve accuracy.
Conditions of Use
Direct use of ASR by a language teacher --Useful only under very controlled conditions.• Illustrating the generation of written
language • Demonstrating the use of notes and
outlines to produce written text• Translating selected sign language
utterances into English text during discussions
ASR: Classroom Use
Prepared Outline
Student’s Screen
Teacher’s Screen
Considerations• Training
– Critical to reach over 90% accuracy– Training with conversation
• Corrections– Familiarity with strategies – Dictate, Spell, Right click
• Equipment– Microphone headsets - design, comfort, and size– Demand on computer processor– Effect of optional settings
Tips for Better Accuracy
• Powerful computer • No other programs running• Consistent microphone placement• Environment• Training• Profile• Join user groups, such as ms-