Top Banner
Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/ ) Enabling the IIHS Vision, Part 1 Brandon Muramatsu Andrew McKinney Peter Wilkins—Our colleague at MIT at 0° C January 2010 1 Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/ )
19

IIHS Open Framework-SpokenMedia

Nov 29, 2014

Download

Education

SpokenMedia automatically transcribes IIIHS video, and enables a process to edit and translate transcripts. Presented by Brandon Muramatsu at the IIHS Curriculum Conference, Bangalore, India, January 5, 2010.
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Enabling the IIHS Vision, Part 1

Brandon Muramatsu

Andrew McKinney

Peter Wilkins—Our colleague at MIT at 0° C

January 2010

1Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Page 2: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

2 Demos For January 2010

SpokenMedia Video/audio transcription, enabling translation Process and tools “Access to high-quality learning must be open to all”

Open IIHS Experience Course/activity design; student interaction “Make curriculum openly available”

2

Page 3: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

“The IIHS Website is our commitment to a different way of looking at things.”

3

– Aromar Revi5 January 2010

Page 4: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

“The Institution will fail or scale based on language.”

4

– Aromar Revi5 January 2010

Page 5: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Our Goals with this Demo

Demonstrate transcripts and translations of IIHS videos

Describe the process and our experiences Transcribe -> Edit -> Translate -> Present

5

Page 6: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

What did we do?

6

AutoTranscrib

e

AutoTranscrib

eEditEdit TranslateTranslate PresentPresent

SpokenMedia

Page 7: IIHS Open Framework-SpokenMedia

The Demo

7

Page 8: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

How did we do it?

8

AutoTranscrib

e

AutoTranscrib

eEditEdit TranslateTranslate PresentPresent

SpokenMedia

Page 9: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

How do we do it?Lecture Transcription

• Spoken Lecture: research project• Speech recognition & automated transcription

of lectures• Why lectures?

– Conversational, spontaneous, starts/stops– Different from broadcast news, other types of

speech recognition– Specialized vocabularies

9

James [email protected]

Page 10: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Spoken Lecture Project

• Processor, browser, workflow

• Prototyped with lecture & seminar video– MIT OCW (~300 hours, lectures)– MIT World (~80 hours, seminar speakers)

Supported with iCampus MIT/Microsoft Alliance funding

James [email protected]

10

Page 11: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

How Does it Work?Lecture Transcription Workflow

11

Page 12: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

SpokenMedia Process

12

We used a portion of the SpokenMedia process for the demo

Page 13: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

How did we do it?

13

AutoTranscrib

e

AutoTranscrib

eEditEdit TranslateTranslate PresentPresent

SpokenMedia

Page 14: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Edit & Translate: AccuracyAutomatic

TranscriptionHand

TranscriptionTime

AdjustedTranslated

Hindi

I I I मे�रे� खया�ल से�

think think think

once one one नयाजन की एकी मे�ख्या चु�न�ती� है�

and central

so challenge central

the of

challenger planning challenge of

planning is planning

nice legitimacy is

legitimacy of legitimacy of

of government government सेरेकी�रे की एकी ऐसे� मे�ख्या से�स्था�न की� रूप मे� वै�धती�

government as as

14

Page 15: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

SpokenMedia Accuracy Potential

Accuracy Domain Model and

Speaker Model

Internal validity measure

Seed with transcript

Ongoing research by Jim Glass and his team @ MIT

15

Page 16: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

How did we do it?

16

AutoTranscrib

e

AutoTranscrib

eEditEdit TranslateTranslate PresentPresent

SpokenMedia

Page 17: IIHS Open Framework-SpokenMedia

The Player

Simple Player

Hopes for more features Bookmarks Create snippets

17

Page 18: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Check it out for yourself

Demo site: http://oki-dev.mit.edu/spokenmedia

all the videos from IIHS website…it’s not just Bish!

18

Page 19: IIHS Open Framework-SpokenMedia

Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)

Thank You!

Brandon Muramatsu, [email protected]

Andrew McKinney, [email protected]

19Unless otherwise specified this work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License (creativecommons.org/licenses/by-nc-sa/3.0/us/)