Page 1
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
`
Frans WardTechnical Product ManagerSURFnet Advanced Services
MediaMosa Transcripting Technology Scouting Project and
Proof of Concept
Friday, October 28, 11
Page 2
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 3
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 4
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly
• Archiving is not enough: disclosure and reusing is required!
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 5
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly
• Archiving is not enough: disclosure and reusing is required!
• The use of speech technology is needed (Reduce human effort).
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 6
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 7
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly.
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 8
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly.
• Archiving is not enough: disclosure and reusing is required!
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 9
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly.
• Archiving is not enough: disclosure and reusing is required!
• Adding Metadata is the key component here.
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 10
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• The number of AV-archives on the Internet increases rapidly.
• Archiving is not enough: disclosure and reusing is required!
• Adding Metadata is the key component here.
• The use of speech technology is needed (Reduce human effort).
Disclosure of audiovisual archives
UK National Film and Television Archive, Berkhamstedhttp://www.flickr.com/people/footage/
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 11
SURFnet. We make innovation work1
Huge amount of workand no time-coded relations with video
Adding metadata, the traditional approach:Manual annotation
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 12
SURFnet. We make innovation work1
Adding metadata, the new approach:Using speech-to-text technology for metadata generation
Speech Recognition(Speech-to-Text)Time-coded Transcript
Indexing and Search:Search on fragment level
Audio Extraction
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 13
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGY
• Transcripting: conversion of speech into an electronic text document.
• Automatic Speech Recognition (ASR) seems to be the ideal technology for this.
• In combination with Optical Character Recognition (OCR) of slides.
• Goal: to provide additional metadata for searching in video / lecture recordings.
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 14
SURFnet. We make innovation work1
MEDIAMOSA TRANSCRIPTING TECHNOLOGYThe Technology Scout Project. The process is complex...
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 15
MEDIAMOSA TRANSCRIPTING TECHNOLOGY SCOUTING PROJECT
MediaMosaTranscription by Spraak /Cmu Sphinx
Multi-SourcePlayer
Partners:
• Enhanced Search• Optional Subtitles• Mashup info
Lecture Recording
End User Application
• Recognize the Speech• Produce time-coded
Transcript
• Recording of Teacher• Recording of Slides• Reference material
• Transcode into audio• Store all into an asset
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 16
MEDIAMOSA TRANSCRIPTING PROJECT
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 17
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
MEDIAMOSA TRANSCRIPTING PROJECT
Friday, October 28, 11
Page 18
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
MEDIAMOSA TRANSCRIPTING PROJECTSubtitles:
Friday, October 28, 11
Page 19
SURFnet. We make innovation work1
MediaMosa 3.5
Focus on transcription technology (speech-2-text) and flexible workflows
• Development is started• beta release available: december 2011
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11
Page 20
SURFnet. We make innovation work1
MediaMosa Directions
Q&A
MediaMosa
MediaMosa
MediaMosa
Thanks
for yo
ur
attenti
on!
WWWhttp://mediamosa.org
Online Demohttp://demo.mediamosa.org
Forumhttp://mediamosa.org/forum
Issue Trackerhttp://mediamosa.org/trac
Source Codehttps://github.com/mediamosa
Slidesharehttp://www.slideshare.net/MediaMosa
Twitterhttp://twitter.com/mediamosa
MediaMosa @ 5th TF-Media WorkshopPorto, October 26, 2011 - SURFnet. We make innovation work
Friday, October 28, 11