JOANNEUM RESEARCH Forschungsgesellschaft mbH DIGITAL Institute for Information and Communication Technologies Steyrergasse 17 8010 Graz, Austria Tel. +43 316 876-1240 Fax +43 316 876 - 1191 Georg Thallinger [email protected] [email protected] www.joanneum.at/digital The research leading to the presented results has received funding from the European Union‘s Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 610370. digpos292 014 IC o SOLE A Enable a more comprehensive and immersive experience of live events A Provide better coverage for events spread over large areas A Festival, e.g. Dranouter, Glastonbury – many stages, many parallel performances Motivation Scenario Immersive Coverage of Spatially Outspread Live Events Processing Flow Linear client Interactive client Integrated content and scene representation Content compositing UI Semantic scene creation UI Content analysis Content preselection Interactive playout Professional Content Capture User Generated Content Capture Directed playout Live playout UI UGC Booking System http://www.icosole.eu/ A Omnidirectional cameras and microphones capture action on stage and audience A Users sign up for delivering UGC in return for incentives A Automatic analysis and filtering of incoming streams A Users can watch edited live TV stream, content on second screen web A Entire coverage of the festival is available for later replay Object-based binaural rendering q Upload BWF files with object- based audio metadata q Using different main micropho- nes and mixing techniques q Head tracking for binaural mode 3D audio rendering q Recordings mixed to achieve professional 3D audio q Render object-based audio in real-time using Web Audio API q Rendering to loudspeakers and headphones Playout (server) q GStreamer-based mixing engine running on Linux server q Cocoa GUI running on OSX client q Communication via zeroMQ + JSON Content pre-caching q Mitigate latency issues when zapping in DVB setups q Optimize time-to-first-picture q Will be integrated in playout client for live playback on TV Quality analysis q Technical quality assurance for UGC q Noise Level q Luminance q Blur The Wall of Moments q Short video fragments captured by users attending the festival q Deliver unique festival experi- ence for remote users q Selection based on popularity, friends, music preference etc. Festival Explorer q Web-based client for interactive scene navigation q DASH adaptive streaming of video tiles from 4k overview q Pan & zoom control q Dynamic audio mix bitdash – HD streaming q Platform agnostic MPEG-DASH clients q Smooth content consumption q Live/catch-up and on-demand streaming q HTML5/JS based
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
JOANNEUM RESEARCH Forschungsgesellschaft mbH
DIGITAL Institute for Information and
Communication Technologies
Steyrergasse 17 8010 Graz, Austria
Tel. +43 316 876-1240 Fax +43 316 876 - 1191
Georg [email protected]
[email protected] www.joanneum.at/digital
The research leading to the presented results has received funding from the
European Union‘s Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 610370.
digpos292 014
ICoSOLE
A Enable a more comprehensive and immersive experience of live events
A Provide better coverage for events spread over large areas
A Festival, e.g. Dranouter, Glastonbury – many stages, many parallel performances
Motivation
Scenario
Immersive Coverage of Spatially Outspread Live Events
Processing Flow
Linearclient
Interactiveclient
Integratedcontent
andscene
representation
Contentcompositing
UI
Semanticscene
creationUI
Contentanalysis
Contentpreselection
Interactiveplayout
ProfessionalContentCapture
User GeneratedContentCapture
Directedplayout
Liveplayout
UI
UGCBookingSystem
http
://w
ww
.icos
ole.
eu/
A Omnidirectional cameras and microphones capture action on stage and audience
A Users sign up for delivering UGC in return for incentives
A Automatic analysis and filtering of incoming streams
A Users can watch edited live TV stream, content on second screen web
A Entire coverage of the festival is available for later replay
Object-based binaural rendering q Upload BWF files with object- based audio metadata
q Using different main micropho- nes and mixing techniques
q Head tracking for binaural mode
3D audio rendering q Recordings mixed to achieve professional 3D audio
q Render object-based audio in real-time using Web Audio API
q Rendering to loudspeakers and headphones
Playout (server)q GStreamer-based mixing engine running on Linux server
q Cocoa GUI running on OSX client
q Communication via zeroMQ + JSON
Content pre-cachingq Mitigate latency issues when zapping in DVB setups
q Optimize time-to-first-picture
q Will be integrated in playout client for live playback on TV
Quality analysisq Technical quality assurance for UGC
q Noise Level
q Luminance
q Blur
The Wall of Momentsq Short video fragments captured by users attending the festival
q Deliver unique festival experi- ence for remote users
q Selection based on popularity, friends, music preference etc.
Festival Explorerq Web-based client for interactive scene navigation
q DASH adaptive streaming of video tiles from 4k overview
q Pan & zoom control
q Dynamic audio mix
bitdash – HD streamingq Platform agnostic MPEG-DASH clients
q Smooth content consumption
q Live/catch-up and on-demand streaming
q HTML5/JS based
Related Documents
