42 42 42 42 2 4 42 42 42 42 42 42 Large Scale Processing for Multimedia Analysis Master Seminar, WIntersemester 2011/2012 Dr. Harald Sack, Dr. Peter Tröger Jörg Waitelonis, Christian Hentschel, Bernhard Quehl, Haojin Yang, Magnus Knuth Hasso-Plattner-Institut for IT-Systems Engineering University of Potsdam Dienstag, 18. Oktober 11
45
Embed
Large Scale Processing for Multimedia Analysis - Introduction
Slides for the Introductory lecture of the master seminar 'Large Scape Processing for Multimedia Analysis', Fall semester 2011/12, Hasso-Plattner-Institute, University of Potsdam, 18.10.2011
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
4242 42 4224424242 42 4242Large Scale
Processing for Multimedia Analysis
Master Seminar, WIntersemester 2011/2012
Dr. Harald Sack, Dr. Peter TrögerJörg Waitelonis, Christian Hentschel,
Bernhard Quehl, Haojin Yang, Magnus KnuthHasso-Plattner-Institut for IT-Systems Engineering
University of Potsdam
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
(1) Dozenten(2) Organisatorisches(3) Worum geht es überhaupt(4) nächste Schritte
4242 42 4224424242 42 4242Large Scale Processing for
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
■ Research Topics□ Semantic Web Technologies□ Ontological Engineering□ Information Retrieval□ Multimedia Analysis & Retrieval□ Social Networking□ Data/Information Visualization
■ Research Projects
Semantic Technologies & Multimedia Retrieval
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Dr. Harald Sack
■ Senior Researcher am Hasso Plattner Institut□ Leiter der Forschungsgruppe
,Semantische Technologien & Multimedia Retrieval‘
□ Research Projects
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
■ Seit Februar 2010 Senior Researcher am HPIim Bereich „Verlässliche Many-Core Systeme“
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Administratives:□Semesterwochenstunden: 4□ECTS: 6□Leistungsfeststellung: □Schriftliche Ausarbeitung zum Vortragsthema (Umfang ca. 20 Seiten)□Umsetzung einer vorgegebenen Implementierungsaufgabe im Team□Präsentation der Ergebnisse
<IMG WIDTH="100" ALT="Close-up view of Endeavour's crew cabin prior to docking with the International Space Station" TITLE="Close-up view of Endeavour's crew cabin prior to docking with the International Space Station" SRC="/images/content/549665main_2011-05-18_1600_100-75.jpg" HEIGHT="75" ALIGN="Bottom" BORDER="0" /></a><p><a href="/mission_pages/shuttle/shuttlemissions/sts134/multimedia/index.html">› STS-134 Multimedia</a></p>
...
‣Google Multimedia Search relies on link context
How does Google find Multimedia?
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
How do you find something in an audiovisual archive?
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Step 1: Digitalization of analog data
Step 2: Annotation with (textbased) metadata
How to Search in Multimedia Archives?
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
How to Search in Multimedia Archives?• manual anotation with text-based
descriptive metadata
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
How to Search in Multimedia Archives?• manual anotation with text-based
descriptive metadata
...how to extract metadatain an automated way?
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Face Detection
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Face Detection
Genre Analysis
Classification:StudioIndoor
News Show
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Face Detection
overlay text
Genre Analysis
Classification:StudioIndoor
News Show
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Face Detection
overlay text
Genre Analysis
Classification:StudioIndoor
News Show
scenetext
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Face Detection
overlay text
Logo Detection
Genre Analysis
Classification:StudioIndoor
News Show
scenetext
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Automated Multimedia Analysis
Face Detection
overlay text
Logo Detection
Genre Analysis
Classification:StudioIndoor
News Show
scenetext
Audio-Mining
structuralanalysis
AutomatedSpeech
Recognitionspeaker
identification
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
• Visual Analysis• Structural Analysis• Intelligent Character
Recognition (ICR)• Character/Logo
Detection• Character Filtering• Character Recognition
• Genre Analysis &Categorization
• Face / Body / Object •Detection•Tracking•Clustering
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
video
• Decomposition of time-based media into meaningful media fragments of coherent content that can be used as basic element for indexing and classification
Structural Analysis
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
video
• Decomposition of time-based media into meaningful media fragments of coherent content that can be used as basic element for indexing and classification
Structural Analysis
scenes
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
video
• Decomposition of time-based media into meaningful media fragments of coherent content that can be used as basic element for indexing and classification
Structural Analysis
scenes
shots
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
video
• Decomposition of time-based media into meaningful media fragments of coherent content that can be used as basic element for indexing and classification
Structural Analysis
scenes
shots
subshots
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
video
• Decomposition of time-based media into meaningful media fragments of coherent content that can be used as basic element for indexing and classification
Structural Analysis
scenes
shots
subshots
frameskey frames
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
• Preprocessing• Character Identification• Text Preprocessing
• Text Filtering• Adaption of script geometry (Deskew)• Image quality enhancement
• Optical Character Recognition (OCR)• Standard OCR software (OCRopus)
• Postprocessing• Lexical analysis • Statistical / context based filtering
Ermittlungen nachBombenfunden
Intelligent Character Recognition
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
• Result: Video segments with time-based metadata annotations
• Metadata consist of combined low level / high level feature descriptors• Metadata serve as a basis for traditional and semantic retrieval
Metadata Extractiontime
Automated Multimedia Analysis
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
nächste Woche:□Kickstart - Einführung in die zugrundeliegende Technologie□Finden/Organisation der studentischen Arbeitsgruppen□Ausgabe/Zuweisung der Arbeitsthemen□Aufbereitung/Ausgabe der notwendigen Materialien
in 14 Tagen:□Treffen mit den einzelnen Arbeitsgruppen nach Vereinbarung□Einarbeitung, etc...
Large Scale Processing for Multimedia Analysis
Master Seminar, WIntersemester 2011/2012
Dienstag, 18. Oktober 11
Large Scale Processing for Multimedia Analysis, Master Seminar, WS 2011/2012, Harald Sack, Peter Tröger
(1) Dozenten(2) Organisatorisches(3) Worum geht es überhaupt(4) nächste Schritte
4242 42 4224424242 42 4242Large Scale Processing for