ISSLOD2011 - Semantic Multimedia

Semantic MultimediaIndian Summer School on Linked Data

Leipzig, 15 Sep. 2011

Dr. Harald SackHasso-Plattner-Institut for IT-Systems Engineering

University of Potsdam

Harald Sack, Hasso-Plattner-Institute for IT-Systems Engineering, Indian Summer School on Linked Data, Leipzig, 12-18. Sep. 2011

■ HPI was founded in October 1998 as a Public-Private-Partnership

■ HPI Research and Teaching is focussed onIT Systems Engineering

■ 10 Professors and 100 Scientific Coworkers■ 450 Bachelor / Master Students ■ HPI is winner of CHE-Ranking 2010

http://hpi.uni-potsdam.de/

■ Research Topics□ Semantic Web Technologies□ Ontological Engineering□ Information Retrieval□ Multimedia Analysis & Retrieval□ Social Networking□ Data/Information Visualization

■ Research Projects

Semantic Technologies & Multimedia Retrieval

Overview(1) Multimedia and Semantics(2) Multimedia Metadata and Ontologies(3) Semantic Multimedia Analysis(4) Semantic Multimedia Retrieval

Semantic MultimediaIndian Summer School on Linked Data, Leipzig, 15 Sep. 2011

1. Multimedia and Semantics

Communication is the activity of conveying meaningful information

Sender

Information

Encoding

Message

Receiver

Information

Decoding

Message

Channel

Claude E. Shannon: ,A mathematical theory of communication‘, Bell System Technical Journal, vol. 27, pp. 379–423, 623-656, July, October, 1948

Shannon‘s Model of Communication

Claude E. Shannon(1916-2001)

Sender

Information

Encoding

Message

Receiver

Information

Decoding

Message

Channel

Claude E. Shannon: ,A mathematical theory of communication‘, Bell System Technical Journal, vol. 27, pp. 379–423, 623-656, July, October, 1948

Shannon‘s Model of Communication

Claude E. Shannon(1916-2001)

Sender

Information

Encoding

Receiver

Information

Decoding

Message Message

Channel

Message Message

Channel

MEDIA: In communications, media (singular medium) are the storage and transmission channels or tools used to store and deliver information or data.

TEXT: In literary theory, a text is a coherent set of symbols that transmits some kind of informative message.

TEXT: In literary theory, a text is a coherent set of symbols that transmits some kind of informative message. Images

Video / Audio

InteractiveElements

time-independent

time-dependent

video / animation

time-independent

One Small Step ...This video shows Neil Armstrong climbing down the lunar module ladder to the lunar surface. The video compares existing footage with the partially restored video. The thumbnail image shows the new footage on the left and the old on the right.

• Information is encoded in media content• Media content contains implicite semantics

SEMANTIC MULTIMEDIA facilitates • explicite semantic annotation • of multimedia content • on different levels of abstraction w.r.t.

• time, • space, and • provenance.

dbpedia:Neil_Armstrong

(1)Identify media fragment(2)Annotate with explicite semantics

dbpedia:Astronautdbpedia:Flag

(1)Identify media fragment(2)Annotate with explicite semantics

2. Multimedia Metadata and Ontologies

How can we put (semantic) metadata at the appropriate place within the media?

What is ,Metadata‘?

What is ,Metadata‘?„Metadata is defined as data providing information about one or more aspects of the data“ (informal Definition, Wikipedia)

„Metadata is structured, encoded data that describe characteristics of information-bearing entities to aid in the identification, discovery, assessment, and management of the described entities.“ (W.R. Durell, 1985)

„Metadata is machine understandable information about web resources or other things.“ (T.Berners-Lee, Axioms of Web Architecture: Metadata, W3C, 1997)

Metadata

•Simple example: bibliographic metadata

Identification viaISBN / ISSNauthor(s)titel...

Classification viacategorieskeywordsabstract...

Structured Metadata• name-value pairs (e.g. author=‘Ernest Hemingway‘)

• typed (e.g. author is of type string)

• Meaning (semantics) of structured data is only implicite, i.e. it relies on mutual agreement about the proper usage of the data (e.g. Standardization for Dublin Core)

• Title: A name given to the resource. • Creator: An entity primarily responsible for making the resource. • Subject: The topic of the resource. • Description: An account of the resource. • Publisher: An entity responsible for making the resource available. • Contributor: An entity responsible for making contributions to the

resource.....

http://dublincore.org/documents/dces/

Structured Metadata• can also be structured hierarchically

Systema Naturae (1735)

Carl von Linné(1707-1787)

Structured Metadata• Classification Systems, as e.g. Dewey Decimal Classification

DDC 23 (2011)• 4 volumes• >4.000 pages• >45.000 classes• >96.000 registered terms

DDC 1 (1876)• 44 pages

10 Main DDC Classes000 Computer science, information & general works100 Philosophy & psychology200 Religion300 Social sciences400 Language500 Science600 Technology700 Arts & recreation800 Literature900 History & geography

Melvil Dewey(1851-1931)

http://www.oclc.org/dewey/

Unstructured Metadata• Text based metadata without a predefined structure, where the meaning

(semantics) is determined implicitely by the (natural language) content. • e.g. abstract/summary

Melville Louis Kossuth (Melvil) Dewey was an American librarian and educator, inventor of the Dewey Decimalsystem of library classification, and a founder of the Lake Placid Club.. Dewey was born in Adams Center, New York, the fifth and last child of Joel and Eliza Greene Dewey. He attended rural schools and determined early that his destiny was to be a reformer in educating the masses. At Amherst College he belonged to Delta Kappa Epsilon, earning a bachelor's degree in 1874 and a master's in 1877....

Authoritative vs. non-authoritative Metadata

Authoritative Metadataare generated by a reliable (authoritative) source, as e.g. • the author of the original information• a certified expert

Non-authoritative Metadataare created by an unreliable source, as e.g.

• the user• Social Tagging Systems

Collaborative Annotation -- Social Tagging

AuthorRessource

authoritativeMetadata

non-authoritativeMetadata

breakfast

to buy © E.C. Publications, Inc.

Collaborative Annotation -- Folksonomies

http://www.wordle.net/

Semantic Metadata• can be structured or semi-structured metadata• the semantics of metadata is defined explicitely in a formal way (Ontologies) and

therefore machine readable (as well as machine understandable)

"An ontology is an explicit, formal specification of a shared conceptualization. The term is borrowed from philosophy, where an Ontology is a systematic account of Existence. For AI systems, what ‘exists’ is that which can be represented.“ (Thomas R. Gruber, 1993)

conceptualization: abstract model (domain, relevant terms, relations)explicit: semantics of all terms must be definedformal: machine understandableshared: consensus about ontology

Example for Semantic Metadata

publication

• titel

• keywords

• ...

properties

publication

• titel

• keywords

• ...

properties

publication

journal

• titel

• keywords

• ...

properties

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by1..n

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

has a1..n

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

femaleis a

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

femaleis a

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

femaleis a

entity

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

femaleis a

entity

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

femaleis a

entity

relation

publication

journal

publisherpublishes

• titel

• keywords

• ...

properties

Autorwrites

is written by

Personis a

address

• surname

• first name

• street...

properties

Springer Verlag

Harald Sack

Internetworkingis a

femaleis a

entity

relation

Semantic Metadata• enable the definition of formal Axioms

• e.g. „It is not possible that the publishing date is earlier than the birth date of the author of the publication.“

• enable deduction of new facts• e.g. „All men are mortal.“

„Socrates is a man.“ „Therefore Socrates is mortal.“

• semantic Metadata enable to make implicitely giveninformation explicite with the help of deduction andinference

Raffael: The School of Athens, 1510

Multimedia Metadata Description Languages• for time-based media

• annotatation of temporal media fragments

Multimedia Metadata Description Languages• for media with spatial extend

• annotatation of spatial media fragments

metadata

Multimedia Metadata• MPEG-7

The MPEG-7 standard, formerly named „Multimedia Content Description Interface“, provides a rich set of standard tools to describe multimedia content. Both human users and automatic systems that process audiovisual information are within the scope of MPEG-7.

• Components of the MPEG-7 Standard• MPEG-7 Systems• MPEG-7 Description Definition Language• MPEG-7 Visual• MPEG-7 Audio• MPEG-7 Multimedia Description Schemes MDS• MPEG-7 Reference Software• MPEG-7 Conformance• MPEG-7 Extraction and Use of Descriptions

MPEG-7 Description of a Video Segment <Mpeg7 xmlns="..."><Description xsi:type="ContentEntityType"> ... <Video> <TemporalDecomposition> <VideoSegment> <CreationInformation>...</CreationInformation> <TextAnnotation> <KeywordAnnotation> <Keyword>mouse</Keyword> </KeywordAnnotation> </TextAnnotation> <MediaTime> <MediaTimePoint>T00:05:05:0F25</MediaTimePoint> <MediaDuration>PT00H00M31S0N25F</MediaDuration> </MediaTime> </VideoSegment> </TemporalDecomposition> </Video> ... </Description></Mpeg7>

MPEG-7 Description of a Still Image

MPEG-7 and the Semantic Web• MDS Upper Layer represented in RDF(S)

(2001: Hunter, later with link to ABC upper ontology)• MDS fully represented in OWL-DL

(2004: Tsinaraki et al., DS-MIRF model)• MPEG-7 fully represented in OWL-DL

(2005: Garcia & Celma, Rhizomik model)• MDS and Visual Parts represented in OWL-DL

(2007: Arndt et al., COMM model, re-engineering of MPEG-7 with DOLCE design patterns)

Example: Tagging with an MPEG-7 Ontology

• Localize a region → Draw a bounding box

• Annotate the content → Interpret the content → Tag ,Astronaut‘

:Reg1 foaf:depicts dbpedia:Astronaut

Example: Tagging with an MPEG-7 Ontology

mpeg7:image

mpeg7:depicts

Man on the Moon

mpeg7:spatial_decomposition Reg1

mpeg7:StillRegion

rdf:type

mpeg7:depicts

dbpedia:Astronaut

mpeg7:SpatialMask

mpeg7:polygon

mpeg7:Coords

Media Fragment Identification• Multimedia data has temporal and spatial dimension• pinpoint access on media fragments (on the web) with media fragment

identifiers• (W3C Media Fragments URI 1.0, Juli 2009, Working Draft)• simple examples

• requires different handling of media data by http client-server transactions

http://www.example.com/example.ogg#track=‘audio‘

http://www.example.com/example.ogg#track=‘audio‘&t=10s,20s

http://www.example.com/example.ogg#track=‘video‘&xywh=160,120,320,240

How do we find something in a Multimedia Archive?

How does Google find a video?

How do you find something in an audiovisual archive?

Step 1: Digitalization of analogue data

Step 2: Annotation with (textbased) metadata

• Manual annotation of AV-content with descriptive metadata

...can this also be achieved in an automated way?

Automated AV-Media Analysis

automated content-based analysis is•difficult (error prone) and•complex

Genre Analysis

Face Detection

Genre Analysis

Face Detection

overlay text

Genre Analysis

Face Detection

overlay text

Logo Detection

Genre Analysis

Face Detection

overlay text

Logo Detection

Genre Analysis

scenetext

Face Detection

overlay text

Logo Detection

Genre Analysis

scenetext{

Audio-Mining

structuralanalysis transcription speaker

identification

• Result: Video segments with time-based metadata annotations

• Metadata consist of combined low level / high level feature descriptors• Metadata serve as a basis for traditional and semantic retrieval

Metadata Extractiontime

e.g., person xylocation yzevent abc

e.g., bibliographical data,geographical data,encyclopedic data, ..

Video Analysis /Metadata Extraction

Entity Recognition/ Mapping

Semantic Multimedia Analysis

Some Examples of Automated Video Analysis

• Structural Analysis• Intelligent Character Recognition (ICR)

• Character/Logo Detection• Character Filtering• Character Recognition

• Audio Analysis • Speaker Detection • Automated Speech Recognition (ASR)

• Genre Analysis / Categorization•graphic / real• indoor / outdoor•day / night•...

• Face/Body Detection, Tracking & Clustering

• Automated subdivision of AV media data by structural segmentation• Subdivision of data streams in contentual coherent segments

Structural Analysis

scenes

Structural Analysis

scenes

Structural Analysis

scenes

subhots

Structural Analysis

scenes

subhots

frames

Structural Analysis

scenes

subhots

frames

Structural Analysis

• Shot Boundary Detection

• Identification of• Hard Cuts• Drop Outs• Soft Cuts, as e.g., Dissolve, Wipe, Cross-Fade, etc.

Analytical Shot Boundary Detection• Analysis of Luminance/Chrominance Histograms• Analysis of Edge Distribution• Analysis of Motion Vectors

Machine Learning• Classification of Hard/Soft Cuts based on Image Features• K-Nearest Neighbor• Random Forrest • Support Vector Machines

Histogram Difference Analysis

Motion Vector Analysis

Structural Analysis

• Identification of• Hard Cuts

91930 91931 91932919299192891927

Feature Analysis• Luminance Histogram Difference• Chrominance Histogram Difference• Edge Distribution

Structural Analysis

• Identification of• Hard Cuts• Drop Outs

Drop Out

Histogram/Chrominance Difference Analysis

Structural Analysis

Fade Out

Fade In

Structural Analysis

Analytical Shot Boundary Detection• Analysis of Luminance/Chrominance Histograms• Analysis of Edge Distribution• Analysis of Motion Vectors

Machine Learning• Classification of Hard/Soft Cuts based on Image Features• K-Nearest Neighbor• Random Forrest • Support Vector Machines

Histogram Difference Analysis

Motion Vector Analysis

Structural Analysis

Face-DetectionFace ClusteringFace Tracking

Character DetectionCharacter Recognition

Logo-Detection

Genre Detection

Intelligent Character Recognition• Preprocessing• Character Identification• Text Preprocessing

• Text Filtering• Adaption of script geometry (Deskew)• Image quality enhancement

• Optical Character Recognition (OCR)• Standard OCR software (OCRopus)

• Postprocessing• Lexical analysis • Statistical / context based filtering Ermittlungen nach

Bombenfunden

• Preprocessing• Character Identification

Filtering• Local Binary Patterns (LBP)• Histogram of Oriented Gradients

Intelligent Character Recognition

Intelligent Character Recognition• Preprocessing

• Character Identification• Text Preprocessing

• Text Filtering• Adaption of script geometry (Deskew)• Image quality enhancement

• Optical Character Recognition (OCR)• Standard OCR software (OCRopus)

• Postprocessing• Lexical analysis • Statistical / context based filtering Ermittlungen nach

Bombenfunden

Automatisierte Audio- und Videoanalyse

Original Image Bounding Box

Advanced Image Enhancement

Standard OCR (OCRopus)

Context-based Spell Correction

timeVideo Analysis /Metadata Extraction

Named Entity Recognition

Main Problem in NER: Ambiguity of Terms

jaguar

Example: „Jaguar“ in different contexts

rainforest

jaguar

rainforest

Steve McQueen

jaguar

rainforest

Steve McQueen

jaguar

rainforest

Steve McQueen

jaguar

Context matters!

Named Entity Recognition• Mapping keyterms (text) to semantic entities

• Context Analysis and Disambiguation

JaguarKeyterm / User Tag

Jaguar (Car)

Jaguar (Cat)

Jaguar (OS)

Jaguar (Aircraft)

Semantic Entities

RDF graph to find relations between entities co-occurringin a text maintaining the hypothesis that disambiguationof co-occurring elements in a text can be obtained byfinding connected elements in an RDF graph [7]. In orderto regard the special compilation of non-textual data, staticand user-genrated metadata in audio-visual content our novelapproach combines the use of semantic technologies andLinked Data with linguistic methods.

III. METHOD

According to a study about structure and characteristicsof folksonomy tags [8] an average of 83% of user-generatedtags are single terms. Also, an average of 82% of thereviewed tags are nouns. Based on these study results, weignore tag practices, such as camel case (”barackObama”)and treat tags as subjects or categories describing a resource.As a tag could also be part of a group of nouns representingan entity or a name (”flying machine”,”albert einstein”) thetags stored as single words without any given order have tobe combined in term groups of two or more terms to findall appropriate entities. Hence, every tag or group of tagswithin a given context may represent a distinct entity. Theterm combination process and subsequent mapping of termsand term groups to entities are described in sect. III-B.

To disambiguate ambiguous terms we combine two meth-ods: a co-occurences analysis of the terms in the context inWikipedia articles and an analysis of the page link graph ofthe Wikipedia articles of entity candidates. The scores forboth analysis steps are calculated to a total score.

A. Context Definition

Metadata exists in a certain context and has to be inter-preted according to this context. For tags of audio-visualcontent we identified two dimensions:

• temporal dimension• user-centered dimensionIn the temporal dimension a context can be defined as the

entire video, a segment or a single timestamp in the video.The user-centered dimension classifies a context by howmany users created the concerning metadata - only tags by acertain user or all tags regardless of which user. Fig. 1 showsthe combinations of the two dimensions of contexts formetadata in audio-visual content the interpretation regardingthe significance of a context.

Audio-visual content also provides the opportunity tosupply spatial information. Thus, tags in the same regionof a video frame are considered as related to each other.In the current approach we did not consider this contextdimension.

To describe our approach we use a sample context of ourtest set (see sect. IV). This sample context is composed oftags by only one user at a certain timestamp in the video.The video containing this sample context is a presentation

Figure 1. Dimensions of context definition in audio-visual content

by Dr. Garik Israelian at the TED conference3 entitled ”Howspectroscopy could reveal alien life”4. Our sample contextconsists of the tags ”hubble”, ”spitzer”, ”carbon”, ”dioxide”,”methan”, ”co2”, and ”water”.

B. Preprocessing

Term Combination: Our combination algorithm takesall tags of a specified spatio-temporal context (at a certaintimestamp/in a certain segment of a video, of a singleURL/image and generates every possible combination of atmost three terms of the context in every possible order. Inthat way we make sure to rectify groups of single termsthat belong together. We chose to generate combinationsof three words to make sure to also hit named entitiesconsisting of more than two words, such as ”public keycryptography” or ”alberto santos dumont”. About 90% ofthe DBpedia [9] labels consist of at most three words, butless than 5% consist of 4 words. Due to these numbersand performance issues we decided to limit the number ofterms to be combined to three. Subsequently in this paperby terms we will refer to single terms as well as generatedterm groups. The number c of combinations is calcultaed byc =

�jk=1

n!(n�k)! .

For our sample context containing 7 tags and at most3 terms in a combination (j = 3), 259 combinations aregenerated.

Term Mapping: The terms then have to be mapped tosemantic entities. For our approach we use entities of theLinked Open Data Cloud [10], in particular of the DBpedia,version 3.5.1.

DBpedia provides labels for the identification of distinctentities in 92 languages. We use English and German aswell as Finnish labels, as we noticed that neither English northe German labels contain important acronyms as labels, butthe Finnish language version does. As tagging users prefer tokeep it simple and short[2], resources dealing with ”DomainName System” would rather be tagged with ”DNS” than”Domain Name System”.

After simple string matching of the terms of the contextto DBpedia URIs, the URIs are revised for redirects and

3http://www.ted.com4http://yovisto.com/play/14415

Context Analysis and DisambiguationWhat defines a Context in AV-Data?

• Temporal Coherence • Spatial Coherence• Provenance

III. METHOD

B. Preprocessing

�jk=1

n!(n�k)! .

Spatial Dimension

III. METHOD

B. Preprocessing

�jk=1

n!(n�k)! .

Temporal Dimension

Spatial Dimension

III. METHOD

B. Preprocessing

�jk=1

n!(n�k)! .

User-centered Dimension

Temporal Dimension

Spatial Dimension

Preprocessing

Term Combination

Term Mapping

Entity Candidate Disambiguation

Co-Occurence Analysis

Semantic Graph Analysis

Score Calculation

1956 Stevejaguar

McQueenrim wheel

Steve McQueen ../resource/Steve_McQueen

jaguar ../resource/Jaguar_Cars

wheel rim ../resource/rim_(wheel)

1956 ../resource/1956

NER Custom Workflow

Preprocessing

Term Combination

Term Mapping

Entity Candidate Disambiguation

Co-Occurence Analysis

Score Calculation

1956 Stevejaguar

McQueenrim wheel

Steve McQueen ../resource/Steve_McQueen

jaguar ../resource/Jaguar_Cars

wheel rim ../resource/rim_(wheel)

1956 ../resource/1956

NER Custom Workflow

only if there is no spatialinformation for compositeterms available

Semantic Multimedia AnalysisNamed Entity Recognition Workflow

Term Combination

1956Steve

wheeljaguar

McQueen

Term Combination

1956Stevewheel jaguar

McQueenrim

Assigning Entity Candidates

1956Stevewheel jaguar

McQueenrim

7 entity candidates

2 entity candidates

36 entity candidates

1 entity candidate

Cooccurrence Analysis

„jaguar“http://dbpedia.org/resource/Jaguar_(Cats)

1956 wheel rimsteve mcqueen

context tags:

score: 0.00

„jaguar“http://dbpedia.org/resource/Jaguar_Cars

context tags:

score: 0.87

jaguarKeyterm / User Tag

LOD Cloud

1956 Stevejaguar

McQueenrim wheel

context

Jaguar (Car)Steve McQueen

Jaguar (Cat)Jaguar (OS)

Searching is not always just searching...

Searching is not always just searching

a simple example:

I‘m looking for a book by Earnest Hemingway with the title ,For Whom the Bell Tolls‘ in the first German edition...“

Wem die Stunde schlägt. - Ernest H E M I N G W A Y. (Stockholm usw., Bermann-Fischer Verlag, 1941) 560 S. 8“

II 1, 2506, 34548

Searching is not always just searching

...but what if...

I really liked the book ,For Whom the Bell Tolls‘ but I have no idea what I should read next....

...but what if...

I really liked the book ,For Whom the Bell Tolls‘ but I have no idea what I should read next....

Exploratory Search• What, if the user does not know, which query string to use?

• What, if the user is looking for complex answers ?

• What, if the user does not know the domain he/she is looking for?• What, if the user wants to know all(!) about a specific topic?

• ...,Browsing‘ instead of ,Searching‘• ...to find something by chance• ...serendipitous findings• ...to get an overview• ...enable content based navigation

How to implement an exploratory search?

Video Analysis /Metadata Extraction

Semantic Video Search

Data is a precious thing and will last longer than the systems themselves. (Tim Berners-Lee) http://linkeddata.org/

The Web of Data - The Semantic Web

http://dbpedia.org/

dbpedia:For_Whom_the_Bell_Tolls

What facts for dbpedia:For_Whom_the_Bell_Tollsare relevant?

http://dbpedia.org/page/For_Whom_the_Bell_Tolls

DBPedia - the Semantic Wikipedia

...use heuristics

Exploratory Search

dbpedia-owl:author

dbpedia:Ernest_Hemingwaydbpedia:For_Whom_the_Bell_Tolls

Exploratory Search

dbpedia-owl:author

Exploratory Search

dbpedia-owl:author

Exploratory Search

dbpedia-owl:author

Exploratory Search

dbpedia-owl:author

dbpedia:Raymond_Carver

dbpedia-

owl:influenced_by

Exploratory Search

dbpedia-owl:author

dbpedia-

owl:influenced_by

dbpedia:Jack_Kerouac

Exploratory Search

dbpedia-owl:author

dbpedia-

owl:influenced_by

dbpedia:Jack_Kerouac

dbpedia-owl:influenced_by

dbpedia:Jerome_D._Salinger

Exploratory Search

dbpedia:Jack_Kerouac dbpedia:Raymond_Carverdbpedia:Jerome_D._Salinger

Exploratory Search

dbpedia-owl:notableWork

Exploratory Search

dbpedia-owl:notableWork dbpedia-owl:notableWork

Exploratory Search

dbpedia-owl:notableWork dbpedia-owl:notableWork dbpedia-owl:notableWork

Exploratory Search

...and how does an exploratory search look like?

http://mediaglobe.yovisto.com:8080

Semantische SuchtechnologienExplorative Suche in audiovisuellen Daten

J. Waitelonis, H. Sack, Z. Kramer, J. Hercher:Semantically Enabled Exploratory Video Search, in Proc. of Semantic Search Workshop (SemSearch10) at the 19th Int. World Wide Web Conference (WWW2010), 26-30 April 2010, Raleigh, NC, USA, 2010.

Harald Sack, Hasso-Plattner-Institute for IT-Systems Engineering, Indian Summer School on Linked Data, Leipzig, 12-18. Sep. 2011http://bit.ly/SeMEX

Contact:Dr. Harald SackHasso-Plattner-Institut für SoftwaresystemtechnikUniversität PotsdamProf.-Dr.-Helmert-Str. 2-3D-14482 Potsdam

Homepage:http://www.hpi.uni-potsdam.de/meinel/team/sack.html http://www.yovisto.com/Blog: http://moresemantic.blogspot.com/E-Mail: harald.sack@hpi.uni-potsdam.de joerg.waitelonis@hpi.uni-potsdam.deTwitter: lysander07 / biblionomicon / yovisto

Thank you for

your Attention!

ISSLOD2011 - Semantic Multimedia

linked dataleipzig

harald sackhassoplattner

imagesharald sack

semanticsmediaharald

image textharald sack

2011overview1 multimedia

semantics2 multimedia

kind of informative

Technology

Multimedia Databases - Universitas...

RUSHES—an annotation and retrieval engine for multimedia.....

Content-based search and browsing in semantic multimedia...

Semantic Multimedia Database Information Retrieval

Semantic Building Information Model and Multimedia for ...

Virtual Campfire - Collaborative Multimedia Semantization...

A unified multimedia and semantic perspective for data...

01 Semantic Multimedia - Kickoff Sommersemester 2012

Einführungsvortrag 'Semantic Multimedia', Seminar SS2010,.....

Semantic adaptation of multimedia documents

02 Semantic Multimedia - Einfuehrungs-Workshop SS2012

Context-Based Semantic Tagging of Multimedia Data ·...

A Semantic Multimedia Web: Create, Annotate, Present and...

A SEMANTIC-BASED MIDDLEWARE FOR MULTIMEDIA COLLABORATIVE...

Bridging the Semantic Gap in Multimedia Information...

Semantic-Driven Multimedia Retrieval with the MPEG Query ...