MULTIMEDIA DATABASES AND MPEG7 Rahmi Volkan Başar Department of Computer Engineering METU May, 2013
May 22, 2015
MULTIMEDIA DATABASESAND MPEG7
Rahmi Volkan Başar
Department of Computer Engineering
METU
May, 2013
Multimedia Databases
• Introduction
• Capabilities of DB Types
• Search on MMDB
• Multimedia Content Description
• Research Fields
Multimedia Data
• Text: using a standard language (SGML, HTML)
• Graphics: encoded in CGM, postscript
• Images: bitmap, JPEG, MPEG
• Video: sequenced image data at specified rates
• Audio: recordings in a string of bits in digitized form
Database vs Multimedia Database
• Databases– well structured data organization
– efficient storage of large amounts of data
– querying
– transactional support for concurrent users
– numbers, strings
• Multimedia Databases– large content
– different structures
– not easily searched/queried
Use Cases
• Repositories: central location for data maintained by DBMS, organized in storage levels
• Presentations: delivery of audio and video data, temporarily stored, ‘VCR-like functionality’
• Collaborative: complex design, analyzing data
Capabilities
• Relational Databases
– Atomic / Tables
– Data relation – Common Foreign Keys
– Record: Content – No meta information
– A predefined set of domains for columns
• Hard to extend
• BLOB data type exist
Capabilities
• Object Oriented Databases
– Schema is “Class”
– All data is “Object”
– References
– New data types
• Easy. New class is a new data type.– Appropriate for multimedia data
Capabilities
• Object Relational Databases
– In addition to RDBMS
• Object references
• New types– Multimedia
– MMDBMS
• Extensible ORDBMSs
Search
• Collection of data. How to search?
– Any standards?
– Workarounds?
• Search: Retrieve similar images…
– Fast, Correct
• Content-based
– New techniques?
Search
• Content Based Retrieval Facilities
– Supported by MMDBMS
• Organize and Manage accordingly
– Compare based on a number of features
• Shape/Color/Texture
• Meta-Data?
– Always.
Content Based Retrieval
• Accurate representation of the multimedia objects in the database
– For accuracy and efficiency
– Combination: Different features
• Similarity Search
– High-dimensional feature vectors
• Special multi-dimensional indexing structures
• Dimension reduction methods.
Multimedia Content Description Standard: MPEG-7
• Influential XML based multimedia meta-data standard• Description of the storage media:
– Format, Image Size, Audio Quality, Video Frames etc.
• Creation and production information:– Creation date and location, title, genre, etc.
• Content semantic description:– Events, concepts, objects, etc.
• Content structural description:– Shot and key frames with color, texture and motion
features, etc.
• Metadata about the description:– Author, version, creation date, etc.
MPEG-7
• Expression of multimedia data
• Missing: Search for Implicit Data
– The meaning of the structure: Not expressed
– Ex. A video: length, format, name, dates etc.
• Gender: Documentary, Interview, Movie
• Theme: Science, Sports, Horror
• No consideration on search engines
MPEG-7
• Search:
– XPath, XQuery
– Semantic Views Query Language
Simple MPEG7 Example<Mpeg7>
<Description xsi:type="SemanticDescriptionType">
<Semantics>
<Label>
<Name> Car </Name>
</Label>
<Definition>
<FreeTextAnnotation>
Four wheel motorized vehicle
</FreeTextAnnotation>
</Definition>
<MediaOccurrence>
<MediaLocator>
<MediaUri> image.jpg </MediaUri>
</MediaLocator>
</MediaOccurrence>
</Semantics>
</Description>
</Mpeg7>
MPEG7 Details
• Standardizes 3 parts:
– Description tools
• Descriptors (D)
• Description Schemes (DS).
– Description Definition Language (DDL)
• To specify these schemes
– System tools
MPEG7 Details
• Descriptors (D)– Representation of a feature
• Syntactic and Semantic
– Low-level audio or visual features • Color, motion, texture etc
– Audiovisual content • Location, time etc
• Objects can be described– Several descriptors.
MPEG7 Details
• Description Schemes (DS) describe
– Specification of the relations
• Between Descriptors
• Between Description Schemes
– Relations can be structural and semantics
– High-level audiovisual (AV) features
• Regions, segments, events etc
MPEG7 Details
• Description Definition Language
– Based on XML
• Defines the structural relations between descriptors– Creation and modification of description schemes
– Creation of new descriptors.
MPEG7 Details
• System Tools
– Deal with Descriptor management
• Binarization
• Synchronization
• Transport
• Storage
MPEG7 Details - Overview
MPEG7 Details
• Next Slide
– Description of a Video Segment
MPEG7 Details
• How to extract semantics?
– i.e. Intelligent Information Retrieval
– Drawback of the standard
– Ontology help required:
• Domain Specific Ontology (Football, Location)
• Automatically extract information
• Use for a better search result
Research Fields
• Design: still in research
• Queries: techniques need to be modified
• Rest:
– Modeling: complex objects, wide range of types
– Storage: representation, compression, buffering during I/O, mapping
– Performance: physical limitations, parallel processing
• Thank you!
• Questions?
References
• Wikipedia: Various Pages• Computer Science and Engineering Department
Resources:– University of Notre Dame– Northumbria University– Carnegie Mellon University– Boston College– Simon Fraser University– Georgia Institute of Technology
• Interview with A. Anil Sinaci