Database Systems and WWW Applications Digital Libraries ... · PDF file1 30.10.2002 INF 312, Ifi, UiO, høst2002 1 Database Systems and WWW Applications Digital Libraries Multimedia

1

30.10.2002 INF 312, Ifi, UiO, høst2002 1

Database Systems and WWW Applications

Digital Libraries

Multimedia Database Systems

Earl F. Ecklund, Jr.

30 October 2002

30.10.2002 INF 312, Ifi, UiO, høst2002 2

Contents

• Database Systems and WWW Applications– Internet DB Architecture– Internet Applications

• Digital Libraries

• Multimedia Database Systems

2

30.10.2002 INF 312, Ifi, UiO, høst2002 3

Internet Application Architecture: Today

Application messages

Browser Browser

Physical Middle Tier

Data Sources

Client Tier

ORDBMS

WEB/APP Server

Middle-TierApplication

Data Integration,Storage, Query,Management

OtherDataSources

Gateways

OLE/DBData source

authoringtools, etc.

HTTPHTTP

Remote messages

Nori, A., Databases in Internet Applications: Case Studies, in: Postmodern DBS, UC Berkeley, Spring 1999

30.10.2002 INF 312, Ifi, UiO, høst2002 4

Internet Applications• Entertainment

– Games, Music, Films, Multi-person chat• Public information

– Maps, Tax return helper• Advertisement

– Interactive catalogues for products and services• Medicine

– Diagnosis, Consultation, Remote surgery• Education

– Learning-on-demand (for a degree),virtual museums, tour remote spaces

• Engineering– Collaborative design, remote parallel simulation services

• Publishing– Submit, Review, Proof-editing (text and graphics)

• Tele-communication– Conferencing

• ...

3

30.10.2002 INF 312, Ifi, UiO, høst2002 5

Contents

• Database Systems and WWW Applications

• Digital Libraries– Definitions

– Underlying concepts

– Digital Libraries Initiative

– Digital Libraries (examples)


30.10.2002 INF 312, Ifi, UiO, høst2002 6

DefinitionsIn the Stanford Digital Library project, we view long-term digital library systems as collections of widely distributed, autonomously maintained services. Of course, a digital library system must include services that allow users to search over collections of information objects. Examples of searchable collections include traditionallibrary collections, digital images, e-mail archives, video, on-line books, and scientific article citation catalogs (containing onlymetadata about the articles, not the articles themselves).

While searching services are valuable, they are not the only kind of service in the digital library of the future. Remotely usable information processing facilities are also important digital library services. These services provide support for activities such as document summarization, indexing, collaborative annotation, format conversion, bibliography maintenance, and copyright clearance.

The Stanford Digital Library Technologies Project

4

30.10.2002 INF 312, Ifi, UiO, høst2002 7

Definitions

Digital libraries are organizations that provide the resources, including the specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities.

The Digital Library Federation (DLF)

Note:CS researchers tend to focus on digital libraries as content collected on behalf of user communities, while librarians focus on digital libraries as institutions or services.

30.10.2002 INF 312, Ifi, UiO, høst2002 8

Definitions1. Digital libraries are a set of electronic resources and associated technical capabilities for creating, searching and using information, including the full life cycle of the resources. In this sense they are an extension and enhancement of information storage and retrieval systems that manipulate digital data in any medium (text, images, sounds; static or dynamic images) and exist in distributed networks. The content of digital libraries includes data, metadata that describes various aspects of the data (e.g., representation, creator, owner, reproduction rights), and metadata that consists of links or relationships to other data or metadata, whether internal or external to the digital library.

2. Digital libraries are constructed -collected and organized- by [and for] a community of users,and their functional capabilities support the information needs and uses of that community. They are a component of communities in which individuals and groups interact with each other, using data, information, and knowledge resources and systems. In this sense they are an extension, enhancement, and integration of a variety of information institutions as physical places where resources are selected, collected, organized, preserved and accessed in support of a user community. These information institutions include, among others, libraries, museums, archives and schools, but digital libraries also extend and serve other community settings including classrooms, offices, laboratories, homes and public spaces)

Borgman et al. 1996

5

30.10.2002 INF 312, Ifi, UiO, høst2002 9

Contents







30.10.2002 INF 312, Ifi, UiO, høst2002 10

Notions

• Content – the items in the library collection• Annotation – information added to (associated with) an image• Subject matter – focus of a collection, topics used in classification• Catalog – database (card file) of bibliographic records• Classification – assigning call number, adding keywords

• Rights to use - permissions• License agreements – contractual right to use• Copyright• Watermark – a subliminal pixal pattern to identify a digital work• Copy detection – verifying copying, searching for copies

• Search (40% of search queries on the web are reported to be single words)

• Metasearchers (services that provide unified query interfaces to multiple search engines. Thus users have the illusion of a single combined document source. Three main tasks: choosing the best sources to evaluate a query; evaluating the query at these sources; merging the query results from these sources.)

6

30.10.2002 INF 312, Ifi, UiO, høst2002 11

File Formats– Image formats

• TIFF Tagged Image File Format• GIF Graphics Image File Format• JFIF JPEG File Format• SPIFF Still Picture Interchange File Format• PICT Macintosh Picture• TGA TrueVision Targa file (bit mapped images)• EPS Encapsulated PostScript• CGM Computer Graphics Metafile• PhotoCD (Kodak)

– Picture and video formats• JPEG Joint Photographic Expert Group• Motion JPEG• MPEG Moving Pictures Expert Group

– Document formats• PostScript (Adobe)• PDF Portable Document Format (Adobe)

30.10.2002 INF 312, Ifi, UiO, høst2002 12

Compression

• Compression

– lossless • color 25%-50%-67%

• B/W 50%-90%;

– lossy up to 95%

• Compression formats

– CCITT Group III or Group IV

– JPEG

– JBIG An international compression standard– LZW.Subsampling (lossy)

• Compression schemes

– LZW Lempel-Ziv-Welch (lossless)

– MPEG Group of Pictures: IBBBPBBBPB…I– QuickTime (Apple)

7

30.10.2002 INF 312, Ifi, UiO, høst2002 13

Images in the Digital LibraryMost image-database systems store descriptive information about the images in a traditional text-based information retrieval system. An additional field containing the filename of the image is added to each record in this text-based system to link it to an image file. Images are selected by querying the text-based system. When the query is specific enough, the user requests a selected image (or a set of images) to view. Extensions to user-interface software look up the filename field(s) in the text record(s) and display the image(s), often in a new window. Each system handles the text/image relationship in its own way, and standards need to be developed to enable the interchange of image files among systems

Much research remains in the field of image databases, particularly with respect to image-quality needs. Further studies need to stratify types of collections, as well as users and uses of those collections, relating each to a series of required image qualities.

Howard Besser and Jennifer TrantIntroduction to Imaging: Issues in Constructing an Image Databasehttp://www.getty.edu/research/institute/standards/introimages/

30.10.2002 INF 312, Ifi, UiO, høst2002 14

Metadata and Cataloging

• Metadata Data about data (structure and access)

• OPAC On-line Public Access Catalog

• Content description structured vocabulariesdata-structure guidelines

• MARC MAchine Readable Cataloging

• Dublin Core a classification scheme

• Indexing abbreviation for works

organized for reference• MPEG7 metadata about MPEG data

8

30.10.2002 INF 312, Ifi, UiO, høst2002 15

The Dublin Core metadata standard• Document A work that is mostly textual in nature, but may include images, maps, tables or other formats.

– Abstract

– Article (a short published or pre-published piece) o Preprint Refereed Unrefereed

– Bibliography

– Correspondenceo Email Letter Listserv Newsgroup Postcard

– Essay

– Factsheet

– Manuscript

– Minutes

– Monograph (books and book-like objects) o Autobiography Biography Collection Manual Novel Play

– Pamphlet (and booklets) – Paper (conference paper or other short work) – Poem

– Serial (items published on a serial basis) o Journal Magazine Newsletter Newspaper Proceedings

– Story

– TechReport

– Thesiso Doctoral Masters

– Webpageo Organizational Personal

30.10.2002 INF 312, Ifi, UiO, høst2002 16

The Dublin Core (continued)• Image

– Interactiveo 3DObject VR (Virtual Reality)

– Movingo Animation Film

– Photographo Aerial Landscape Portrait

– Graphic

• Sound All sound formats, including MIDI files. – Ambient Effect Music Narration Speech

• Software Binary executables and source code. Both may be further subdivided with the name of the programming language used.

– Executableo Game

– Source

• Dataset Alphanumeric collections of data. – Bibliographic

– Cartographic

– Spectral

– Spreadsheet

– Statistical

o Miscellaneous Work of another or undetermined type. This is the default type if not explicitly stated.

9

30.10.2002 INF 312, Ifi, UiO, høst2002 17

Information Retrieval - Text

• Basic searching techniques:

– Linear search (can do regular expressions)– Inverted files– Hash tables

– Signature files (compressed linear search)

• Linear search requires no extra space, linear complexity in size, no preprocessing

• Inverted files cannot search for arbitrary expressions, (usually) must start at the beginnings of words, building index takes n log n time length of file (n words). Index overhead ranges from 25% to 200%.

• Hash coding is sensitive to the exact spelling of the word, and tends to scatter words nearly spelled the same; requires preprocessing and has slight storage overhead.

30.10.2002 INF 312, Ifi, UiO, høst2002 18

Information Retrieval - Images

• Indexes made for other purposes

– Citations

– Reviews– …

• Thumbnails

• Exploit layout formats (e.g., newspaper columns)

• Image alignment

– Centering

– Feature analysis, normalize rotation and X-Y orientation)

• Complementation: an image of a red rose will not normally have the keyword "red". Thus image features and associated words can complement and even disambuguate each other.

10

30.10.2002 INF 312, Ifi, UiO, høst2002 19

Contents







30.10.2002 INF 312, Ifi, UiO, høst2002 20

US Digital Libraries Initiative (Phase I)

• University of California, Berkeley – Work-centered digital information services

• University of California, Santa Barbara – Spatially referenced map information

• Carnegie Mellon University – Full-content search and retrieval of video

• University of Illinois at Urbana-Champaign – Federating repositories of scientific literature

• University of Michigan – Intelligent agents for information location

• Stanford University – Interoperation mechanisms among heterogeneous services

Shared vision: an entire Net of distributed repositories, where objects of any type can be searched within and across indexed collections

11

30.10.2002 INF 312, Ifi, UiO, høst2002 21

US Digital Libraries Initiative (Phase I)

• University of California, Berkeley

– http://elib.cs.berkeley.edu/

– also see http://sunsite.berkeley.edu/• University of California, Santa Barbara

– http://www.alexandria.ucsb.edu/

• Carnegie Mellon University

– http://www.informedia.cs.cmu.edu/

• University of Illinois at Urbana-Champaign – http://dli.grainger.uiuc.edu/idli/idli.htm

• University of Michigan

– http://www.si.umich.edu/UMDL/

• Stanford University

– http://www-diglib.stanford.edu/

30.10.2002 INF 312, Ifi, UiO, høst2002 22

Examples of Technology Impact

• University of California, Berkeley

– Multivalent Documents

– Robust Hyperlinks and Robust Locations– TilePics

– …

• Carnegie Mellon University

– Informedia Digital Video Library System

– …• Stanford University

– Archival Digital Libraries Repositories

– Large Scale Copy Detection

– Google Search Engine

– …• …

12

30.10.2002 INF 312, Ifi, UiO, høst2002 23

Multivalent Documents

• Multivalent Annotations– Stored separately from the document they annotate– Appear in situ – as if part of the content of the document

• Hyperlinks• Highlights• Notes• Copy editor markup (executable)

– Three classes of behavior• Spans (anchored to points or intervals)

– E.g., Hyperlinks, Rollovers

• Lenses (anchored to geometric regions)– E.g., Bit Magnify, Optical Character Recognition

• Structures (within the document tree)– E.g., Extracting references in bibsys format

– Combining Annotations• Notemarks

– E.g., outlining, man pages

30.10.2002 INF 312, Ifi, UiO, høst2002 24

Robust Hyperlinks and Robust Locations

• URLs can be made robust

– if a web page moves to another location anywhere on the web, you can find it.

• Even if that page has been edited.

• Robust Hyperlinks

– URLs are augmented with a five or so words to make a content-based lexical signature

– If the URL's address-based portion breaksFeed the signature into any web search engine to find the new site of the page.

13

30.10.2002 INF 312, Ifi, UiO, høst2002 25

TilePics

• A file format designed to store tiled data of arbitrary type in a hierarchical, indexed format in order to provide fast retrieval.

• a fixed sized header

• tile index data• an optional gap• contiguous tile data

• optional attribute data

• Encapsulate a large amount of related, static data in a single file. • A one or two-dimensional dataset• At multiple scales of resolution or abstraction. • Tileable, based on x,y coordinates for quick localized access

• Store data at multiple levels of resolution • in multiple layers of tiles• each layer relates to the next by the same scale factor

• Zoom by drawing just the relevant tiles at the next layer down

30.10.2002 INF 312, Ifi, UiO, høst2002 26

Informedia Digital Video Library System

• IDVLS attempts to automate cataloging by:

– Recognizing speech

– Understanding text and language– Segmenting text

– Recognizing text within imagery

– Segmenting video

– Analysing video structure

– Image matching based on perceived color– Region matching for content-based image retrieval

– Detecting video shot boundaries

14

30.10.2002 INF 312, Ifi, UiO, høst2002 27

Archival Digital Libraries Respositories

File System

InfoMonitor

usersusers

Archival Repository Web Server

30.10.2002 INF 312, Ifi, UiO, høst2002 28

Large Scale Copy Detection

• CDS: Content Detection System– content publishers register their valuable digital content– CDS crawls the web

• compares the web content to the registered content

– notifies the content owners of illegal copies.

• Key challenges– accuracy, in terms of high precision and recall, – scalability, in terms of coping with several terabytes of data

(or several tens of millions of web pages)– resiliency to “attacks”

• Two prototypes– SCAM (Stanford Copy Analysis Mechanism, for text)– FRAUD (Finding Replicas of AUDio)

15

30.10.2002 INF 312, Ifi, UiO, høst2002 29

Google Search Engine

• PageRank: A Citation Importance Ranking

– Number of backlinks (~ citations)• Approximation of importance

• Citation analysis literature– Citation indexes

• Extreme variation in importance

– Large database of links: propagation

• Idealized Model

B

C

A

1 2

B and C arebacklinks of A

l1,2 = 1

l2,1 = 0

N number of outgoing linksni = li,j on page i (includes multiple

j=1, i≠j links to the same page)

N WiWj = (li,j — ) PageRank of page j

i=1, i≠j ni

Σ

Σ

30.10.2002 INF 312, Ifi, UiO, høst2002 30

Papers on the Creation of Google

• The Anatomy of a Large-Scale Hypertextual Web Search Engine, by Sergey Brin, Lawrence Page

• Dynamic Data Mining: Exploring Large Rule Spaces by Sampling, by Sergey Brin, Lawrence Page

• Computing Iceberg Queries Efficiently, by Min Fang, Narayanan Shivakumar, Hector Garcia-Molina, Rajeev Motwani, and Jeffrey D. Ullman

• The PageRank Citation Ranking: Bringing Order to the Web, by Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd

• Extracting Patterns and Relations from the World Wide Web,by Sergey Brin

• Finding near-replicas of documents on the web, by Narayanan Shivakumar, Hector Garcia-Molina

• Efficient Crawling Through URL Ordering, by Junghoo Cho, Hector Garcia-Molina, Lawrence Page

16

30.10.2002 INF 312, Ifi, UiO, høst2002 31

Contents







30.10.2002 INF 312, Ifi, UiO, høst2002 32

ACM Portal: ACM Digital Library

Bibliographic information, abstracts, reviews, and the full-text for articles published in ACM periodicals and proceedings since its founding in 1947 are available in the library together with selected works published by affiliated organizations.

As of October 15, 2002, the Digital Library contains: – over 102,500 full-text articles from journals, magazines, and

conference proceedings.

– Tables of Contents with over 33,000 citations from articles published in journals and magazines from 1954 forward.

– Tables of contents with more than 69,000 citations from articlespublished in over 1100 volumes of conference proceedings since 1970.

17

30.10.2002 INF 312, Ifi, UiO, høst2002 33

ACM Digital Library

• The Digital Library presents all material associated with an article:– Bibliographic data

includes the title, author(s), publication, volume, issue, and page numbers of an article.

– Index termscompiled using article keywords and the ACM Computing Classification System.

– Abstracts available for most articles in the Digital Library.– Reviews from ACM Computing Reviews (Selected articles) – Full-text view or download complete articles.

Most articles are available in PDF -- some are available in other formats including HTML, postscript, and LaTeX.

– DOIWhen ACM submits a reference query and it is matched, a Universal Resource Name (URN) in the form of a Digital Object Identifier (DOI) is returned and inserted as an external link from ACM's site to the source for the material.

30.10.2002 INF 312, Ifi, UiO, høst2002 34

University of Oslo Digital Library Project

• “Post graduate theses in the digital library” (Hovedfagsoppgaver i digitalt bibliotek) is a pilot project where theses will be published in full text on the world wide web.

• A step in establishing a digital library where the University ofOslo shall keep electronic teaching materials and documents.

• A joint project between the USIT SGML group, the University of Oslo library, and other institutions, including the Institute ofInformatics

• Students are to use Microsoft Word and a template file providedby the project.

• Microsoft Word documents using the template styles can by automatically converted to HTML and SGML.

• http://www.digbib.uio.no/ (in norwegian)

18

30.10.2002 INF 312, Ifi, UiO, høst2002 35

Contents



• Multimedia Database Systems– Definitions

– Example Application

• MM QoS Requirements

– MMDBMS Requirements

– MMDBMS Concepts

30.10.2002 INF 312, Ifi, UiO, høst2002 36

Definitions

• Multimedia (MM); loosely: any system that can be used to present information in more than one form: text, graphics, still images, animation, sound, video, special computer-generated effects.

The system should have user-friendly interactive interfaces that helpthe communication of complexly structured data.

• MMDBSs: are the DBSs that manage MM data, facilitate MM for presentations, and use specific tools for the storage, management, and retrieval of MM data.

19

30.10.2002 INF 312, Ifi, UiO, høst2002 37

Data Flow for a Multimedia Network Server

Network

StorageMultimedia server

Buffers Buffers

Graphics/videohardware

Audiohardware

Client

30.10.2002 INF 312, Ifi, UiO, høst2002 38

Contents







– MMDBMS Concepts

20

30.10.2002 INF 312, Ifi, UiO, høst2002 39

Multimedia-Supported Learning of Practical Medical Procedures

• Provide realistic visualization of required practical skills• Proven to be pedagogically beneficial to view the multimedia lesson on

a procedure in a “learning on demand” setting before observing it in the clinic

• Lessons involve realistic multimedia elements (video and audio) recorded in Oslo hospitals, with expert commentary,

• Over 17,000 multimedia elements in OKSE-basen database. • Mostly on CD-ROM.• LoD over the Internet would enable

– Greater flexibility (time and location) for students– Other applications

• Paramedics review skills on demand in emergency situations• Doctors take courses in their office for lifelong learning

– Incremental release and revision of lessons or skill segments

2

30.10.2002 INF 312, Ifi, UiO, høst2002 40

Selective Multimedia Quality is Critical

• Quality of Service (QoS): – The collective effects of service performance which determine

the degree of satisfaction of a user of the service.– Performance, not operation (non-functional requirements,

independent of functional requirements)

• Video accuracy, for example, when draining the chest. – The video must accurately show location of arteries, ribs,

where the drain can safely be inserted to avoid arteries.• Audio fidelity, for example, when breathing is difficulty.

– The audio must be clear enough to differentiate between stridor, an obstruction of the large airways, and asthmatic breath sounds.

• Timing accuracy. – Some procedures should be viewed in near real time, possibly at

reduced video resolution and reduced audio fidelity.

• The critical quality focus may shift within a lesson. – The infrastructure should shift resources to the critical qualities

(and ignore others if necessary). 3

21

30.10.2002 INF 312, Ifi, UiO, høst2002 41

Use Case Model• A Use Case specifies a sequence of actions that the system can perform

and that yields an observable result of value to a particular actor.

– A “use case” is a story of how an actor achieves its goal using the system under design.

– The purpose of a use case is to define a piece of coherent behavior without revealing or dictating the internal structure of the system.

– A use case model structures a system’s use cases, representing its requirements and interface behavior to serve as input to its designers and implementers.

Record Element Recording Tech

Create Lecture Lecture Producer

Store Data

<<include>>

<<include>>

4

30.10.2002 INF 312, Ifi, UiO, høst2002 42

Use Case Model• A Use Case specifies a sequence of actions that the system can perform

and that yields an observable result of value to a particular actor.

– A “use case” is a story of how an actor achieves its goal using the system under design.

– The purpose of a use case is to define a piece of coherent behavior without revealing or dictating the internal structure of the system.

– A use case model structures a system’s use cases, representing its requirements and interface behavior to serve as input to its designers and implementers.

Record Element Recording Tech

Create Lecture Lecture Producer

Store Data

<<include>>

<<include>>

4

22

30.10.2002 INF 312, Ifi, UiO, høst2002 43

Main MSLoPP Use Cases

• Record multimedia elements

• Produce a lecture (a skills lesson, a presentation)• View a lecture

Use Case 3. View a lectureGoal: A medical student reviews a procedure over the internet, with acceptable QoS. Comments: The skills lab is linked to the LoD server by an 155 Mbyte/second ATM NetworkScenario: A student goes to the skills lab at the medical school and starts an internet browser.

(Alternatively, the student could connect to the internet from home.) The student logs in to the MSLoPP LoD server, then finds and selects the lesson on draining the chest. The system accesses the multimedia database for the selected lesson to determine what multimedia elements are to be included, what resources are required and what levels of QoS should be used to presentthe lesson. The system confirms that the student’s environment has adequate resources for playing the presentation. Then the system sets up the environment and starts playing the presentation. The student views the procedure, using pause, rewind, and play controls to review and replay key parts of the lesson, such as the location of the insertion point relative to the ribs and arteries. The student logs off from the LoD server, ending this use case.

Other Requirements: The LoD server must be able to detect the multimedia software available on theviewing station, and determine the QoS capabilities of the software and the viewing environment.

The student must be able to follow the focus to the emphasized elements of the presentation. Timing critical elements must be presented in near real time. Concurrent elements specified to be synchronized must be presented in synchrony.

30.10.2002 INF 312, Ifi, UiO, høst2002 44

Medical LoD Use Case Model

7

{ Notify of Event - Ask Question < 1 minute }

Paramedic

Conferee

{ Data Sequence strict in-order }

{ Time Delay < 10 msec }

Editor

Recording Tech

Set Synch Point

Producer

Remote Procedure

Doctor

View PresentationViewer

Teleconference

Lecture Producer

Edit Lecture

Record Stream<<extend>>

Setup BindingPlay Element

Create LectureStore Data

<<inc lude>>

<<include>>

<<inc lude>>

Oracle

Answer Question

<<include>>

Find Element

<<include>>

<<include>>

<<include>>

Play One Element

<<include>><<include>>

<<include>>


Find Lecture

<<include>>

Play Lecture

<<include>>

<<include>>

<<include>>

Ask Question

<<include>>

Notify of Event

<<include>>

<<include>>

View Answer

<<include>>

<<include>>

<<extend>>

View Lecture

<<include>>

<<include>>

<<extend>>

Student Produce Element

<<include>>

<<include>>

<<include>>

<<include>>

23

30.10.2002 INF 312, Ifi, UiO, høst2002 45

{ Notify of Event - Ask Question < 1 minute }

Paramedic

Conferee

{ Data Sequence strict in-order }

{ Time Delay < 10 msec }

Editor

Recording Tech

Set Synch Point

Producer

Remote Procedure

Doctor

View PresentationViewer

Teleconference

Lecture Producer

Edit Lecture

Record Stream<<extend>>

Setup BindingPlay Element

Create LectureStore Data

<<inc lude>>

<<include>>

<<inc lude>>

Oracle

Answer Question

<<include>>

Find Element

<<include>>

<<include>>

<<include>>

Play One Element


<<include>>


Find Lecture

<<include>>

Play Lecture

<<include>>

<<include>>

<<include>>

Ask Question

<<include>>

Notify of Event

<<include>>

<<include>>

View Answer

<<include>>

<<include>>

<<extend>>

View Lecture

<<include>>

<<include>>

<<extend>>

Student Produce Element

<<include>>

<<include>>

<<include>>

<<include>>

Medical LoD Use Case Model

7

30.10.2002 INF 312, Ifi, UiO, høst2002 46

Contents







– MMDBMS Concepts

24

30.10.2002 INF 312, Ifi, UiO, høst2002 47

Requirements for MMDBSs

• represent arbitrary data types and specification of programs thatinteract with arbitrary data sources;

• query and modify (update, insert, delete) MM data; includingretrieval of MM data via associative search within MM data (minimally, text);

• specify and execute abstract operations on MM data, e.g., play, fast forward, pause, and rewind one-dimensional data like audioor text; to display, expand, and compress two-dimensional data like bit-mapped images;

• deal with heterogeneous data sources in a uniform manner; thisincludes access to data in these sources and migration of data from one data source to another.

Ability to ...

30.10.2002 INF 312, Ifi, UiO, høst2002 48

Requirements - 2

• MM & object-oriented data modeling concepts;• management of several kinds of magnetic and optical storage

devices appropriate for MM data handling;

• uniform management of very large data volumes => management of tertiary storage and multi-level storagehierarchies;

• support for realtime data processing =>appropriate scheduling and resource allocation techniques;

• support for storage and processing parallelism (performancerequirements);

• support for distribution => appropriate distributed DBMS concepts.

MM data storage and retrieval:

25

30.10.2002 INF 312, Ifi, UiO, høst2002 49

Storage space requirements for uncompressed digital multimedia data (examples)

Media type Specifications Data rate per sec.

Voice-quality audio 1 channel, 8-bitsamples at 8 kHz

64 Kbits

MPEG-encoded audio Equiv. to CD quality 384 Kbits

CD-quality audio 2 channels, 16-bitsamples at 44.1 kHz

1.4 Mbits

MPEG2-encoded video 640x480 pixels/frame,24 bits/pixel

0.42 Mbytes

NTSC-quality video 640x480 pixels/frame,24 bits/pixel

27 Mbytes

HDTV-quality video 1280x720 pixels/frame,24 bits/pixel

81 Mbytes

30.10.2002 INF 312, Ifi, UiO, høst2002 50

Requirements - 3

• “soft” realtime transfer requirements• “hard” transaction deadlines

• synchronization between different data streams (data types)

• user interactions (synchronous and asynchronous)

Realtime and synchronization issues:

=> dependent on data distribution, storage devices, compression techniques for the various data types,buffer management techniques, scheduling algorithms,data placement techniques, and communication bandwidth

26

30.10.2002 INF 312, Ifi, UiO, høst2002 51

Contents







– MMDBMS Concepts

30.10.2002 INF 312, Ifi, UiO, høst2002 52

DBMS Concepts

• Data modeling: temporal object-oriented modeling and presenting (HCI) of multimedia data+ extra data types & operations

• Query processing and optimization: browsing, contentaddressing

• Storage management: optimization techniques

• Transaction management: realtime processing for readtransactions (presentations),write transactions (authoring) use a advanced transaction model(e.g., checkout-checkin with versioned data)

27

30.10.2002 INF 312, Ifi, UiO, høst2002 53

User Interface Design for MM Applications

• User interaction and user interfaces become much more complex if MM data is involved.

• State-of-the-art: buttons, text entry, scrollable areas, ...-> does not support interaction with continuous media

• New devices (e.g., cameras, microphones, loudspeakers, ...) have to be taken into account in addition to keyboard, mouse, monitor, and external devices (e.g., VCRs, ...) for input and output handling:

- simultaneous control of different devices- efficient handling of user interrupts- standardized interaction paradigms- support for pen + voice input- ...

30.10.2002 INF 312, Ifi, UiO, høst2002 54

Object-Oriented Data Modeling + ...

• text• graphic

• image

• audio

• speech

• video• generated media

Data types and operations for:

Temporal relationships:- Synchronization and realtime processing

Quality-of-Service:- to handle average delay, speed ratio, utilization,

jitter, skew, and reliability.

28

30.10.2002 INF 312, Ifi, UiO, høst2002 55

Required Data Model Concepts and Related Work

• Time independent data types

• Time dependent data types (continuous types)

• Temporal concepts: valid, transaction, and play time

• Temporal data models: TIGUKAT, T_Chimera, Mediadoc, SGML/HyTime, ...

• Multimedia data models: AMOS, SGML/HyTime, LMDM, ...

30.10.2002 INF 312, Ifi, UiO, høst2002 56

Concepts of TOOMM

Logical Data Model

Video 1

Video 2

Audio 1

Multimedia Objects

CPO1

Atomic Presentation Objects

Presentation Model

P_Video 13

P_Video 14

P_Video 15

P_Audio 11CompositePresentationObject

29

30.10.2002 INF 312, Ifi, UiO, høst2002 57

Example: Modeling a Video Object

Frame 0

Timestamp 0

Frame 1

Timestamp 1

Frame n

Timestamp n

TA 0

TA 1

TA n

Video 1

30.10.2002 INF 312, Ifi, UiO, høst2002 58

Type Hierarchy

MMDT

PTI_MMDT

PictureText Graphics

CGM

Video

PTD_MMDT

Stream

Audio Music Animation

LDU

Anim.

Component

Event

NoteSampleFrame

30

30.10.2002 INF 312, Ifi, UiO, høst2002 59

Type Hierarchy

MMDT

PTI_MMDT

PictureText Graphics

CGM

Video

PTD_MMDT

Stream

Audio Music Animation

LDU

Anim.

Component

Event

NoteSampleFrame

30.10.2002 INF 312, Ifi, UiO, høst2002 60

Play Time

Components of a stream multimedia object

TA 0

TS 0 LDU 0

TA 1

TS 1 LDU 1

TA n

TS n LDU n

Play Time

Components of a CGM multimedia object

TA 0

event 0 TS 0

TA 1

event 1 TS 1

TA n

event n TS n

31

30.10.2002 INF 312, Ifi, UiO, høst2002 61

EER Diagram

P_MMDT

P_Text P_Picture P_Graphics

P_PTI_MMDT

P_Music P_Anim.

P_CGM

P_Audio P_Video

P_Stream

P_PTD_MMDT

Temporal_reference

start stop1:11:1

1:11:11:10:1

MMDT1:1 0:M

Effect1:1 0:M

CPO

Parallel

Temporal_Relationships

Serial

0:M1:1

1:1

1:M

1

2

30.10.2002 INF 312, Ifi, UiO, høst2002 62

EER Diagram

P_MMDT

P_Text P_Picture P_Graphics

P_PTI_MMDT

P_Music P_Anim.

P_CGM

P_Audio P_Video

P_Stream

P_PTD_MMDT

Temporal_reference

start stop1:11:1

1:11:11:10:1

MMDT1:1 0:M

Effect1:1 0:M

CPO

Parallel

Temporal_Relationships

Serial

0:M1:1

1:1

1:M

1

2

32

30.10.2002 INF 312, Ifi, UiO, høst2002 63

Example: Using Temporal References

P_Video 1

P_Video 2

P_Text 3

Composite multimedia presentation

Recursive temporalreference list

Reference: Truedeviation: 0time_point: NA

Reference: Truedeviation: -5time_point: NA

Reference: Truedeviation: NAtime_point: 15

1

2

3

Multimediapresentation

objects

timeActual play time valueTemporal references

30.10.2002 INF 312, Ifi, UiO, høst2002 64

Building Multimedia Presentations

P_Video

Video

CPO

Actual play time valueTemporal references

time

P_start P_stop

Start Stop

33

30.10.2002 INF 312, Ifi, UiO, høst2002 65

Type: CPOName: Lecture_19_2_1998MTU_duration:1/44100

Type: P_AudioName: P_Audio 1Speed: 1Start: 0Stop: 31752000p_start.get_time_point()=0p_stop.get_time_point() = 31752000

Type: AudioName: PMC_Lecture_hour1_clip_1LDU_duration: 1/44100Duration: 31752000Content description:- (0, 4988, “Lecturer talks about files”)- (4989, 12134, “Lecturer talks about

directories”)

Type: P_VideoName: P_Video 1Speed: 1Start: 0Stop: 18000p_start.get_time_point()=0p_stop.get_time_point() = 31752000

Type: VideoName: PMC_Lecture_hour1_scene1LDU_duration: 1/25Duration: 1800Content description:- (0, 4988, “Lecturer talks about files”)- (4989, 12134, “Lecturer talks about

directories”)

Type: ParallelName: TR 1Temporal relationshiptype: EqualSkew tolerance: 80 ms

Type: P_HTMLName: P_HTML 1p_start.get_time_point() =3987233p_stop.get_time_point() = 7234443

Type: P_HTMLName: P_HTML 2p_start.get_time_point() =10234234p_stop.get_time_point() = 16230933

Type: P_Light_PenName: P_Light_Pen 1p_start.get_time_point() =4457111p_stop.get_time_point() = 6283324

Type: HTMLName: File System

Type: HTMLName: Directory Example

Type: Light_PenName: Drawing_objectsLDU_duration: 200Content description:- (0, 100, “Draw a bow in File System”)- (101, 200, “Draw a dot”)

ExampleCPO

30.10.2002 INF 312, Ifi, UiO, høst2002 66

Query Processing and Optimization

- Browsing: efficient location of data elements in very large amounts of data,exact-match (pattern-matching) queries (e.g., text)and similarity-based queries (e.g., images, ...)-> query refinement-> set-oriented and navigation-oriented browsing techniques

- Content addressing: efficient location of data with complex data types like images (difficult to access in realtime using pattern-recognition techniques) comprises: natural language understanding,speech processing,vision,and user modeling

34

30.10.2002 INF 312, Ifi, UiO, høst2002 67

Meta-Data Management

• Meta-data needed especially for continuous data to support retrieval

• Textual data describing contents of audio and video segments

• Content search mostly performed on meta-data• Problems:

– Modeling of meta-data

– Meta-data acquisition

– Association of meta-data to “real” data

30.10.2002 INF 312, Ifi, UiO, høst2002 68

Storage Management Issues

• addressing techniques

• access paths

• data placement techniques:clustering, partitioning, allocation

• system buffer management:paging, ...

• disk scheduling:sweeping, deadline-driven, …

35

30.10.2002 INF 312, Ifi, UiO, høst2002 69

Data Placement

• Clustering and partitioning:

– data striping and data interleaving

• Allocation:

– contiguous placement

– constrained placement

– log-structured placement

Controller

Sector 0Sector 1Sector 2



Logical sector 0

30.10.2002 INF 312, Ifi, UiO, høst2002 70

Disk Scheduling

• Traditional algorithms

– FIFO (first come, first served)

– SSTF (shortest seek time first)– SCAN (elevator algorithm)

1.Generation MM algorithms

– EDF (earliest deadline first)

– SCAN-EDF

– GSS (grouped sweeping scheme)2.Generation MM algorithms

– two-phase algorithms

- reduce seek time- reduce rotational latency- increase throughput- fair stream access- real-time constraints?

36

30.10.2002 INF 312, Ifi, UiO, høst2002 71

MMDBS: Conclusions

• investigated functionality needed to support MM applications

• illustrated how object-oriented and other modern DBMS technologies can be applied to realize MMDBMS

• alternative “levels” of application support by DBMS

• open issues:- effective storage models- MM query languages and processing techniques(handling of imprecise queries)

- ...

• Role of (MM)DBS in distributed MM systems

30.10.2002 INF 312, Ifi, UiO, høst2002 72

Conclusions - State-of-the-Art

• Multimedia file systems and multimedia storage servers for special multimedia applications exist today

• Implement the presented concepts

• Acceptable performance• Multimedia database systems are still under development,

certain aspects are solved

• Retrieval problems not yet solved in a satisfying manner