Giuseppe Riccardi, Marco Ronchetti

1

Giuseppe Riccardi, Marco RonchettiUniversity of Trento

Giuseppe Riccardi University of Trento 2

Outline

�Searching Information

�Next Generation Search Interfaces

�Needle

�E-learning Application

�Multimedia Docs Indexing, Search and Presentation

�Demo

�Conclusion


Searching Information(Past)

�Query model

�State-of-the-art -> bag of words

�Loyal Users

�Indexing

�Web pages, pdf, doc...

�State-of-the-Art ->index data structures

�Evaluation

�$$$

�Increasing the quality of the retrieved docs through

�Ranking

�Crawling


“Where is Mississippi”(11/30/2006)


Outline“Microsoft Reorg a bulwark against Google”

(11/30/2006)


Searching Information(Present)

�Meta-Engines

�Docs are retrieved by de-facto standard search engines

�Query-Answer pairs extraction (e.g. ask.com)

�Docs are

�Clustered (many-to-one) (e.g. vivissimo.com)

�Visualized via multiple views (many-to-many)


Outline“Microsoft Reorg a bulwark against Google”

(11/30/2006)


Companies“Microsoft Reorg a bulwark against Google”


Topics“Microsoft Reorg a bulwark against Google”


Stories“Microsoft Reorg a bulwark against Google”


Read/Center Story“Microsoft Reorg a bulwark against Google”


“Microsoft Reorg a bulwark against Google”


What is next?

�Typical tasks of web users

�Transactional

�Navigational

�Informational

�Task-Driven search

�Information search is part of a given task

�Business Intelligence (e.g. Decision Making)

�E-Learning ( e.g. Student E-Tutoring)

�Vertical Search Engines


Next Generation Information Search Interfaces

� Search Multimedia Documents� Indexing, Ranking� Large Scale � Real-time

� Query� Multimodal Input (text, gesture, speech)

� Vertical Engine� Limited Domain (e.g. business, education)� Structured & Annotated Content (not free!)� Certification of the results

� “What is the success rate of medication X?”� Results

� Multimedia Presentation� Bandwitdth


Needle Research Program

� Search Multimedia Content� Audio, Video, Metadata streams

� Indexing� Video� Automatic Speech Recognition (Unlimited-CSR)� Semantic Segmentation � Topic segmentation� Domain Ontology

� Input� Natural Language Query (Spoken or Text or

Multimodal)� Presentation

� Multimedia Presentation� Usability


Large amount of different kinds of educational resources.

Video lecturesBooksSlidesInteractive whiteboard streams

Goal : Information Search Interfaces for e-Learning

Needle: E-Learning Domain


Where is the content?Domain: Education Domain

� MSRI� Math-CS research/advanced topics (skewed)� Video/Audio lectures� Presentation vgs from video close shots

� MIT� Courseware (syllabus, lecture notes)� Video/Audio lectures� Wide range of topics

� University of Trento� Video/Audio lectures� Synch powerpoint presentation-video-audio� Skewed topics (CS & other)


System Components

� LODE� Content Creation � video lecture acquisition and synchronization with the

learning materials, and of their reproduction in a web browser.

� Needle : Interface for searching though the multimedia content and generating the multimedia documents.


LODE

LODE is a software for low-cost acquisition of lectures – no special requirements for the end user.

Streaming or Download

Off-Line(DVD)

•Good quality audio and video +•Images of the slides projected in class •Tools for navigating the lecture(by section title, by other indexes or through a time-slider) •Annotating video lectures withdocuments . •One DVD for a 50-hours class (MP4).


System Architecture

Transcripts

Slides

InteractiveWhiteboard

Forums

MeetingRecordings

Audio

Video


DB Structure

5 main entities: Actor ( the

teacher), Event (a lecture) , Series (a course), View (part

of a document) and Document (a MS PowerPoint presentation).


Lecture Topics

86%

8% 6%

Computer Science

Meteorology

Sociology

Multimedia Database(2003-Present)

Languages

24%

76%

English

Italian


Database Statistics

277198634

416

201Sociologia del Turismo [2003]

401Programmazione 2 [2003]

401Lab. Programmazione 2 [2003]

151Lab. Sistemi Operativi [2004]

151Lab. di Algoritmi e Strutture Dati [2004]

401Ingegneria del Software [2004]

401Architettura degli Elaboratori [2004]

33Science Faculty Seminars [2005]

401Programmazione 2 [2006]

24862Corso Meteorologia [2005]

401Machine Learning [2006]

401Distribuited Systems - Design [2005]

10SSSW05 [2005]

10SSSW06 [2006]

10WeeNet Summer School [2006]

HTL06

hoursspeakershoursspeakers

ItalianEnglish


Utterance Length Statistics

Words

Fre

quen

cy

Min : 1Max : 78Average : 19,9


Multimedia Indexing(speech driven)

“operatore new”Speech

Video

Slides

time


Multimedia Indexing(Metadata driven)

“operatore new”Slides

Video

time


Prototype

� Multimedia data streams (Audio, Video, ASR, Metadata)

� Indexing� Multimedia docs search

� Present & Browse


DemoWith Angela Fogarolli, Alessandro Bertacco (UNITN)


E-learning evaluationKirkpatrick’s 4 levels� Level 1 Reactions (Qualitative)

� Did they like it? Was the material relevant to their work?

� Level 2 Learning (Quantitative)� formal to informal testing to team assessment

and self-assessment.� Level 3 Behavior (Qualitative)

� Are the newly acquired skills, knowledge, or attitude being used in the everyday environment of the learner?

� Level 4 Results (Quantitative)� measures the success of the program in terms

that managers and executives can understandKirkpatrick, D.L. (1994).

Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler.


� Multimedia database with resource of other kind (interactive whiteboard recording, discussion, real and virtual meeting registration).

� Ontologies linking to offer knowledge-supported search.� Training of Unlimited-ASR

� Portable (domains)� Spoken Language Understanding (Query/Doc)� Semantic indexing� Evaluation

� E-learning domain� Content Creation

� Inter-University collaborative efforts

Future Research


� Information Search� Past & Present

� Next Generation Information Search� Needle

� Multimedia Documents� Indexing� Search� Presentation

� Content Creation� Inter-University collaborative efforts

Conclusion