SeViAnno 2.0: Web-Enabled CollaborativeSemantic Video Annotation Beyond the Obvious

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-1

TeLLNet

LearningLayers

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

SeViAnno 2.0: Web-Enabled Collaborative

Semantic Video Annotation Beyond the

Obvious

Petru Nicolaescu & Ralf Klamma

RWTH Aachen University

Advanced Community Information Systems (ACIS)

{lastname}@dbis.rwth-aachen.de

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-2

TeLLNet

LearningLayers

Responsive Open

Community Information

Systems

Community Visualization

and Simulation

Community Analytics

Community Support

Web

An

alytics

Web

En

gin

eeri

ng

Advanced Community

Information Systems (ACIS)

Requirements

Engineering

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-3

TeLLNet

LearningLayers

Agenda

Motivation

Background

A Web information system for video annotation

– Collaborative

– Near real-time

– Cloud-enabled

SeViAnno 2.0: design and implementation

Conclusions and outlook

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-4

TeLLNet

LearningLayers

Motivation or

„We kill people based on metadata“ (David Cole)

Good tools for automatic multimedia annotation

available „for special domains“

Good single user tools for multimedia annotation

Beyond the obvious: (Web) tagging the non-visible

Collaborative multimedia annotation

Decoupled infrastructure

+

+

-

-

-

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-5

TeLLNet

LearningLayers

Related Work

SVCAT

VideoAnnEx

videoANTM-OntoMat 2.0

SeViAnno 2.0

Tool Platform Usage Tagging Method

Desktop Web-based Single user Collaborative Manual Automatic

++ ++ + ++

++ ++ + ++

++ ++ ++

++ ++ ++

++ ++ ++

M-OntoMat 2.0

+ Via-tool

SVCAT

VideoAnnEx

videoANT

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-6

TeLLNet

LearningLayers

EU FP7 Learning Layers Project

How can we scale up video tagging to support informal learning at the work place

Objectives

– Mobile creation of videos with learning materials (physical-digital world)

– Tag the non-obvious for informal learning in communities (social media layer)

– Scaffold meaningful learning by exploiting semantic tagging information (social semantic layer)

Two regional clusters

– Construction (Germany)

– Healthcare (UK) http://learning-layers.eu/

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-7

TeLLNet

LearningLayers

Decoupled Multimedia Annotation

Information System Architecture

3-tier architecture

Cloud-enabled

solutions

Advantages

– Separation of

concerns

– Scalability

– Rapid development &

deployment

– Application

customization

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-8

TeLLNet

LearningLayers

Multimedia Metadata Web Services

Tethys (former i5Cloud)

LAS

SeViAnno MPEG-7

Metadata Services

Connectors

(REST, HTTP, Ajax)

User Management

Object Management

Session Management

Cloud Video Transcoder

Tethys Data Storage

Video Transcoding

Scaling Management

Cloud Video Upload

ROLE SDK

NRT

Collaboration

Widget/Space

Management

Streaming

Server

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-9

TeLLNet

LearningLayers

Multimedia Metadata User Interface

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-10

TeLLNet

LearningLayers

Near Real-Time (NRT) Collaboration

Data structure dependent

– Tree-like data structures with operational transformations

– e.g. XML-based (MPEG-7)

NRT updates propagation

– Underlying infrastructure and used protocols (e.g. XMPP)

Preserving data consistency and user intention

– NRT video operations

– Concurrent updates

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-11

TeLLNet

LearningLayers

Space (shared by multiple users)

SeViAnno 2.0 & ROLE Framework

Web application (composed of widgets)

Widget (collaborative web

component)

EU FP7 ROLE Project

http://role-sandbox.eu/

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-12

TeLLNet

LearningLayers

User Interface Distribution

Limited space and interaction possibilities

Trend to using multiple personal

devices

Lack of tools and methodologies for Web-based DUIs

Lehrstuhl Informatik 5

(Information Systems)

Prof. Dr. M. Jarke

I5-NiKl-0714-13

TeLLNet

LearningLayers

Conclusion and Future Work

Open Source Development and Cloud Deployment

(ask for details)

Strong and scalable service infrastructure

(Distributed) Web widget and mobile interfaces

Challenges

– WebRTC infrastructure for update propagation

– Operational transformation algorithms for collaborative

semantic annotations

– Parallel video processing in the cloud (MapReduce)

– OpenID Connect (OIDC) for security and privacy