Top Banner
Deliverable 9.3 Final Project Report PROJECT FINAL REPORT Grant Agreement number: 287911 Project acronym: LinkedTV Project title: Television Linked To The Web Funding Scheme: Collaborative project Period covered: from 01/10/2011 to 30/03/2015 Name, title and organisation of the scientific representative of the project's coordinator: Dr. Lyndon Nixon MODUL University (AT) Tel: +43 1 320 3555 533 E-mail: [email protected] Project website address: http://www.linkedtv.eu/
64

 · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Jul 10, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Deliverable 9.3 Final Project Report

PROJECT FINAL REPORT

Grant Agreement number: 287911

Project acronym: LinkedTV

Project title: Television Linked To The Web

Funding Scheme: Collaborative project

Period covered: from 01/10/2011 to 30/03/2015

Name, title and organisation of the scientific representative of the project's coordinator: Dr. Lyndon Nixon

MODUL University (AT)

Tel: +43 1 320 3555 533

E-mail: [email protected]

Project website address: http://www.linkedtv.eu/

Page 2:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 2/64

Dissemination level1 PU

Contractual date of delivery 31th March 2015

Actual date of delivery 06th May 2015

Deliverable number D9.3

Deliverable name Final Project Report

File LinkedTV_D9.3 Publishable.docx

Nature Report

Status & version Final, V0.7

Number of pages 64

WP contributing to the deliverable

WP 9

Task responsible Fraunhofer

Other contributors all Partners

Author(s) Joachim Köhler, Heike Horstmann, Martha Merzbach; Fraunhofer Lyndon Nixon; Modul

Reviewer all Partners

EC Project Officer Thomas Küpper

Keywords Final project report

Abstract (for dissemination) This document comprises the final report of LinkedTV. It includes a publishable summary, a plan for use and dissemination of foreground and a report covering the wider societal implications of the project in the form of a questionnaire.

1 • PU = Public • PP = Restricted to other programme participants (including the Commission Services) • RE = Restricted to a group specified by the consortium (including the Commission Services) • CO = Confidential, only for members of the consortium (including the Commission Services))

Page 3:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 3/64

Table of contents

1 Final publishable summary report ................................................... 4

1.1 Executive summary ............................................................................................... 4

1.2 Project context and objectives ............................................................................... 5

What is Linked Television? .................................................................................... 5

Objectives .............................................................................................................. 5

1.3 Scientific and technological results ........................................................................ 9

1.3.1 LinkedTV platform .................................................................................. 10

1.3.2 Media analysis ....................................................................................... 13

1.3.3 Linking video to Web content ................................................................. 16

1.3.4 Editor Tool ............................................................................................. 19

1.3.5 Personalisation of Linked Television ...................................................... 19

1.3.6 Presentation of Linked Television .......................................................... 22

1.3.7 Multiscreen Toolkit ................................................................................. 25

1.3.8 Linked Television Scenarios................................................................... 27

1.4 Impact, dissemination and exploitation ................................................................ 30

1.4.1 Impact .................................................................................................... 30

1.4.2 Dissemination ........................................................................................ 31

1.4.3 Exploitation ............................................................................................ 34

1.5 Additional information .......................................................................................... 35

2 Use and dissemination of foreground ........................................... 36

2.1 Section A (public) ................................................................................................ 36

2.1.1 List of scientific publications ................................................................... 37

2.1.2 List of dissemination activities ................................................................ 58

Page 4:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 4/64

1 Final publishable summary report

1.1 Executive summary

Television is changing. Already by the end of this decade, it will be radically transformed. With ubiquitous online access to audiovisual content, consumption on multiple screens and devices and convergence with the Web, the broadcast TV market faces new challenges and is looking now for solutions. LinkedTV provides technology, knowledge and experience for content owners and TV broadcasters.

After 42 months of research and development completed by 12 partners across Europe, the LinkedTV project has produced services, tools and documents, enabling a new generation of TV applications and making the vision of Linked Television become reality.

Linked Television is the seamless interweaving of TV and Web content into a single, integrated experience. It means watching the news and getting background information on the stories; it is seeing a painting in a TV programme and identifying the artist and the museum where it hangs. LinkedTV is making this possible and cost-effective for content owners and broadcasters by offering a platform, which handles the complete end-to-end workflow of audiovisual analysis, annotation and enrichment as well as personalisation to each viewer, as well as the tools to build client applications giving TV viewers access to this rich information and content layer linked to the TV programme.

Innovative technology identifies the concepts and topics in the TV programme, selects the most appropriate information and content to present for each concept and topic. Manual curation complements the accuracy of the automated services. Dedicated client applications can be built retrieving the programme enrichments from the Platform, eased by using LinkedTV’s developer toolkit to handle the presentation on and synchronisation across screens and devices.

The LinkedTV technology is based on research results that extended the state-of-the-art in many areas. New algorithms and methods for automatic decomposition of audiovisual content, the association of content segments with objects and scene labels, text and audio analysis and event and instance-based labelling of content segments have been developed to provide annotations on fragment level. The annotated media fragments are further enhanced by methods of named entitiy recognition and enriched with additional content discovered by Web mining approaches. Research on personalisation and contextualisation resulted in technologies that ensure the relevance of the enrichments for the consumer.

With several in-depth user studies, the project gathered valuable insights into the interests of TV consumers. This knowledge as well as the acquired experience in tool development and platform management will be exploited by LinkedTV partners in future projects and consultancy offers.

Research results, knowledge and the LinkedTV technology were successfully disseminated in a large number of publications, at conference workshops, industry events and exhibitions. Further information is available on the project website (http://www.linkedtv.eu) and on the showcase website (http://showcase.linkedtv.eu).

Page 5:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 5/64

1.2 Project context and objectives

What is Linked Television?

Television is changing. In fact, we think by the end of this decade it will be radically transformed. Linear broadcast channels will be stuff of the past: influenced by the Web and the fact TV will be watched on any type of screen (not just the “TV set”), we will browse and search our way to the audio-visual content we want to see. We will not just consume, we will interact, mainly by sharing our experience with others, but also with dedicated programming we will be interacting with the content itself.

The core of “television” will survive – the daily news programme, the live sports events – but the experience we have with TV will change. Today, TV and the Web remain very different experiences. Their integration is weak – typically no more fine-grained than at the level of the programme itself. Our Web-enabled televisions could provide us so much more information about what we see in the TV programme, but they do not, because they do not know what is in the TV programme.

Linked Television is the seamless interweaving of TV and Web content into a single, integrated experience. It is watching the news and getting background information on the stories at your fingertips; it is seeing a painting in a TV program and identifying the artist and the museum where it hangs. It is making this possible and cost-effective for the content owners and broadcasters by automating the programme enrichment, and personalising the links to each viewer.

By building on Web and broadcast specifications and standards, Linked Television is intended to become a solution for the whole industry, not a proprietary fragment. As Web and TV converge, linking between them will be not just possible, it will be necessary, in order to provide new, interactive services around audio-visual material. Our Public Showcase provides an insight into this new experience and the tools we provide to enable Linked Television. Please have look on http://showcase.linkedtv.eu.

Objectives

The overall goal of the LinkedTV project was to demonstrate “Television linked to the Web”. This was represented by a set of scenarios, which correspond to the benefits of seamlessly interweaving Web and TV content into an integrated experience. The scenarios are realised by a LinkedTV platform and (client side) video player which can support this interweaving of TV and Web content. These technologies were enabled by and built upon innovative research results in the areas of video analysis, annotation and hyperlinking, user interfaces and interaction, and personalization. The results were widely made public through a concerted dissemination action, as well as prepared for potential transfer of technology and industry uptake (commercialization) by a dedicated exploitation focus. The project itself run smoothly through a professional and competent approach to management. These different aspects were managed within the EU project by a clear division into work packages and tasks with respective responsible partners for the completion of each distinct activity:

Page 6:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 6/64

Figure 1: Work package structure of LinkedTV

The work packages were in line with the project objectives as listed below:

• Apply and improve media analysis approaches for the semantic analysis and linking of audiovisual information (WP1)

• Enable Web-scale hypervideo through the conceptual annotation of media (WP2) • Develop intuitive interfaces for Web and IPTV-based concept browsing and content

presentation in video (WP3) • Personalise selected concepts and content presentations to the user's profile and context

(WP4) • Create an end-to-end platform for LinkedTV services (WP5) • Realize the scenarios where LinkedTV services are used to enhance the TV/video

experience (WP6) • Disseminate project results in the community (WP7) • Exploit project results commercially (WP8)

The project objectives over its 42 months duration are summarized in the below diagram:

ICT

WP1Video analysisand annotation

WP2Linking hyper-

video to Web content

WP3Interface and description

Fundamental Research

WP5aArchitecture

and Interfaces

WP5bBackend

ComponentIntegration

Platform Development

and Integration

Management WP 9

WP6Scenarios

Pilot Experiments

WP7, 8Dissemination, Exploitation

6aEnvironment

6bCultural Heritage

6cMedia Art

User Requirements, Feedback

Standardisation

Exploitation

Platform provision

Frameworkprovision

API, formatdefinition

Target platform

requirements

WP4Contextualisation

and personalisation

WP5cClient Variants

Browser & Set-top Box

Page 7:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 7/64

Figure 2: Major milestones and phases of the project

The project started in Year 1 with an initial planning and first concrete steps towards the realization of the scenarios in an end-to-end workflow (LinkedTV platform, delivery to player), which integrated first outputs of the research and development work packages. Already these first achievements were presented at conferences and events, reflecting the scientific contributions of the project and raising its visibility in the academic and industry communities.

The second year of LinkedTV has been focused on taking the individual research and development results of the technical work packages, integrating them and making them available as an end-to-end workflow on an integrated LinkedTV platform. This required definition of both the internal data and activity flows between different components in one functional block (such as Analysis, Annotation, Personalisation Filter) and the overall data and activity flow between each block mediated by the LinkedTV platform. The functionality and usefulness of the platform is validated by the LinkedTV Scenarios through their realisation using the exposed LinkedTV services and user experience when playing back the enriched television content via the LinkedTV player. Dissemination continued to raise the visibility of the project in the academic and industry communities while individual technical work was published and presented in each domain’s leading conferences and workshops. Exploitation has guided the direction of the technical work with respect to market trends and industry demand; as the form of the technology offer has matured, first LinkedTV Business Models have been designed and discussed within the consortium.

In the following third year, LinkedTV finalised the technical research and development activities, which made Linked Television possible based on experience with and scenario evaluations of the Year 2 results. The LinkedTV scenarios have also been finalised in the domains of Cultural Heritage and News with working demonstrators based on the final technology produced in the project.

6 12 18 24 3630 42

Research

Develop-ment

Trials

Implementation Plan

Milestones Proof of Concept

Research 1 Research 2

Development 1 Development 2

Evaluation andDemonstration

Final Platform

Final Results

Pilot 1 Pilot 2Feedback 1

Basic Concepts Refined Concepts Revised Concepts

Final Results

Evalutation1 Evaluation 2

Month

Page 8:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 8/64

Dissemination of LinkedTV online and at events was further increased. First exploitation ideas of the second year have been developed further and became more enhanced.

The final phase of LinkedTV has been focused on evaluating the different results of the project, from individual outcomes – software packages or Web services, which provide a client with a particular functionality – to the joint outcome of the end-to-end workflow for TV analysis, annotation, enrichment and personalisation (via the LinkedTV Platform) as well as playout (Multiscreen Toolkit + LinkedTV Player). In addition, sustainability was an important topic to ensure all of the software and services could remain available after project end. Technical evaluations were complemented with user trials for evaluating user acceptance and interest in what LinkedTV offers. The complete technical output was packaged as the LinkedTV Solution and a common exploitation strategy to market this as a commercial offer to industry clients such as broadcasters has been drawn up. Final opportunities were used for further visible dissemination of the project and its results at events and online.

Page 9:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 9/64

1.3 Scientific and technological results

After 42 months of R&D completed by 12 partners across Europe, the LinkedTV project has produced services, tools and documents, which will enable a new generation of online applications that can interweave TV and Web content for the benefit of viewers.

For content owners, the key output is the LinkedTV platform. The platform triggers the process of media analysis, annotation, linking to related content (enrichment), storage of the resulting richer content metadata and provision via a REST API of access to clients for presentation and personalisation of enrichments. This can integrate with existing media asset management systems to add enrichment capability to current media playout.

The first step of enrichment is the media analysis. Analysis tasks that are being addressed in LinkedTV include the (semi)-automatic decomposition of the audiovisual content, the association of content segments with object and/or scene labels, text and audio information analysis, and event and instance-based labelling of content segments. In particular, media annotation labels distinct fragments of the video content with the concepts that are relevant to that fragment.

Those annotations on media fragment level are the basis for the next step, the media enrichment. It computes for each concept annotated to a media fragment a set of links to related Web content and online information about that concept. Therefore, a set of services fine-tuned to collect the most relevant Web links have been developed in LinkedTV, covering social media as well as supporting extracting links from trusted sites on provided white lists.

Complementing the accuracy of the automated analysis, annotation and enrichment services of LinkedTV, the Editor Tool is a Web based interface to the Platform that allows human curators to check, correct and complete the content segmentation and the annotations and enrichments attached to each segment.

Based on an innovative approach to user modelling that supports both implicit and explicit capture of user preferences, the annotations and enrichments are filtered to allow for a personalised user experience. Therefore, a considerable tool set and a dedicated workflow for extracting, learning and modelling of user information, usage and behaviour has been developed.

For the creation of innovative LinkedTV applications, a developer toolkit enables the implementation of HTML5-based applications that run and synchronise across multiple devices and screens. The multiscreen toolkit provides full access to all LinkedTV platform data and functionality and supports all key interaction modalities, thus simplifying the provision of the Linked Television experience to a customisation of the application user interface and user experience.

Two main scenarios have already been implemented in the LinkedTV project with the broadcasters RBB (LinkedNews) and AVROTROS (LinkedCulture). A third scenario used the LinkedTV platform to support a Media Arts concept focusing on the protests at Gezi Park in Istanbul.

LinkedCulture shows the Dutch TV programme “Tussen Kunst en Kitsch” (similar to the BBC‘s Antiques Roadshow) enriched with further information about the art objects on show and topics in discussion, satisfying viewers interest.

Page 10:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 10/64

The LinkedNews service shows how local German broadcaster RBB’s daily news programme is enhanced by links to relevant information about the subjects in the news, answering questions potentially raised by the new bits of information, sometimes even uncovering further details not mentioned in the programme itself.

The following sections present the main scientific and technological results in more detail.

1.3.1 LinkedTV platform

A platform architecture has been designed, which covers the server and client side based requirements for video analysis, enrichment and playout as needed for the envisioned user scenarios. The main design criteria have been openness, scalability, multi-user support, exchange of components and flexible connection to existing media platforms. As depicted in Figure 3 the LinkedTV system can be understood as consisting of a horizontal, or temporal, dimension following the workflow of the processing from video import to enriched playout, and a vertical dimension, consisting of three conceptual layers:

a) the backend LinkedTV Platform with Metadata Repository, import, streaming and REST API interfaces;

b) the different specialized automatic services for analysis, annotation, Media Fragment generation, and enrichment;

c) the user layer which includes all tools and components for user interaction and consumption, such as curation and playout by administrators, editors or end consumers.

Figure 3: LinkedTV Platform Architecture Overview

Page 11:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 11/64

The LinkedTV workflow is processed from the left to the right, starting with the import of a video resource, followed by the analysis and annotation through connected services such as visual analysis and segmentation, automatic speech recognition (ASR), subtitle extraction, and named entity recognition. The results are stored in the Metadata Repository and handed over for enrichment and final curation by the Editor Tool. The enriched videos can be played out through a streaming server to different target devices such as HbbTV TV sets or Second Screen devices. Optionally, the playout may be personalized to show enrichments according to the user profile, which has been previously defined manually or by behaviour tracking.

The main components of the LinkedTV Platform Backend are:

a) The Linked Media Service Bus, based on Enterprise Service Bus technology, which covers the functionalities for ingestion, import, file transfer, notification, message routing, event publishing, service integration, scheduled processing and more.

b) The Metadata Repository, which is basically an RDF Triple Store with a Sparql Endpoint.

c) The Streaming server which includes the capability to provide Media Fragments, and

d) REST API interfaces for accessing services and updating data within the platform

The platform is set up on Linux with Open Link Virtuoso as RDF Triple Store. In addition, a MySQL data base is used for the administration and status management of complete media resources.

Administration Tool and Dashboard

The Administration Tool allows to configure the components connected to the platform for analysis and enrichment and to define resources and execution parameters. The current platform contains the components developed in LinkedTV, but they could be complemented or substituted by products or services from third parties, which e.g. provide alternative analysis methods. Once set up, the workflow runs completely automatically in the background. The Dashboard allows editors to monitor the current processing status. Additionally, it provides the possibility to manually upload single media resources and their related files and then stepwise trigger the different LinkedTV workflow steps.

Page 12:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 12/64

Figure 4: The LinkedTV Platform Dashboard

As shown in Figure 4, all information for a particular media resource such as title, locator, related files and especially the status of the workflow steps can be observed together. Whenever a certain status is reached, the next step can be triggered manually by clicking a button. Some steps, especially the analysis procedures, may take quite long for certain analysis methods. In addition, the editor can use a monitoring service to show the state of video processing with the detected segments, key frames, annotations, or possibly errors.

Scalability, Hosting, Installation

The design of the platform has considered a flexible scalability, configuration and deployment for the needs of different customers from broadcast, WebTV or archives:

• Scalability, performance and reliability: the current version allows to serve pilot customers with an analysis of up to three videos in parallel. The number of end consumers is not restricted by the platform and only restricted by the delivery framework. For increasing needs, the platform is designed for running on multiple servers, but this would require additional extensions, e.g. to dynamically distribute analysis steps on several servers with load-balancing and fail-over strategies.

• Flexible extension of business and customization functionalities: in order to serve different customers, it is possible to extend the workflow with further services such as for usage statistics, licensing, billing and royalty fee models, handle service level agreements or integrate local production workflows such as sign-off procedures.

LinkedTV supports at present three Hosting Models for a flexible deployment:

1. Local installation and integration at the customer site. This model is in particular suitable, when an integration and adaption into the local production workflow is required.

2. Installation and hosting by a specific classical hoster like Host Europe or Strato. In this case the Platform administration could be done by either the customer or by partner

Page 13:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 13/64

Condat. This model is more suitable if the LinkedTV Platform is used as an external service with single service endpoints.

3. Hosting in a cloud service like Amazon Web Services or the Google Cloud Platform, where the LinkedTV Platform would be provided as a cloud service, or Platform as a Service (PaaS).

The setup procedure for a specific customer follows a Plan-Install-Configure-Integrate-Test-Run process. In general, the LinkedTV Platform is not an out-of-the-box product, because there are many different services and parameters to be tailored according the specific usage scenario:

• how many videos have to be analyzed per day, • which analysis methods are needed for the type of materials, • which hosting model should be used, • where should we store the results and which clients should be connected.

At present we use a manual setup procedure, for an increasing number of customers we foresee a semi-automatic installation procedure, software update mechanisms and use of monitoring and service tracking tools.

1.3.2 Media analysis

Starting with the automatic decomposition of the audiovisual content, LinkedTV developed methods for shot and scene segmentation that are generally applicable to any kind of video content, as well as chapter/topic segmentation algorithms that are specifically designed for the content that was used in the LinkedTV scenarios (LinkedNews and LinkedCulture). For shot segmentation, we used a combination of lightweight general-purpose local and global descriptors, dedicated detectors for specific types of transitions between shots (e.g., wipes and dissolves), and GPU- or CPU-based parallel processing. The result is a highly accurate detector of shot boundaries, which runs many times faster than real-time on a regular PC. For general-purpose scene segmentation, our method also exploits CPU-based parallel processing and the results of shot segmentation, together with algorithmic and software optimizations to speed up the video processing. Together, shot and scene segmentation run at about 7 times faster than real-time. For addressing specialized video segmentation needs of the LinkedTV scenarios (e.g., segmenting the portion of the video where a single art object is presented as a different chapter), we developed customized methods that effectively combine the shot segmentation results with the results of visual concept detection and object re-detection (both of these techniques are highlighted below).

Page 14:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 14/64

For annotating the different video segments with concept labels, the first important step is to represent the video segment with appropriate features. To this end, we devised new features and representation methods. These include video tomographs, which are 2D slices of the video volume with time being one of their two dimensions, and colour extensions of different local visual descriptors (SURF2, ORB3, BRISK4), which were combined with state of the art feature representations (VLAD5). Then, in order to effect visual concept detection, we used the above representations as input to new machine-learning-based concept detection schemes. Among the methods we developed is a two-layer stacking architecture that exploits multi-label learning techniques, such as ML-kNN, for improving concept detection accuracy. This, together with improvements such as the introduction of bagging in our SVM-based concept detection process, and careful consideration of data scaling in SVM-based concept detection, resulted in significant accuracy and computational efficiency gains. We also examined the use of binary vs. non-binary local descriptors and their combinations for concept detection, and the use of local descriptors vs. DCNNs6 and their combinations, and as a result we identified a number of combinations that can give us an excellent balance between accuracy and speed of the annotation process.

In order to similarly understand the audio track of the video, we developed methods for speaker identification, speech recognition (ASR), keyword extraction from ASR transcripts (or any other text, such as subtitles), and the extraction of non-speech audio features. For improving the LinkedTV speech recognition approach, we combined techniques such as Deep Neural Networks for acoustic modelling, an N-gram language model and a Recurrent Neural Network Language Model. For speaker identification, using an i-vector paradigm was our main contribution. These developments allowed us not only to benefit from a complete toolset for video analysis (being able to extract useful information from the visual, audio and text modalities), but also to combine modalities towards improved analysis of the content. Such techniques that were developed include methods for combining the visual and audio signals for topic segmentation, and the fusion of visual and textual (coming from ASR) cues for improving visual concept detection.

Extending the concept-based annotation of video with more complex labels (events) and with instance-level annotations was another area of interest. For event detection, we leveraged the analysis results discussed above (e.g., concept detection), and developed new event-based annotation schemes and efficient machine-learning methods. These include the Mixture Subclass Discriminant Analysis (MSDA) method and multiple extensions of it (e.g., GSDA, GSDA-LSVM combination), that gradually endowed it with beyond-state-of-the-art accuracy and efficiency characteristics. Other machine-learning methods that we developed include SRECOC and RDSVM. With respect to instance-level annotation of video, we developed video object re-detection methods that featured increased accuracy and speed compared to the state of the art. These were achieved by introducing GPU-based parallel processing, by taking into account the video structure (shot

2 Speeded Up Robust Features 3 Oriented FAST Rotated BRIEF, a visual feature descriptor 4 Binary Robust Invariant Scalable Keypoints 5 Vector of Locally Aggregated Descriptors 6 Deep Convolutional Neural Networks

Page 15:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 15/64

segmentation results), and by exploiting multiple images (different views) of the sought object, among others. Moreover, face detection and tracking methods were also implemented using the Viola-Jones detector, skin colour detectors and a spatio-temporal face matching method that exploits the shot segmentation results. The outcomes of face detection combined with our videoOCR method for optical character recognition from banners and logos (based on the tesseract engine) can be used for building a database of known speakers, thus allowing the verification of the identified speakers through audio analysis.

In order to facilitate the combined use of the various video, audio and text analysis techniques in LinkedTV, we developed the LinkedTV Analysis REST Service. This is a web service that integrates the media analysis techniques and allows for easily processing of any video and its related textual data (e.g., subtitles, metadata), aggregating the results of the processing in a single file that is available in different formats, including a MPEG-7 compliant one.

In addition to developing the above methods and tools, we also paid particular attention to their evaluation. This was done by performing a multitude of internal evaluations (both on LinkedTV data and on other available datasets), by performing user trials (particularly for the editor tool, where its assessment by human editors is a necessary evaluation step), and by participating to international benchmarking activities. The latter include different tasks of the MediaEval and TRECVID benchmarks. These benchmarking activities helped us in assessing, for instance, the suitability of different temporal granularities for video hyperlinking (MediaEval), or the goodness of our methods of concept and event detection in video (TRECVID SIN, MED), and also helped us to examine the progress we made from year to year.

Page 16:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 16/64

1.3.3 Linking video to Web content

Multimedia systems typically contain digital documents of mixed media types, which are indexed on the basis of strongly divergent metadata standards. This severely hampers the inter-operation of such systems. Therefore, machine understanding of metadata coming from different applications is a basic requirement for the inter-operation of distributed multimedia systems. In the context of LinkedTV, we have to deal with metadata standards that come from both the broadcast industry and the web community. Furthermore, the content is processed by automatic multimedia analysis tools which have their own formats for exchanging their results. One of the main goal of LinkedTV is to enrich seed video content with additional content, personalized to a particular user, that come from diverse sources including broadcast archives, web media, news and photo stock agencies or social networks.

Therefore, this step represents formally metadata provided by multimedia analysis processes over a LinkedTV seed video program and analyses textual resources in order to provide either structural information or related multimedia content that could be used for enriching this seed video. The work carried out was therefore structured around two axis:

• Converting both the legacy metadata and the results of the automatic multimedia analysis into RDF that are stored in a triple store within the LinkedTV platform. Named entity recognition is made in parallel on either the transcripts provided by the broadcaster or the automatic ASR. The named entities extracted can also be expanded and are themselves used in additional RDF annotations.

• Mining, retrieving and discovering additional content that enriches specific media fragments of the seed video programme being watched by the LinkedTV user. This additional content is looked up from a curated list of web sites using different methods ranging from structured queries (e.g. Europeana) or search APIs (e.g. social media web sites) to focused crawl and offline indexing and analysis processes.

This additional content is provided to apply a personalisation layer on top of these suggestions, and for displaying the additional information in the rich LinkedTV player.

The LinkedTV Core Ontology and the Linked Media Principles

The vision of “Linked Media” in the LinkedTV project is a vision of a potentially Web scale layer of structured and semantic metadata about media items, connected to the Web of Data, so that agents can determine significant links between different online media resources. Today's Web is increasingly non-textual in content, yet Web systems are still widely tuned to the processing of textual information from Web pages, even when considering the widely used search engines, or in fact the hyperlinking mechanism that so fundamentally underlies the concept of the Web. Integrating all of this, rapidly expanding in scale, non-textual content into the existing Web infrastructure is a significant challenge which is still being addressed in work on media analysis, annotation, management, search and retrieval. As Television and the Web converge, this disjunction between the traditional end user consumption of audiovisual streams and the interaction with online media resources within web sites becomes very clear. LinkedTV aimed to seamlessly integrate content from both sources into a single interactive experience, which requires that cross-network media

Page 17:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 17/64

shares a common description and an agreed means to access and process that description. LinkedTV system needs to be able to compare media item descriptions in order to generate links between media that can be used in enriching a TV programme or enabling end users to browse within a collection of media items from different sources.

Hence, we produced the LinkedTV core ontology: http://data.linkedtv.eu/ontologies/core. This ontology has also been aligned with the LinkedTV User Model Ontology (LUMO) in order to enable personalization.

The LinkedTV Annotation Tools Suite

At first, we developed a tool named TV2RDF, which has both a human friendly user interface and a REST API, which is fully integrated in the LinkedTV platform. The REST API takes as input an abstract identifier of a media resource stored by the LinkedTV platform, some legacy metadata, subtitles and results of media analysis, and it is responsible for generating an RDF description of all metadata following the LinkedTV core ontology.

Named Entities play a key role in the semantic annotations of the TV broadcasts. We have therefore setup a framework for enriching the seed video program with multimedia content and encyclopedia knowledge. The first step of our approach is to extract named entities, associate them with types or categories and disambiguate them with unique identifiers, which are generally linked data resources. This complements the annotations of the media analysis and generates new annotated video fragments.

Technically, we have first developed the SemiTag and then THD tools (also known as EntityClassifier) that are able to deal with both German and Dutch languages. We have also developed the NERD open source platform that integrates numerous NER extractors. The platform has been released at http://nerd.eurecom.fr. It includes an individual dashboard for monitoring a user activity in analyzing textual documents. As of January 2015, the NERD community includes more than 200 registered developers, 2000+ different users, which have triggered the analysis of more than 1.6 million documents.

Page 18:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 18/64

Figure 5: NERD dashboard depicting the extractors used and the remaining quota for a particular user

The LinkedTV Enrichment Tools Suite

Afterwards, we developed a suite of enrichment services that are tailored for the LinkedCulture scenario (TVEnricher) and for the LinkedNews scenario (TVNewsEnricher). The enrichment content is obtained from four types of sources:

a. by crawling and indexing web sites described in a white list specified by the content partners, b. by querying the API or SPARQL endpoint of the Europeana digital library network which is

publicly exposed, c. by querying multiple social networking APIs, d. by hyperlinking to other parts of TV programs within the same collection using a Solr index.

The IRAPI collector for harvesting and indexing the white list web sites provided by the scenario partners has been developed, which is in particular, a specific Metadata Extraction Service. The components enabling to store and to retrieve media fragments with annotations while providing a notification mechanism within the Linked Media Layer implementation of the end-to-end demonstrator have been implemented.

As for TV2RDF, the TVEnricher services also re-use the Open Annotation model to serialize the results. The motivation for this annotation differs: in the former case, the motivation is to annotate the seed video program, while in the latter, the motivation is to link to other media items.

Both have been contributed to the popular MediaFragment.js library (see https://github.com/tomayac/Media-Fragments-URI/) and were packaged as a node.js version of this library, published as open source at https://github.com/pasqLisena/node-mediafragment. We developed a generic media fragment player (http://linkedtv.eurecom.fr/HyperTED) and a node.js based media fragment server named MaFFiN (https://github.com/pasqLisena/maffin)

Page 19:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 19/64

Figure 6: HyperTED media fragment player and suggested TED talks recommendations based on semantic annotations

1.3.4 Editor Tool

Besides automatic analysis techniques, we also worked on the design and implementation of a manual annotation tool (also known as the editor tool (ET)). The purpose of this tool is to serve the editors of LinkedTV content, producers and curators, allowing them to view and edit, if necessary, the automatic annotations and, most importantly, the automatic enrichments. A first version of the annotation tool was developed and was subsequently extended and refined. For the development of the latest version of the ET, selected end-users participated in user trials and interviews in order to obtain feedback on the previous version of the tool and new insights into the day-to-day workflow of editing teams. New user requirements were identified as part of this process, and changes were made in accordance with them to both the front-end (user interface) of the tool and to its back-end (e.g., the architecture of the ET, the underlying data model and the way that the ET saves its data in the LinkedTV platform). The result is a tool that can bring the automatic video analysis tools and their output closer to the established workflows of video editors, saving them time without depriving them of their editorial control over the content, which is a fundamental requirement of video editors.

1.3.5 Personalisation of Linked Television

With the content annotated and enriched with a lot of quality hyperlinks, it seems apparent from the sheer mass of possibly linked entities within a single video that a user can be easily overwhelmed by the number of information. Named entities such as cities, persons, organizations might be of interest to some for looking-up background information, whereas others might only be interested in keywords or related sources for, e.g., art objects. For an enhanced viewing experience, personalization is thus a key technique to rank hyperlinks based on the user's preferences, possibly even more fine-granular based on the situation that he or she is currently in (e.g., not willing to read too much text in the afternoon hours but preferring videos).

Page 20:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 20/64

The overall goal of the project on personalization and contextualization is thus to model and build-up this preference of the user via a user profile. In principle, this can either be done by explicit information, i.e., by asking the user directly what his or her interest is, or by implicit behaviour tracking, which is mainly done by deriving the interest information from his or her interaction with the LinkedTV player. Moreover, additional contextualised information such as the attention of the viewer (e.g., “is he or she even looking at the screen currently?”) can be taken into account so that the preferences reflect changes of the current situation of the viewer. The ranking can be then either used to sort the hyperlinks, showing all of them but starting with the most promising, or used to boil down the overall links to only a selection of the highest-ranked hyperlinks.

As a crystallization point for the user profile, a preference-based ontology (in contrast to existing, content-driven linked-open-data (LOD) ontologies such as, e.g., dbpedia or yago) was developed. Within the course of LinkedTV, the decision was made to branch first into an implicit and an explicit ontology track for comparison. The main difference here was the ontology derived for explicit modeling aimed at user-friendly entities, whereas the implicit ontology has strong connections to the most prominent LOD vocabularies, which prevail in Semantic Web applications. While the focus moved to the implicit approach, the strength and insight gained from both initial ontologies have been merged and extended to form the LinkedTV User Model Ontology (LUMO, released as CC-SA). LUMO is designed to be light-weight so that the preferences can be realistically learned even with limited access to user interaction data. Further design aspects are a uniform user pertinent vocabulary, contextual features and hierarchical and non-taxonomical concept connections at the schema level. The expressive power has been ensured by modelling the user profiles on carefully designed archetypes, which also resulted in the expansion of art-related vocabulary to accommodate the “Tussen Kunst en Kitsch” scenario. An update containing the new concepts, new axioms (such as “hasTopic”, “hasSubtopic”), revised semantics and new object properties (“hasCurrency”, “hasPOI”) are included in the final version and were released to the public over the course of this project.

As stated before, a user might want to watch different content depending on, e.g., whether he or she is alone in the room or with somebody else. Looking at the screen for a certain amount of time can indicate a stronger interest in the current show than for proportions where he or she is more occupied with the second screen device. In order to contextualize a user's profile based on his or her current situation and need, contextualization feature extraction has been developed and integrated in the Attention/Context tracker. Kinect SDK has been compared against other methods for performance, and the accuracy of these experiments has been verified by marker-based ground truth, i.e., by comparing the results of the camera with the results of physical sensors. Features include face expression analysis, joint attention, head pose estimation and screen fixing time analysis.

A main source of information about user interests is the interaction with the player while videos are presented. Bookmarking a video for later viewing can indicate an interest in the current content, skipping the current video is an indication of disinterest. When a user consumes a video, attention/context trackers send the necessary events to the player, where they are enriched with VideoID/time information, alongside other player interaction (e.g., a user pressing the skip button).

Page 21:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 21/64

The service GAIN (General ANalytics Interceptor) fuses the data and builds up measures for user interest in a given context/videoID. Here, content is described in terms of shots, which are a subtype of media fragments and the smallest amount of content to which the interest clue can be related to. A Shot can contain multiple entities linked to the domain ontology by LOD identifiers. GAIN was extended and merged into the InBeat platform over the course of this project, to be part of a self-contained toolkit that allows to analyze user preferences and on-demand recommendation. The toolkit also supports contextualization as a temporal interaction type. InBeat has been very competetive in international benchmarks such as the CLEF NewsReel'14 news recommendation contest.

In order to create user profiles, by learning these user interests through the above mentioned behaviour capturing and learning, several tools have been established and bundeled under a modelling component API which handles storage and update of the profile and provision to all other parts of the LinkedTV personalisation workflow, namely Linked Profiler. The bundle includes: (a) LUMO wrapper, which interprets the semantic description of captured user preferences under the LUMO vocabulary, (b) Simple Learner, which is responsible for learning a user profile over time and content consumption, and (c) the association rules produced via the InBeat preference learner engine. The Linked Profiler has been evaluated with users as part of the user trials, asking them to rate the top interest/disinterest concepts, showing high and significant correlation to the user's expectations.

Once a model is established, a reasonar is needed in order to perform content and concept filtering based on the semantic description of a user profile and the concept items. LiFR, the Lighweight Fuzzy semantic Reasoner, performs this task with state-of-the-art speed and resource requirements. It has been released as open source (LGPL). LiFR has been evaluated extensively with manually annotated concepts (to ensure its quality under ideal settings) as well as automatically annotated and enriched content. Further, its algorithmic performance in terms of memory consumption and time was measured in comparison to other prominent fuzzy reasoners. It could be fortified that LiFR not only performs well in the LinkedTV context, but can also be realistically employed in limited-resource devices, thus enabling it to run on the user client so as to preserve user privacy in the personalization setting of LinkedTV.

For the experimental explicit user profiling branch, LUME (LinkedTV User Model Editor) provides an intuitive web-based user interface for the end users of LinkedTV. For curation, LUME was also extended with a curatable knowledge base called LUMOPedia, where content can be accessed easily on a web-based front-end whereas a backend provides various reasoning services in an efficient and scalable manner.

All of these tools together play an essential part in the personalization/contextualization-internal workflow, but they build upon the annotation and enrichment of the videos, the player API and information from the LinkedTV platform; finally, they were extended based on the demands from the scenarios. The tools described above have had stable releases, and open-source licenses are either already established or under preparation. Their state-of-the-art performance has been evaluated on LinkedTV internal data as well as internationally established benchmarks, as well as performance tests with comparable implementations.

Page 22:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 22/64

Based on the whole implicit preference workflow, several user-driven experiments and evaluations could be conducted. In collaboration with the scenarios, the user trials on implicit personalization were conducted; this process involved automatically building implicit user profiles from user interactions, on the actual LinkedTV application, and compared against manual ratings as derived from direct user feedback. Further, questionnaires filled out by the users were used for qualitative analysis of the personalization service. Presented with their automatically generated user profiles, the users were also asked to rate their satisfaction with these profiles and the resulting recommendation for several content items. It can be witnessed that the personalized recommendation yields good correlation between the explicit user rating and the automatically generated recommendation degree.

To this end, the evaluation results have consolidated the success in the performance of the personalization workflow, both within the general LinkedTV workflow as well as beyond it, thus validating its application to any given scenario and interface.

1.3.6 Presentation of Linked Television

With an in-depth user study we investigated the information needs and requirements for rich video applications in the News domain. Based on the outcome of the initial studies we designed an application, and refined it through a series of design iterations. We evaluated the prototype using a task-based approach. The evaluation showed that the prototype succeeds in fulfilling many of the users’ needs and requirements identified in preliminary studies. The participants’ reception of the application was very positive. I like it, it is useful; It is so much better to have this tool than not to have this tool; The best is the simplicity. It is easy to use; it gives you easy access to different kinds of information with just one click.

An important requirement for the application is that it needs to support passive as well as active use. TV viewers sometimes want to lean back and not be disturbed while at other times they want to lean forward and actively explore information. The prototype supports both types of use by providing two different modes. The lean back mode presents condensed information related to the entities in the news in the form of slides (a paragraph of text illustrated by an image). This mode is automatic and requires no user interaction, although interaction is possible if desired. The lean forward mode enables the user to explore information related to a specific news item. This mode is designed to support the information needs that were most prominent in the initial studies: coverage of the news items in other sources, opinions and in-depth information about the news item, a timeline of past events leading up to the news event and tweets near the news and near the user. These different types of information were organized in the interface in different sections.

Page 23:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 23/64

Figure 7: Screenshot of LinkedTV News application

The resulting application (see Figure 7) was presented at academic conferences, ACM TVX 2014 and ESWC 2014. The application was a finalist for the Digital Design prize of the Ibero American Biennial 20147 and was shown in the Central de Diseño in Matadero Madrid. It will itinerate to several Latin American countries being shown in design and art museums and venues.

An important observation of the user study is that TV viewers have different types of information needs. In the prototype application, different types of information are selected to satisfy these needs and they are organized into sections, e.g. in other sources and a timeline. Another important observation of the user study is that some information needs are triggered by a news item as a whole and not by a single entity that occurs in a news item. These two observations had impact on the design of the applications for the use case partners, the generation of related content and the moderation and selection of this content in the Editor Tool. The result is captured in the LinkedCulture and LinkedNews demonstrators as further described below.

In LinkedTV we also studied in the news domain which individual entities are interesting for viewers to receive information about. The results of the study provide user-centered guidelines to apply entity detection and selection for enriched video experiences. Users are primarily interested in receiving information about named entities of type organization and person rather than location. Subtitles alone are not sufficient as a source for named entities for video enrichment, as users are

7 http://www.bid-dimad.org/

Page 24:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 24/64

also interested in named entities that are not directly mentioned in the subtitles but are related to them by topic. Information from Wikipedia should be complemented with other sources, as participants expressed, that they would like to get information from Wikipedia in just about half of the entities in which they were interested.

We developed LinkedCulture for the cultural heritage program from AVROTROS ”Tussen Kunst en Kitsch“. LinkedNews was developed in collaboration with RBB for the regional News show RBB Aktuell. The applications demonstrate how enrichments coming from different sources on the Web can be accessed in an easy and integrated way. Using the Multiscreen Toolkit, the applications demonstrate how the video and the enrichments can be accessed seamlessly over different devices. The applications were demonstrated at the leading industry events IFA and IBC in 2014 and shown by LinkedTV partners at various academic conferences and events.

In LinkedCulture the user starts the application on the tablet and selects an episode or chapter to watch. The video is then beamed to the TV. When a chapter is finished, the user explores different types of information about the artwork discussed in the chapter, such as information about the related persons, background articles and related artworks. Finally, the user chooses the next chapter to watch from the suggestions provided by the LinkedTV system.

LinkedNews combines an HbbTV application with a companion application on a tablet. The HbbTV application provides an overlay over a TV broadcast. It provides a straightforward way to access background information about the entities occurring in a TV programme. The interaction with the interface is done through the remote control of the TV or set-top box. When the user activates the companion application on a tablet, it is automatically synced to the programme on the TV. Information about the entities occurring in the TV programme is now accessible on the tablet. In addition, the user can switch to active mode to explore the rich background and related information organized in different sections.

The demonstrators were evaluated in the final user trials. The evaluations confirmed that the users are interested in different types of information and that the organization of information into separate sections is useful. It allowed the participants to explore the topic from multiple perspectives, while at the same time it allowed users to select what best matched their interests and skip the dimensions they were not interested in. The studies also showed, that the participants like that the application provides the aggregation of information from multiple sources within a single application. To realize this aggregated presentation of Web content into a single application we developed various content processing tools that we collectively refer to as the content proxy. The input for these tools are the locators of the content generated by the LinkedTV system, typically URLs, and the output is a uniform representation of the content found at the URL that can be used by the companion application. The Content proxy is the bridge between the LinkedTV platform and the end-user applications.

Page 25:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 25/64

1.3.7 Multiscreen Toolkit

The advent of touchscreen tablets and smartphones has opened up new possibilities for using video beyond a single screen. In the course of the LinkedTV project a multiscreen toolkit has been developed that enables hands on exploration of future scenarios in this emerging area. Using a tablet as a second screen while watching TV is one of the recent ways in which multiscreen functionality is being applied. There is, however, a wide variety of scenarios involving multiple users and screens that could be conceived of: Allowing people to interact and collaborate with each other, or enabling mobile devices to connect to different screens in physical spaces to name just a few.

The multiscreen toolkit answers to the growing need to be able to work seamlessly with video across devices and screens. The toolkit is based on HTML5 and provides a basis for building and prototyping a wide range of multiscreen solutions. Supporting the HbbTV standards (1.5 and higher) for smart televisions and all major mobile platforms (iOS, Android) allows to work with a wide variety of devices. It avoids the dealing with device specific inputs from touch interfaces or remote controls allowing a full development focus on the application itself. Examples of applications include second screen applications for watching enriched TV programs and collaborative learning tools. The multiscreen toolkit enables easy prototyping of concept and interface ideas in the early phases of design and user evaluations in order to find the best solutions. The source code of the toolkit is published on Github, allowing external developers to use the product in their own way.

Figure 8: Example LinkedTV Multiscreen hypervideo player

In LinkedTV the Hypervideo player was developed based on the multiscreen toolkit supporting sharing of audiovisual content and the associated LinkedTV enrichment. The output can be shown on multiple screens of different device types such as mobile and tablet devices (iOS, Android) as well as HbbTV compatible smart TVs. To handle the player for all these different devices with different, sometimes limited, capabilities we rethought the model to distribute the player. The player

Page 26:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 26/64

application has been shifted from the client to the server only sending its output to the client that displays it on the screen as shown in Figure 9.

Figure 9: Multiscreen Toolkit server side applications

The multiscreen toolkit has been developed on top of the Springfield video platform that is capable of video processing, transcoding and streaming. For LinkedTV a Media Fragment server has been developed, compatible with the Media Fragments standard that is integrated in the platform and allows the player to request only a fragment from a larger video. Figure 10 gives an overview of the Springfield platform integration with the multiscreen toolkit and the Media Fragment server on top of the LinkedTV platform.

Page 27:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 27/64

Figure 10: Multiscreen application stack

Using the toolkit within LinkedTV, multiple players have been developed and tested regarding the different scenario requirements, for example with the LinkedNews and LinkedCulture applications. As one example for the developed versions, Figure 11 shows the LinkedNews application:

Figure 11: LinkedNews application

1.3.8 Linked Television Scenarios

Today’s TV viewers are multitaskers – they have a laptop, tablet or smartphone to hand while consuming TV on the big screen, and they turn to that second device to use the Web for additional information and content. However, companion applications are not well synchronised to this search – largely, their “knowledge” of what is on is limited to identifying the programme and linking to social media conversation around it or a list of cast members. What they do not know, and today cannot know, is what is INSIDE the TV programme at the time the user is viewing it and could be interested in. So viewer’s real needs for Web and TV convergence are still not answered today - how to find out more about something you see in a TV programme if you do not know what it is called, for example?

Page 28:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 28/64

LinkedTV has been working on the solution. By bringing together R&D experts across Europe who could provide the right tools to enable the envisioned interlinking of TV and the Web, we have produced a number of demonstrators where the experience of watching news – from the German broadcaster RBB – or a cultural heritage programme – the Dutch version of Antiques Roadshow from AVROTROS – is enhanced by additional information and content at the viewers fingertips – whether through remote control actions on a HbbTV-supporting SmartTV or through a Web application on their tablet or laptop. This information goes far beyond programme description or cast details like today’s offers – LinkedTV enables linking to information about concepts like persons, places and organisations inside the news story, links to background or related stories, or even browsing similar art objects in European collections while watching the discussion about another art object on screen.

LinkedCulture. This scenario uses the Dutch TV program Tussen Kunst en Kitsch (similar to the BBC’s Antiques Roadshow) which is a production of the public broadcaster AVROTROS. The general aim of the scenario is to describe how the information need of the Tussen Kunst en Kitsch viewers can be satisfied from both their couch and on-the-go, supporting both passive and more active needs. Linking to external information and content, such as Europeana, museum collections but also auction information has been incorporated in these scenarios. We have implemented a companion screen application for watching episodes of the Dutch TV program Tussen Kunst en Kitsch and providing viewers direct access to relevant related information about the art objects being discussed in the program. Viewing can be done on the main TV screen while exploring information on the second screen, and objects and their associated information can be bookmarked to return to later.

LinkedCulture application: demo available on request.

Demo video: https://vimeo.com/108891238

LinkedNews. The basic idea of RBB’s scenario is to enrich the local news program according to the needs and interests of the individual viewer. In some cases this may mean to just watch the daily news show as it is, in another case the viewer may prefer certain topics in some of the news items, and he or she may want to learn more about the topic in question or inform him/herself about one specific aspect. The result will be a personalised, TV-based on-demand service which directly links content concepts to online sources which will be displayed in the LinkedTV service. The LinkedNews scenario has been implemented with two approaches to viewer access and interaction with LinkedTV annotations and enrichments:

(1) A mobile application: starting with a connected device such as a laptop and tablet, the viewer can choose a news programme and begin to watch as well as browse the annotations and enrichments on the same screen. They also have the option to send the news programme video to an available other screen (e.g. their TV set) and continue to browse the LinkedTV enrichment content on their mobile device synchronized to the news programme. This is an adaptation of last years’ LinkedNews tablet interface for use with RBB content and enrichments. In particular, the available navigation for browsing enrichments has been aligned to the dimensions of enrichment supported for the RBB scenario which are related news items from RBB and related news items from other sources;

Page 29:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 29/64

(2) An HbbTV application: starting with a SmartTV or set top box (STB) with HbbTV functionality, the “red button” mode allows the viewer to access the LinkedTV application and begin to explore video chapters and their annotations. For annotated concepts, also relevant information for each concept can be displayed on the main screen. Again, if the viewer wishes to explore further the LinkedTV enrichments they may connect a second screen to the LinkedTV application, which will mirror the list of video chapters and their annotations but offer a more complete browsing of available enrichment links from the LinkedTV system using the LinkedNews interface described above.

LinkedNews application: demo available on request

Demo video of mobile application: https://vimeo.com/112286175

Demo video of HbbTV application: https://vimeo.com/106264077

SocialDocumentary (Media Art) This concept is based around the use of multimodal and tangible interfaces for manipulating and searching multimedia content, focusing on the recent protests in the Gezi Park in Istanbul. We processed with LinkedTV large amounts of user generated content around the protests such as photographs, videos and tweets. We have explored how media linking can enable a new type of exploration of content around a social movement, combined with the usage of innovative interface modalities.

The visitors are invited to interact with the installation by choosing keywords through the manipulation of three cubes. Each cube represents a class of keywords (people, action, emotion). As the videos are cut into segments and tagged with the same keywords, the system will automatically choose the most relevant video according to the position of the cubes. A Kinect-based system allows us to track the two closest visitors’ faces and infer on their interest for the played video. The more the visitors are interested, the higher is the probability to display this video to the next visitors, like in recommendation system on video broadcast platforms. In this way, the installation evolves with its public. Some segments will emerge from the others as the “most popular”.

The installation with all the sources is freely available on github (http://github.com/Zarfab/SocialDocumentary).

Video of the Istanbul installation: https://vimeo.com/113021089

Video of the Mons installation: https://vimeo.com/121008651

Page 30:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 30/64

1.4 Impact, dissemination and exploitation

1.4.1 Impact

The TV and content production industry, in particular for audiovisual content is undergoing severe and fast changes. The main reasons for this development can be summarized as follows:

• With the change from analogue to digital TV, conventional TV companies as public and private broadcasters entered the highly competitive market for digital content.

• New end devices as attractive smartphones and tablets are enabling new usage patterns for audiovisual content. New usage patterns as time-shifted viewing, streaming of audiovisual content, binging, media-meshing as well as second screen and Social TV are steadily increasing.

• New entrants to the market of audiovisual content as Apple, Google, Amazon but also Netflix and similar Internet companies are threatening the business of incumbent companies as public and private broadcasters.

In this changing world of audiovisual content, all involved business players try to position on the market in a new way. New, innovative solutions and content formats are required that will increase the competitiveness of business players in the changing market for audiovisual content.

With the market research within LinkedTV it was possible to provide a systematic overview of the changes in the audiovisual industry and to propose innovative business models for involved players. Furthermore, resulting LinkedTV technology provides interesting opportunities for producers, owners, and distributors of audiovisual content. These opportunities can be summarized as follows:

• Automatic analysis of audiovisual video content on video fragment level. This technology enables better and more cost efficient management of existing content as well as crosslinking of existing content.

• Semi-automatic selection and linking of audiovisual content on fragment level with relevant online content from certified online information sources as online newspapers.

• Development of attractive and innovative applications based on linked audiovisual and online content. These applications can be targeted to different end-devices as second screens or HbbTV smart TVs.

While the LinkedTV solution can be applied by all players on the market, i.e. producers, owners and distributors of audiovisual content, the LinkedTV solution available right after the end of the project is particularly suitable for incumbent public and private broadcasters as well as archives of audiovisual content. Two partners of the LinkedTV consortium are representatives of a public broadcasters and an archive. Thus, during the project it was possible to analyse prevailing processes in depth and to develop targeted solutions for these two types of companies.

Overall the LinkedTV solution enables a more cost-effective management, preservation and leveraging of audiovisual content, linking of audiovisual content with online content on fragment level and development of attractive end-user applications for various channels as second screen and HbbTV.

Page 31:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 31/64

1.4.2 Dissemination

Communication by EU projects to the outside world, including specific target groups of interest, whether they be scientific, academic, industrial or public institutions, including the EC itself, is a very important part of the project’s activity. While of course a focus needs to be kept on achieving meaningful research and development goals, which form the content of the external dissemination, the achievement of the goals should not be seen as separate from the task to announce and share those achievements with the wider communities. This is not just a question of demonstrating the good investment of the European Commission in funding the research work in the first place, but also ensuring that others have the opportunity to learn from and take up project results in their own academic studies, scientific experiments, commercial products, or public services.

As such, we can refer to the following impact-contributing dissemination activities by LinkedTV:

• Public Web site and publication/promotion of all public project outputs: deliverables, demos, tools and services

• Use of Social Web channels

• Press releases

• Participation and presentation at major scientific and industry events

• Showcasing the LinkedTV products and scenarios to industry

Highlights are summarized here, but a full report of dissemination activities and materials is provided in Section 2.1.2 as well as – of course – under http://www.linkedtv.eu (see the section “Demos&materials”).

Website

The website published 146 news items in 11 categories and using a total of 190 tags, and consists of 44 pages across 6 sections. The site content has been completed to the status at the end of the project, e.g. updated lists of LinkedTV Demos, Tools & Services, publications, presentations, events, deliverables and the final newsletter. This is complemented by ensuring that the separate pages for each research, development and scenario activity reflect the outcome of that activity at project end.

www.linkedtv.eu went online 29 November 2011. Google Analytics reports 26 750 sessions from 16 638 unique users providing 53 695 page views to date (April 15, 2015). The site will remain online.

Social Web

Our Twitter account has made 248 tweets to date and has now 220 followers. We are also increasingly mentioned by other accounts or our tweets are retweeted. This reach is validated by TweetReach, e.g. looking at the last week of Twitter activity @linkedtv was retweeted five times and thus reached 1 518 accounts and gained 2 084 impressions.

The Slideshare channel has been used since the beginning of LinkedTV (the first upload was our project introduction slides) as a means to distribute online project material. The materials can then

Page 32:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 32/64

be embedded in the LinkedTV website or elsewhere. We have to date 67 shares (nearly double from a year ago) and 104 followers. There are 32 presentations, where the second year update to the LinkedTV introduction (uploaded 16 months ago) has now gathered 23,300 views. There are 35 documents: the 2 newsletters, poster and 32 deliverables. D1.1 State of the Art for Hypervideo has remained our most popular document, with 4,272 views to date.

Press releases

Following a Fraunhofer IAIS press release on LinkedTV in July 2014 and its publication in the online newsletter in English and German, leading to wide press reporting on the project, the story was carried by Fraunhofer’s own magazine weiter.vorn in January 2015. An English version of this article is planned to be published in summer 2015. MODUL University also achieved Austrian press coverage for LinkedTV in March 2015 and is now planning an English and German press release for international distribution.

Fraunhofer newsletter, July 2014

• (eng.) http://www.fraunhofer.de/en/press/research-news/2014/July/Linking_television.html • (ger.)

http://www.fraunhofer.de/de/presse/presseinformationen/2014/Juli/Fernsehen_und_Internet.html

International press coverage, resulting from this:

• http://www.handelsblatt.com/technologie/vernetzt/projekt-linkedtv-fernsehen-und-internet-nahtlos-verbinden/10133220.html

• http://www.wiwo.de/projekt-linkedtv-fernsehen-und-internet-nahtlos-verbinden/10133530.html

• http://phys.org/news/2014-07-linking-television-internet.html

• http://article.wn.com/view/2014/07/01/Linking_television_and_the_Internet_Fraunhofer_Gesellschaft/

• http://www.absatzwirtschaft.de/content/communication/news/neues-fernsehkonzept-software-liefert-zusatzinformationen-aus-dem-internet;82438

• http://www.sciencedaily.com/releases/2014/07/140723110705.htm

• http://www.sciencenewsline.com/articles/2014072315470015.html

• http://www.pddnet.com/news/2014/07/linking-television-internet

• http://www.tvtechnology.com/article/ibc-second-screen-seachange/272005

Fraunhofer magazine weiter.vorn 1/2015

• http://www.fraunhofer.de/de/publikationen/fraunhofer-magazin/weiter-vorn_2015/weitervorn_1-2015_INhalt/weitervorn_1-15_26.html

Austrian press coverage, March 2015

• https://science.apa.at/site/natur_und_technik/detail.html?key=SCI_20150326_SCI45011703422892392

Page 33:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 33/64

• http://derstandard.at/2000013432345/Forscher-wollen-Fernsehen-und-Internet-verbinden

• http://www.pressreader.com/austria/salzburger-nachrichten/20150326/281698318243722/TextView

• http://www.oe24.at/digital/TV-Internet-wachsen-zusammen/182031775

Participation and presentation at events

Since Table 2 provides the complete list of all events and other dissemination activities of LinkedTV, we will only present some highlights here. Right from the start of the project, featuring the Linked Media idea and LinkedTV’s vision at related international conferences and exhibitions has been an important part of the project’s dissemination activities. In our first year, LinkedTV already showed a first technology demonstration at the EuroITV 2012 conference in Berlin. The demonstrator was based on the documentary scenario. On the pre-conference day of the EuroITV 2012, the 3rd edition of the Future Television workshop was held by LinkedTV.

Making the research community and the media industry aware of the achievements so far and LinkedTV’s vision was also one of the main activities in the second reporting period. The project was presented at a great number of major community events such as ISWC2012, ICMR2013, WWW2013 and ESWC2013 to name only the most prominent ones. Another workshop on FutureTV was organized at the EuroITV 2013. A dedicated LinkedTV session was held at INTETAIN 2013.

In Year 3, we concentrated on key events in the media industry and standardization domain. With a presentation at FIAT/IFTA 2013, we targeted the television archive sector. We presented both LinkedTV applications as well as the LinkedTV solution for video annotation and enrichment successfully to an interested audience at the International Broadcast Conference (IBC) 2014 in Amsterdam. At IFA 2014, the LinkedNews application was shown to a greater public.

Even in the final 6 months of the project, LinkedTV partners held presentations at important event. We continued to target the media industry by presenting at BroadThinking 2015 at the EBU in Geneva, Switzerland and giving an Ignite Talk at EuropeanaTech in Paris, France. Even after the official end of the project, partners continue their dissemination activities at two workshops with LinkedTV promotion during the high profile WWW2015 conference.

Showcasing LinkedTV

The LinkedTV Public Demonstrator is the public face of the project to the world and to potential adopters and users of LinkedTV technologies. Its focus is on offering external parties an opportunity to get to know and, where possible, have hands-on experience of Linked Television. We are aware of the fact that this type of television experience is still unknown and largely unexpected among the general public, and despite the growth in popularity of the so-called 'second screen' experiences, LinkedTV offers a much closer integration of the TV content and second screen content than known today. A key challenge for the Public Demonstrator is to achieve a clear communication of the meaning of Linked Television in an uncomplicated and non-technical manner, despite presenting highly innovative technology which is potentially disruptive to the traditional TV viewing experience.

Page 34:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 34/64

The Public Demonstrator has been launched September 11, 2014 at http://showcase.linkedtv.eu. It highlights the main outputs of the project:

• The LinkedTV Platform

• Editor Tool

• LinkedTV Player

as well as the LinkedCulture and LinkedNews applications.

1.4.3 Exploitation

In order to exploit the results of the LinkedTV project, all partners have provided detailed individual exploitation plans for after the project. The individual exploitation plans vary and include requests for patents, new national and international research projects as well as transfer projects to industry and education. Some partners were able to already commercialize the Linked TV results with industry contracts. Overall, one major achievement of the LinkedTV project are strong individual exploitation plans and achievements.

In addition to the individual exploitation activities the consortium agreed upon a common exploitation strategy. The common exploitation strategy foresees two common products:

• The LinkedTV video enrichment platform • Companion applications built upon a multiscreen toolkit.

Based on the intensive discussions common exploitation was developed around the following agreements:

1. Continuation of the common dissemination for LinkedTV technology after the project at least for 24 months. This includes continuation of the LinkedTV web site, the demo site and the collaborative platforms.

2. Continuation of the collaborating platforms for 24 months, to freeze and maintain the agreed upon Application Programming Interfaces (APIs) for at least 24 months and common agreements about service level agreements and revenue sharing for at least 24 months.

Page 35:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 35/64

1.5 Additional information

Contact and project details Start date: 01 October 2011

Duration: 42 months

Funding Scheme: FP7-ICT collaborative project

Project number: 287911

Project Coordination: Joachim Köhler, Fraunhofer IAIS

Project Scientific Coordination: Lyndon Nixon, MODUL University Vienna GmbH

Consortium:

o Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, Germany o STI8 International Consulting und Research GMBH, Austria o Centre for Research and Technology Hellas, Greece o Vysoka Skola Ekonomicka v Praze, Czech Republic o Université de Mons, Belgium o Stichting Centrum voor Wiskunde en Informatica, The Netherlands o EURECOM, France o Condat AG, Germany o Netherlands Institute for Sound and Vision, The Netherlands o Noterik BV, The Netherlands o Universität St. Gallen, Switzerland o Rundfunk Berlin-Brandenburg, Germany o MODUL9 University Vienna GmbH, Austria

For more information and direct contact, please visit our website: http://www.linkedtv.eu

8 Left the project after M24 9 Entered the project at M25

Television Linked To The Web

Page 36:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 36/64

2 Use and dissemination of foreground

In this section, we give an overview of the dissemination and use of foreground developed in the LinkedTV project. In this project, we have presented results in a variety of scientific publications and demonstrated the results of the project in workshops and at trade fairs. These aspects are described in Section A. The project has produced an integrated software system as well as several software components that can be exploited either immediately or in the near future. This outcome is summarised in Section B.

2.1 Section A (public)

The consortium has presented LinkedTV and its results at various renowned international trade fairs, conferences, exhibitions, congresses, sessions and workshops to the scientific community and potential customers. All in all, LinkedTV participated in 81 events and has produced 124 scientific publications. With regard to the type of participation, a wide spectrum from being on-site with an own booth, presenting a paper, participating at a poster session, giving a tutorial, invited talk or special session, up to co-chairing a conference was covered by the LinkedTV team. A list of all publications within the project is given in Table A1.

Page 37:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 37/64

2.1.1 List of scientific publications

Table 1: TABLE A1 – LIST OF SCIENTIFIC (PEER REVIEWED) PUBLICATIONS, STARTING WITH THE MOST IMPORTANT ONES

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

1 Multimodal Fusion: Combining Visual and Textual Cues for Concept Detection in Video

D. Galanopoulos, M. Dojchinovski, K. Chandramouli, T. Kliegr, V. Mezaris

in book "Multimedia Data Mining and Analytics", A. Baughman, J. Gao, J.-Y. Pan, V.A. Petrushin (Eds.)

Springer 2015 pp. 295-310

DOI 10.1007/978-3-319-14998-1

No

2 Visual Information Analysis for Interactive TV applications

Evlampios Apostolidis, Panagiotis Sidiropoulos, Vasileios Mezaris, Ioannis Kompatsiaris

In book: Encyclopedia of Information Science and Technology, Edition: Third Edition, Chapter: Visual Information Analysis for Interactive TV Applications,

2014 pp. 2208-2218

DOI: 10.4018/978-1-4666-5888-2.ch214

No

3 Local Features and a Two-Layer Stacking Architecture for Semantic Concept Detection in Video

F. Markatopoulou, N. Pittaras, O. Papadopoulou, V. Mezaris, I. Patras

IEEE Transactions on Emerging Topics in Computing

IEEE 2015 (to appear)

Yes (after 06/15)

10 A permanent identifier should be a persistent link to the published version full text if open access or abstract if article is pay per view) or to the final manuscript accepted for publication (link to article in repository). 11 Open Access is defined as free of charge access for anyone via Internet. Please answer "yes" if the open access to the publication is already established and also if the embargo period for open access is not yet over but you intend to establish open access afterwards.

Page 38:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 38/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

4 Analysis of Named Entity Recognition and Linking for Tweets

Leon Derczynski, Diana Maynard, Giuseppe Rizzo, Marieke van Erp, Genevieve Gorrell, Raphaël Troncy, Johann Petrak and Kalina Bontcheva

In Information Processing & Management

vol. 51 n. 2 2015 pp. 32-49 http://www.eurecom.fr/~troncy/Publications/Derczynski_Troncy-ipm15.pdf

Yes

5 Linked Hypernyms: Enriching DBpedia with Targeted Hypernym Discovery

Tomáš Kliegr Journal of Web Semantics

Elsevier November 2014

doi:10.1016/j.websem.2014.11.001

yes

6 Video tomographs and a base detector selection strategy for improving large-scale video concept detection

P. Sidiropoulos, V. Mezaris, I. Kompatsiaris

IEEE Transactions on Circuits and Systems for Video Technology

vol. 24, no. 7 July 2014 pp. 1251-1264

yes

7 When TV meets the Web: towards personalised digital media

Dorothea Tsatsou, Matei Mancas, Jaroslav Kuchar, Lyndon Nixon, Miroslav Vacura, Juliens Leroy, François Rocca, Vasileios Mezaris

In Semantic Multimedia Analysis and Processing, Evangelos Spyrou, Dimitrios Iakovidis, Phivos Mylonas (Eds.). Crc Pr I Llc

July 2014 ISBN 978-1-4665-7549-3.

No

8 Various Approaches to Text Representation for Named Entity Disambiguation

Ivo Lašek, Peter Vojtáš In International Journal of Web Information Systems

Volume 3 issue 3

Emerald 2013 242–259 http://www.emeraldinsight.com/doi/abs/10.1108/IJWIS-05-2013-0016

no

9 Linear subclass support vector machines

N. Gkalelis, V. Mezaris, I. Kompatsiaris, T. Stathaki

IEEE Signal Processing Letters

vol. 19, no. 9 September 2012

pp. 575-578

yes

10 Mixture subclass discriminant analysis link to restricted Gaussian model and other generalizations

N. Gkalelis, V. Mezaris, I. Kompatsiaris, T. Stathaki

IEEE Transactions on Neural Networks and Learning Systems

Volume: 24, Issue: 1

IEEE 2012 pp. 8 - 21 DOI: 10.1109/TNNLS.2012.2216545

yes

Page 39:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 39/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

11 Exploiting multiple web resources towards collecting positive training samples for visual concept learning

O. Papadopoulou, V. Mezaris Proc. ACM Int. Conf. on Multimedia Retrieval (ICMR’15)

Shanghai, China

June 2015 (to appear)

Yes (after 06/15)

12 GERBIL -- General Entity Annotator Benchmarking Framework

R. Usbeck, M. Röder, A. Ngonga Ngomo, C. Baron, A. Both, M. Brummer, D. Ceccarelli, M. Cornolti, D. Cherix, B. Eickmann, P. Ferragina, C. Lemke, A. Moro, R. Navigli, F. Piccinno, G. Rizzo, H. Sack, R. Speck, R. Troncy, J. Waitelonis, L. Weseman

Proc. 24th Int. World Wide Web Conference (WWW'15)

Florence, Italy

May 2015 To appear Yes

13 Entityclassifier.eu and SemiTags: Entity Discovery, Linking and Classification with Wikipedia and DBpedia

M. Dojchinovski, I. Lašek, T. Kliegr, O. Zamazal

In Proceedings of the NIST Text Analytics Conference 2014

NIST Gaithersburg, Maryland, USA

2015 (to appear)

yes

14 A Study on the Use of a Binary Local Descriptor and Color Extensions of Local Descriptors for Video Concept Detection

F. Markatopoulou, N. Pittaras, O. Papadopoulou, V. Mezaris, I. Patras

Proc. 21st Int. Conf. on MultiMedia Modeling (MMM'15)

Sydney, Australia

January 2015

pp 282-293

yes

15 VERGE: A Multimodal Interactive Video Search Engine

A. Moumtzidou, K. Avgerinakis, E. Apostolidis, F. Markatopoulou, K. Apostolidis, T. Mironidis, S. Vrochidis, V. Mezaris, Y. Kompatsiaris, I. Patras

Proc. 21st Int. Conf. on MultiMedia Modeling (MMM'15)

Sydney, Australia

January 2015

yes

Page 40:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 40/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

16 Video hyperlinking Vasileios Mezaris and Benoit Huet

MM 2014, 22nd ACM International Conference on Multimedia

Orlando, Florida, USA.

November 3-7, 2014

yes

17 Automatic fine-grained hyperlinking of videos within a closed collection using scene segmentation

Evlampios Apostolidis, Vasileios Mezaris, Mathilde Sahuguet, Benoit Huet, Barbora Cervenková, Daniel Stein, Stefan Eickeler, José Luis Redondo Garcia, Raphaël Troncy, Lukas Pikora

ACMMM 2014, 22nd ACM International Conference on Multimedia

Orlando, Florida, USA.

November 3-7, 2014

http://www.eurecom.fr/~troncy/Publications/Apostolidis_Troncy-acmmm14.pdf

yes

18 Linking text and visual concepts semantically for cross modal multimedia search

Bahjat Safadi, Mathilde Sahuguet, Benoit Huet

ICIP 2014, 21st IEEE International Conference on Image Processing

Paris, France.

October 27-30, 2014

http://dx.doi.org/10.1109/ICIP.2014.7025622

no

19 LinkedTV at MediaEval 2014 search and hyperlinking task

H.A. Le, Q.M. Bui, B. Huet, B. Cervenková, J. Bouchner, E. Apostolidis, F. Markatopoulou, A. Pournaras, V. Mezaris, D. Stein, S. Eickeler, M. Stadtschnitzer

MEDIAEVAL 2014, MediaEval Benchmarking Initiative for Multimedia Evaluation Workshop

Barcelona, Spain.

October 16-17, 2014

yes

20 No-Reference blur assessment in natural images using Fourier transform and spatial pyramids"

E. Mavridaki, V. Mezaris Proc. IEEE Int. Conf. on Image Processing (ICIP 2014)

Paris, France October 2014

yes

21 Finding and sharing hot spots in Web Videos

José Luis Redondo Garcia, Mariela Sabatino, Pasquale Lisena and Raphaël Troncy

13th International Semantic Web Conference (ISWC'14), Demo Track

Riva del Garda, Italy

October 2014

http://www.eurecom.fr/~troncy/Publications/Redondo_Troncy-iswc14.pdf

Yes

22 CERTH at MediaEval 2014 Synchronization of Multi-User Event Media Task

K. Apostolidis, C. Papagiannopoulou, V. Mezaris

Proc. MediaEval 2014 Workshop, CEUR vol. 1263

Barcelona, Spain

October 2014

yes

Page 41:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 41/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

23 Social Event Detection at MediaEval 2014: Challenges, Datasets, and Evaluation

G. Petkos, S. Papadopoulos, V. Mezaris, Y. Kompatsiaris

Proc. MediaEval 2014 Workshop, CEUR vol. 1263

Barcelona, Spain

October 2014

yes

24 LinkedTV News: A dual mode second screen companion for web-enriched news broadcasts

Lilia Perez Romero, Michiel Hildebrand, José Luis Redondo Garcia and Lynda Hardman

ACM International Conference on Interactive Experiences for Television & Online Video (TVX), Newcastle University, UK

June 2014 http://oai.cwi.nl/oai/asset/22562/22562D.pdf

yes

25 Seamlessly interlinking TV and Web content to enable Linked Television

L. Nixon, V. Mezaris, J. Thomsen

ACM Int. Conf. on Interactive Experiences for Television and Online Video (TVX 2014), Adjunct Proc.

FigShare Newcastle Upon Tyne, UK

June 2014 http://figshare.com/articles/WP_116_Seamlessly_interlinking_TV_and_Web_content_to_enable_Linked_Television/1032595

yes

26 Benchmarking the Extraction and Disambiguation of Named Entities on the Semantic Web

Giuseppe Rizzo, Marieke van Erp and Raphaël Troncy

In 9th International Language Resources and Evaluation Conference (LREC'14)

Reykjavik, Iceland

May 2014 http://www.eurecom.fr/~troncy/Publications/Rizzo_Erp_Troncy-lrec14.pdf

Yes

27 Exploiting the large-scale German Broadcast Corpus to boost the Fraunhofer IAIS Speech Recognition System

M. Stadtschnitzer, J. Schwenninger, D. Stein and J. Köhler

In 9th International Language Resources and Evaluation Conference (LREC'14

European Language Resources Association (ELRA)

Reykjavik, Iceland

May 2014 3887-3890 http://www.lrec-conf.org/proceedings/lrec2014/pdf/858_Paper.pdf

yes

28 Towards Linked Hypernyms Dataset 2.0: complementing DBpedia with hypernym discovery

T. Kliegr., O. Zamazal In 9th International Language Resources and Evaluation Conference (LREC'14)

26 - 31 May 2014

ELRA Reykjavik, Iceland

May 2014 pp. 3517-3523

http://www.lrec-conf.org/proceedings/lrec2014/pdf/703_Paper.pdf

yes

Page 42:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 42/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

29 Fast Shot Segmentation Combining Global and Local Visual Descriptors

E. Apostolidis, V. Mezaris In IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP)

Florence, Italy

May 2014 yes

30 Gradient-free Decoding Parameter Optimization on Automatic Speech Recognition

T. Ngyuen, D. Stein, M. Stadtschnitzer

In IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP)

Florence, Italy

May 2014 DOI: 10.1109/ICASSP.2014.6854203

no

31 Video event detection using generalized subclass discriminant analysis and linear support vector machines

N. Gkalelis, V. Mezaris Proc. ACM Int. Conf. on Multimedia Retrieval (ICMR)

Glasgow, UK April 2014 yes

32 When textual and visual information join forces for multimedia retrieval

Bahjat Safadi, Mathilde Sahuguet, Benoit Huet

ICMR 2014, ACM International Conference on Multimedia Retrieval

Glasgow, Scotland

April 1-4, 2014

http://www.eurecom.fr/publication/4257

no

33 ReSEED: Social Event dEtection Dataset

T. Reuter, S. Papadopoulos, V. Mezaris, P. Cimiano

Proc. 5th ACM Multimedia Systems Conference (MMSys’14)

Singapore March 2014 yes

34 Mining the web for multimedia-based enriching

M. Sahuguet and B. Huet MMM 2014, 20th International Conference on MultiMedia Modeling

Dublin, Ireland

January 8-10, 2014

no

35 A Comparative Study on the Use of Multi-Label Classification Techniques for Concept-Based Video Indexing and Annotation

F. Markatopoulou, V. Mezaris, I. Kompatsiaris

Proc. 20th Int. Conf. on MultiMedia Modeling (MMM'14)

Dublin, Ireland

January 2014

yes

Page 43:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 43/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

36 VERGE: An Interactive Search Engine for Browsing Video Collections

A. Moumtzidou, K. Avgerinakis, E. Apostolidis, V. Aleksic, F. Markatopoulou, C. Papagiannopoulou, S. Vrochidis, V. Mezaris, R. Busch, I. Kompatsiaris

Proc. Video Browser Showdown (VBS'14) at the 20th Int. Conf. on MultiMedia Modeling (MMM'14)

Dublin, Ireland

January 2014

yes

37 KINterestTV - Towards Non–invasive Measure of User Interest While Watching TV

J. Leroy, S. Rocca, M. Mancas, R. Ben Madhkour, F. Grisard, T. Kliegr, J. Kuchar, J. Vit, I. Pirner, P. Zimmermann

Proceedings of the 9th International Summer Workshop on Multimodal Interfaces - eNTERFACE'13, in Innovative and Creative Developments in Multimodal Interaction Systems - IFIP Advances in Information and Communication Technology (IFIP AICT)

Springer Lisbon, Portugal

January 1, 2014

pp. 179-199

doi:10.1007/978-3-642-55143-7_8

no

38 Improving event detection using related videos and Relevance Degree Support Vector Machines

Christos Tzelepis, Nikolaos Gkalelis, Vasileios Mezaris, I. Kompatsiaris

Proc. ACM Multimedia 2013 (MM’13)

Barcelona, Spain

October 21, 2013

pp. 673-676

ISBN: 978-1-4503-2404-5 doi>10.1145/2502081.2502176

yes

39 Social Event Detection at MediaEval 2013: Challenges, Datasets, and Evaluation

T. Reuter, S. Papadopoulos, G. Petkos, V. Mezaris, Y. Kompatsiaris, P. Cimiano, C. de Vries, S. Geva

Proc. MediaEval 2013 Workshop, CEUR vol. 1043

Barcelona, Spain

October 2013

yes

40 GAIN: web service for user tracking and preference learning – a SMART TV use case (system demo)

Jaroslav Kuchar, Tomáš Kliegr Proceedings of the 7th ACM Recommender Systems Conference (RecSys 2013)

12th-16th October 2013

ACM Hong Kong, China

October 2013

pp. 467--468

http://dl.acm.org/citation.cfm?id=2508217

no

Page 44:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 44/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

41 Video event recounting using mixture subclass discriminant analysis

Nikolaos Gkalelis, Vasileios Mezaris, Ioannis Kompatsiaris, Tania Stathaki

Proc. IEEE International Conference on Image Processing (ICIP 2013)

Melbourne, Australia

September 2013

yes

42 Enhancing video concept detection with the use of tomographs

Panagiotis Sidiropoulos, Vasileios Mezaris, Ioannis Kompatsiaris

Proc. IEEE International Conference on Image Processing (ICIP 2013)

Melbourne, Australia

September 2013

yes

43 Web and TV seamlessly interlinked: LinkedTV

Lyndon Nixon At INTETAIN 2013 Springer- Verlag

Mons, Belgium

July 2013 http://www.springer.com/gp/book/9783319038919

no

44 3D Head Pose Estimation for TV setups

Julien Leroy, Francois Rocca, Matei Mancas, Bernard Gosselin

Intetain2013: 5th International Conference on Intelligent Technologies for Interactive Entertainment

Mons, Belgium

July 2013 no

45 VideoHypE: An Editor Tool for Supervised Automatic Video Hyperlinking

Lotte Belice Baltussen, Roeland Ordelman, Jaap Blom

In 11th European Interactive TV Conference (EuroITV 2013

Como, Italy. June 2013 http://link.springer.com/chapter/10.1007%2F978-3-319-03892-6_5

no

46 User Information Needs for Environmental Opinion-forming and Decision-making in Link-enriched Video

Ana Carina Palumbo, Lynda Hardman

In 11th European Interactive TV Conference (EuroITV 2013)

Como, Italy June 2013 http://dl.acm.org/citation.cfm?id=2465973

no

47 Grab your Favorite Video Fragment: Interact with a Kinect and Discover Enriched Hypervideo

Vuk Milicic, José Luis Redondo Garcia, Giuseppe Rizzo, Raphaël Troncy

In European Interactive TV Conference (EuroITV'13), Demo Track

Como, Italy. June 2013 http://www.eurecom.fr/~troncy/Publications/Milicic_Troncy-euroitv13.pdf

Yes

48 Emerging Second Screen Value Networks: Insights for TV Broadcasters

Katarina Stanoevska-Slabeva, Veselina Milanova

In 11th European Interactive TV Conference (EuroITV 2013)

Como, Italy June 2013 yes

Page 45:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 45/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

49 Live Topic Generation from Event Streams

Vuk Milicic, Giuseppe Rizzo, Josè Luis Redondo Garcia, Raphaël Troncy and Thomas Steiner

In 22nd International World Wide Web Conference (WWW'13), Demos Track

Rio de Janeiro, Brazil

May 13-17, 2013

http://www.eurecom.fr/~troncy/Publications/Milicic_Troncy-www13.pdf

Yes

50 Connected Media Experiences: Web based interactive video using Linked Data

Lyndon Nixon, Matthias Bauer and Cristian Bara

In 22nd International World Wide Web Conference (WWW'13), Demos Track

Rio de Janiero, Brazil.

May 13-17, 2013

yes

51 Linked Services Infrastructure: a single entry point for online media related to any Linked Data concept

Lyndon Nixon In 22nd International World Wide Web Conference (WWW'13), Developers Track

WWW conferences

Rio de Janiero, Brazil.

May 13-17, 2013

http://www2013.wwwconference.org/papers/companion.htm

yes

52 Connected Media Experiences: Interactive video using Linked Data on the Web

Lyndon Nixon, Matthias Bauer and Cristian Bara

In the Linked Data on the Web workshop (LDOW2013) at 22nd International World Wide Web Conference (WWW'13)

CEUR-WS

Rio de Janiero, Brazil.

May 13-17, 2013

http://ceur-ws.org/Vol-996/

yes

53 The 2012 Social Event Detection Dataset

Symeon Papadopoulos, Emmanouil Schinas, Vasileios Mezaris, Raphaël Troncy, Ioannis Kompatsiaris

Proc. ACM Multimedia Systems 2013 (MMSys)

Oslo, Norway February-March 2013

no

54 Semantic personalisation in networked media: determining the background knowledge

Dorothea Tsatsou, Vasileios Mezaris and Ioannis Kompatsiaris

7th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP 2012)

Luxembourg December 3-4, 2012

6 pages yes

Page 46:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 46/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

55 EventMedia Live: Exploring Events Connections in Real-Time to Enhance Content

Houda Khrouf, Vuk Milicic and Raphaël Troncy

11th International Semantic Web Conference (ISWC'12), First Prize Winner of the Semantic Web Challenge

Boston, USA November 11-15, 2012

http://www.eurecom.fr/~troncy/Publications/Khrouf_Troncy-iswc12swc.pdf

Yes

56 Creating Enriched YouTube Media Fragments With NERD Using Timed-Text

Yunjia Li, Giuseppe Rizzo, Raphaël Troncy, Mike Wald and Gary Wills

Proc. 11th International Semantic Web Conference (ISWC'12), Demo Session

Boston, USA November 11-15, 2012

4 pages http://www.eurecom.fr/~troncy/Publications/Li_Troncy-iswc12pd.pdf

Yes

57 What Fresh Media Are You Looking For? Extracting Media Items from Multiple Social Networks

Giuseppe Rizzo, Thomas Steiner, Raphaël Troncy, Ruben Verborgh, Josè Luis Redondo Garcia and Rik Van de Walle

Proc. International Workshop on Socially-Aware Multimedia (SAM'12)

Nara, Japan October 29, 2012

6 pages http://www.eurecom.fr/~troncy/Publications/Troncy-saw12.pdf

Yes

58 Enrichment of News Show Videos with Multimodal Semi-Automatic Analysis

D. Stein, E. Apostolidis, V. Mezaris, N. de Abreu Pereira, J. Müller, M. Sahuguet, B. Huet, I. Lasek

NEM-Summit Istanbul, Turkey

October 16-18, 2012

6 pages yes

59 NERD meets NIF: Lifting NLP Extraction Results to the Linked Data Cloud

Giuseppe Rizzo, Raphaël Troncy, Sebastian Hellmann and Martin Bruemmer

Proc. 5th Workshop on Linked Data on the Web (LDOW'12)

Lyon, France April 16, 2012

10 pages http://www.eurecom.fr/~troncy/Publications/Rizzo_Troncy-ldow12.pdf

Yes

60 Defining and Evaluating Video Hyperlinking for Navigating

Roeland J.F. Ordelman, Maria Eskevich, Robin Aly, Benoit Huet and Gareth J.F. Jones

Proc. 3rd International Workshop on Linked Media (LiME'15)

Florence, Italy

May 2015 http://www.eurecom.fr/publication/4566

Yes

61 Concept-based image clustering and summarization of event-related image collections

C. Papagiannopoulou, V. Mezaris

Proc. 1st ACM Workshop on Human Centered Event Understanding from Multimedia (HuEvent'14) at ACM Multimedia (MM'14)

Orlando, FL, USA

November 2014

yes

Page 47:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 47/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

62 Inductive Entity Typing Alignment

Giuseppe Rizzo, Marieke van Erp and Raphaël Troncy

1st International Workshop on Linked Data for Information Extraction (LD4IE'14)

Riva del Garda, Italy

October 2014

http://www.eurecom.fr/~troncy/Publications/Rizzo_Erp_Troncy-ld4ie14.pdf

Yes

63 InBeat: Recommender System as a Service

J. Kuchar, T. Kliegr Working Notes for CLEF 2014 Conference

volume 1180 CEUR-WS

Sheffield, UK September 15-18, 2014

pp. 837-844

http://ceur-ws.org/Vol-1180/CLEF2014wn-Newsreel-KucharEt2014.pdf

yes

64 Learning Business Rules with Association Rule Classifiers

T. Kliegr, J. Kuchar, D. Sottara, S. Vojír

In International Web Rule Symposium 2014 (RuleML 2014)

LNCS 8620 Springer Verlag

Prague, Czech Republic

August 2014 pp. 236-250

http://www.springer.com/computer/ai/book/978-3-319-09869-2

no

65 Orwellian Eye - Video recommendation with Microsoft Kinect

T. Kliegr, J. Kuchar In Conference on Prestigious Applications of Intelligent Systems (PAIS'14) collocated with European Conference on Artificial Intelligence

Frontiers in Artificial Intelligence and Applications, Volume 263

IOS Press Prague, Czech Republic

August 2014 1227-1228 http://ebooks.iospress.nl/volumearticle/37183

YES

66 Head Pose Estimation by Perspective-n-Point Solution Based on 2D Markerless Face Tracking

F. Rocca, M. Mancas, B. Gosselin

Intelligent Technologies for Interactive Entertainment, Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering Volume 136

Springer Chicago, United States

July 9, 2014 pp. 67-76 doi:10.1007/978-3-319-08189-2_8

no

67 Social Documentary: An interactive and evolutive installation to explore crowd-source media content

F. Grisard, C. Kayalar, S. Alacam, Ö. Balaban, Y. Ipek, S. Dupont

Proceedings of the 2nd International Workshop on Interactive Content Consumption at TVX 2014

Newcastle upon Tyne, UK

June 25, 2014

yes

Page 48:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 48/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

68 LinkedTV: Web and TV seamlessly interlinked using semantic technology

Lyndon Nixon and Raphaël Troncy

11th Extended Semantic Web Conference (ESWC'14), EU Project Networking Track

Anissaras, Crete

May 2014 http://www.eurecom.fr/~troncy/Publications/Nixon_Troncy-eswc14.pdf

Yes

69 Augmenting TV Newscasts via Entity Expansion

José Luis Redondo García, Michiel Hildebrand, Lilia Perez Romero and Raphaël Troncy

11th Extended Semantic Web Conference - Posters and Demos Track

Anissaras, Crete

May 2014 http://www.eurecom.fr/~troncy/Publications/Redondo_Troncy-eswc14.pdf

Yes

70 A companion screen application for TV broadcasts annotated with Linked Open Data

Lyndon Nixon, Lotte Belice Baltussen, Lilia Perez Romero and Lynda Hardman

11th Extended Semantic Web Conference - Posters and Demos Track

Springer- Verlag

Anissaras, Crete

May 2014 http://www.springer.com/de/book/9783319119540

no

71 What are the Important Properties of an Entity? Comparing Users and Knowledge Graph Point of View

Ahmad Assaf, Ghislain Atemezing, Raphael Troncy and Elena Cabrio

11th Extended Semantic Web Conference - Posters and Demos Track

Anissaras, Crete

May 2014 http://www.eurecom.fr/~troncy/Publications/Assaf_Troncy-eswc14.pdf

Yes

72 LUMO: The LinkedTV User Model Ontology

Dorothea Tsatsou and Vasileios Mezaris

11th Extended Semantic Web Conference - Posters and Demos Track

Anissaras, Crete

May 2014 yes

73 LiFR: A Lightweight Fuzzy DL Reasoner

Dorothea Tsatsou, Stamatia Dasiopoulou, Ioannis Kompatsiaris and Vasileios Mezaris

11th Extended Semantic Web Conference - Posters and Demos Track

Anissaras, Crete

May 2014 yes

74 Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media

Lyndon Nixon and Raphaël Troncy

2nd International Workshop on Linked Media (LiME'14)

Springer- Verlag

Anissaras, Crete

May 2014 100-114 http://www.eurecom.fr/~troncy/Publications/Nixon_Troncy-lime14.pdf http://www.springer.com/gp/book/9783319119540

Yes

Page 49:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 49/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

75 Media Fragments Indexing using Social Media

Yunjia Li, Raphaël Troncy, Mike Wald and Gary Wills

2nd International Workshop on Linked Media (LiME'14)

Anissaras, Crete

May 2014 http://www.eurecom.fr/~troncy/Publications/Li_Troncy-lime14.pdf

Yes

76 Linked Hypernyms Dataset - Generation Framework and Use Cases

T. Kliegr, V. Zeman, M. Dojchinovski

In Linguistic Linked Data Challenge collocated with LREC 2014

Reykjavik, Iceland

May 2014 pp. 82-87 http://www.lrec-conf.org/proceedings/lrec2014/workshops/LREC2014Workshop-LDL2014%20Proceedings.pdf

yes

77 Linking WebContent Seamlessly with Broadcast Television: Issues and Lessons Learned

Jan Thomsen, Raphaël Troncy and Lyndon Nixon

In 4th W3C Workshop on Web and TV

Munich, Germany

March 12-13, 2014

http://www.eurecom.fr/~troncy/Publications/Troncy-webtv14.pdf

Yes

78 Developing a second screen application for TV broadcasts enriched via Linked Open Data

L. Baltussen, L. Pérez Romero, L. Nyxon, L. Hardman

Poster session, European Data Forum

Athens, Greece

March 19-20, 2014

http://2014.data-forum.eu/sites/default/files/pdf/edf2014_submission_69.pdf

yes

79 An Interactive Device for Exploring Thematically Sorted Artworks

A. Baltazar, P. Baltazar, C. Frisson

MultiMedia Modeling, Lecture Notes in Computer Science Volume 8326

Springer Dublin, Ireland

January 8, 2014

pp. 34-43 doi:10.1007/978-3-319-04117-9_4

no

80 Scenarizing CADastre Exquisse: A Crossover between Snoezeling in Hospitals/Domes, and Authoring/Experiencing Soundful Comic Strips

C. Sabato, A. Giraudet, V. Delattre, Y. Desnos, C. Frisson, R. Giot, W. Yvart, F. Rocca, S. Dupont, G. Vandem Bemden, S. Leleu-Merviel, T. Dutoit

MultiMedia Modeling, Lecture Notes in Computer Science Volume 8326

Springer Dublin, Ireland

January 8, 2014

pp. 22-33 doi:10.1007/978-3-319-04117-9_3

no

Page 50:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 50/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

81 Scenarizing Metropolitan Views: FlanoGraphing the Urban Space

B. Jacobs, L.A. Jacobs, C. Frisson, W. Yvart, T. Dutoit, S. Leleu-Merviel

MultiMedia Modeling, Lecture Notes in Computer Science Volume 8326

Springer Dublin, Ireland

January 8, 2014

pp. 11-21 10.1007/978-3-319-04117-9_2

no

82 Think Before You Link — Meeting Content Constraints when Linking Television to the Web

Daniel Stein, Stefan Eickeler, Rolf Bardeli, Evlampios Apostolidis, Vasileios Mezaris, and Meinard Müller

NEM Summit Nantes, France.

October 28-30, 2013

yes

83 From Raw Data to Semantically Enriched Hyperlinking: Recent Advances in the LinkedTV Analysis Workflow

Daniel Stein, Alp Öktem, Evlampios Apostolidis, Vasileios Mezaris, José Luis Redondo García, Raphaël Troncy, Mathilde Sahuguet, Benoit Huet

NEM Summit Nantes, France.

October 28-30, 2013

yes

84 Datasets and GATE Evaluation Framework for Benchmarking Wikipedia-Based NER Systems

Milan Dojchinovski, Tomáš Kliegr

Proceedings of the DBpedia & NLP 2013 Workshop at ISWC 2013

Volume 1064

CEUR-WS

Sydney, Australia.

October 22, 2013

6 pages http://ceur-ws.org/Vol-1064/Dojchinovski_Datasets.pdf

yes

85 Socially Motivated Multimedia Topic Timeline Summarization

Mathilde Sahuguet, Benoit Huet

Proceedings of SAM 2013, 2nd ACM International Workshop on Socially-Aware Multimedia, In conjunction with ACM Multimedia 2013

Barcelona, Spain

October 21, 2013

http://dx.doi.org/10.1145/2509916.2509925

no

86 LinkedTV at MediaEval 2013 Search and Hyperlinking Task

Mathilde Sahuguet, Benoit Huet, Barbora Cervenková, Evlampios Apostolidis, Vasileios Mezaris, Daniel Stein, Stefan Eickeler, Jose Luis Redondo Garcia, Raphael Troncy, Lukas Pikora

Working Notes in Proc. of the MediaEval 2013 Workshop

Barcelona, Catalunya, Spain.

October 18-19, 2013

yes

Page 51:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 51/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

87 Tell me why! Ain't nothin' but a mistake? Describing Media Item Differences with Media Fragments URI and Speech Synthesis

Thomas Steiner and Raphaël Troncy

In (ICME'13) 1st International Workshop on Media Fragment Creation and Remixing (MMIX'13)

San Jose, USA.

July 15-19, 2013

http://www.eurecom.fr/~troncy/Publications/Steiner_Troncy-mmix13.pdf

Yes

88 Fast object re-detection and localization in video for spatio-temporal fragment creation

Evlampios Apostolidis, Vasileios Mezaris, Ioannis Kompatsiaris

Proc. 1st Int. Workshop on Media Fragment Creation and reMIXing (MMIX'13) at the IEEE Int. Conf. on Multimedia and Expo (ICME 2013)

San Jose, CA, USA

July 15-19, 2013

yes

89 Video event detection using a subclass recoding error-correcting output codes framework

Nikolaos Gkalelis, Vasileios Mezaris, Michail Dimopoulos, Ioannis Kompatsiaris, Tania Stathaki

Proc. IEEE Int. Conf. on Multimedia and Expo (ICME 2013)

San Jose, CA, USA

July 15-19, 2013

yes

90 Transforming Association Rules to Business Rules: EasyMiner meets Drools

Stanislav Vojír, Tomáš Kliegr, Andrej Hazucha, Radek Škrabal, Milan Šimunek

Rule-ML Challenge 2013

Volume 1024

CEUR-WS

July 2013 http://ceur-ws.org/Vol-1004/paper13.pdf

yes

91 VideoHypE: An Editor Tool for Supervised Automatic Video Hyperlinking

Lotte Belice Baltussen, Roeland Ordelman, Jaap Blom

In 5th International Conference on Intelligent Technologies for Interactive Entertainment

Mons, Belgium.

July 2013 http://link.springer.com/chapter/10.1007%2F978-3-319-03892-6_5

no

92 Second Screen Interactions for Automatically Web-enriched Broadcast Video

Lilia Pérez Romero and Myriam C. Traub and Mieke H.R. Leyssen and Lynda Hardman

4th International Workshop on Future Television (FutureTV 2013)

Como, Italy. June 2013 http://oai.cwi.nl/oai/asset/22110/22110B.pdf

yes

93 MediaFinder: Collect, Enrich and Visualize Media Memes Shared by the Crowd

Raphaël Troncy, Vuk Milicic, Giuseppe Rizzo and Josè Luis Redondo Garcia

(WWW'13) 2nd International Workshop on Real-Time Analysis and Mining of Social Streams (RAMSS'13)

Rio de Janeiro, Brazil

May 14, 2013

http://www.eurecom.fr/~troncy/Publications/Troncy-ramss13.pdf

Yes

Page 52:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 52/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

94 Enriching Media Fragments with Named Entities for Video Classification

Yunjia Li, Giuseppe Rizzo, Josè Luis Redondo Garcia and Raphaël Troncy

(WWW'13) 1st Worldwide Web Workshop on Linked Media (LiME'13)

Rio de Janeiro, Brazil

May 13, 2013

http://www.eurecom.fr/~troncy/Publications/Li_Troncy-lime13.pdf

Yes

95 Second screen interaction: an approach to infer TV watcher’s interest using 3D head pose estimation

Julien Leroy, Francois Rocca, Matei Mancas, Bernard Gosselin

First Worldwide Web Workshop on Linked Media (LiME-2013)

Springer Rio de Janeiro, Brazil

May 13-17, 2013

doi:10.1007/978-3-319-03892-6_7

yes

96 Using Explicit Discourse Rules to Guide Video Enrichment

Michiel Hildebrand, Lynda Hardman

(WWW'13) 1st Worldwide Web Workshop on Linked Media (LiME'13)

Rio de Janeiro, Brazil

May 13, 2013

http://dl.acm.org/citation.cfm?id=2487968

yes

97 Learning with the Web: Spotting Named Entities on the intersection of NERD and Machine Learning

Marieke van Erp, Giuseppe Rizzo and Raphaël Troncy

(WWW'13) 3rd International Workshop on Making Sense of Microposts (#MSM'13), Concept Extraction Challenge

Rio de Janeiro, Brazil

May 13, 2013

http://www.eurecom.fr/~troncy/Publications/Rizzo_Troncy-msm13.pdf

Yes

98 Tracking and analyzing the 2013 Italian Election

Vuk Milicic, José Luis Redondo Garcia, Giuseppe Rizzo, Raphaël Troncy

In 10th European Semantic Web Conference (ESWC'13), Demos Track

Montpellier, France

May 2013 http://www.eurecom.fr/~troncy/Publications/Milicic_Troncy-eswc13.pdf

Yes

99 Parallel Faceted Browsing

Sven Buschbeck, Anthony Jameson, Adrian Spirescu, Raphaël Troncy, Houda Khrouf, Osma Suominen and Eero Hyvönen

ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'13) Interactivity Track

Paris, France.

April 27-May 2, 2013

http://www.eurecom.fr/~troncy/Publications/Buschbeck_Troncy-chi13.pdf

Yes

100 GAIN: Analysis of Implicit Feedback on Semantically Annotated Content

Jaroslav KUCHAR, Tomáš KLIEGR

WIKT 2012 Nakladatelstvo STU

Bratislava November 22-23, 2012

p. 75–78 http://wikt2012.fiit.stuba.sk/data/wikt2012-proceedings.pdf

no

Page 53:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 53/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

101 Recognizing, Classifying and Linking Entities with Wikipedia and DBpedia

DOJCHINOVSKI, Milan, KLIEGR, Tomáš

WIKT 2012 Nakladatelstvo STU

Bratislava November 22-23, 2012

p. 41–44 http://wikt2012.fiit.stuba.sk/data/wikt2012-proceedings.pdf

no

102 Antiques Interactive L. Baltussen and J. Oomen PATCH'12: Workshop on Personalized Access to Cultural Heritage

Nara, Japan November 2012

4 pages http://dl.acm.org/citation.cfm?id=2390875

no

103 Audio Fingerprinting for Media Synchronisation and Duplicate Detection

R. Bardeli, J. Schwenninger, and D. Stein

Proc. Media Synchronisation Workshop

Berlin, Germany

October 11, 2012

4 pages accepted

no

104 A Demonstrator for Parallel Faceted Browsing

Sven Buschbeck, Anthony Jameson, Raphaël Troncy, Houda Khrouf, Osma Suominen and Adrian Spirescu

Proc. Intelligent Exploration of Semantic Data Workshop (IESD'12)

Galway, Ireland

October 8-12, 2012

4 pages http://www.eurecom.fr/~troncy/Publications/EventMap-iesd12.pdf

Yes

105 Social Event Detection at MediaEval 2012: Challenges, Dataset and Evaluation

Symeon Papadopoulos, Emmanouil Schinas, Vasileios Mezaris, Raphaël Troncy and Ioannis Kompatsiaris

Proc. MediaEval Benchmarking Initiative for Multimedia Evaluation

Pisa, Italy October 4-5, 2012

2 pages yes

106 Association Rule Mining Following the Web Search Paradigm

Radek Škrabal, Milan Šimunek, Stanislav Vojír, Andrej Hazucha,Tomáš Marek, David Chudán, Tomáš Kliegr

Proc. of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2012)

LNCS 7524 Springer- Verlag

Bristol, UK September 24-28, 2012

4 pages http://link.springer.com/chapter/10.1007%2F978-3-642-33486-3_52

yes

107 Contextualised user profiling in networked media environments

Dorothea Tsatsou, Lyndon Nixon, Matei Mancas, Miroslav Vacura, Rüdiger Klein, Julien Leroy, Jaroslav Kuchar, Tomáš Kliegr, Manuel Kober, Maria Loli, Vasileios Mezaris

Proc. 2nd International Workshop on Augmented User Modeling in conjunction with 20th Conference on User Modeling, Adaptation and Personalization (UMAP 2012)

Montreal, Canada

July 16-20, 2012

yes

Page 54:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 54/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

108 Semi-Automatic Video Analysis for Linking Television to the Web

D. Stein, E. Apostolidis, V. Mezaris, N. de Abreu Pereira, and J. Müller

Proc. FutureTV Workshop

Berlin, Germany

June 2012 8 pages yes

109 LinkedCulture: browsing related Europeana objects while watching a cultural heritage TV program

Lyndon Nixon, Lotte Baltussen and Johan Oomen

In Personalised Access to Cultural Heritage (PATCH) workshop, co-located with the International User Interfaces Conference (IUI 2015)

ACM Digital Library

Atlanta, USA March 2015 http://dl.acm.org/citation.cfm?id=2678025.2716266&coll=DL&dl=GUIDE&CFID=506518242&CFTOKEN=28862090

no

110 Linking socially contributed media with events

Xueliang Liu and Benoit Huet Multimedia Systems November 2014

ISSN: 0942-4962 http://dx.doi.org/10.1007/s00530-014-0436-3

no

111 Bag-of-Entities text representation for client-side (video) recommender systems

J. Kuchar, T. Kliegr In First Workshop on Recommender Systems for Television and online Video (RecSysTV)

ACM RecSys 2014

Foster City, Silicon Valley, USA

October 6-10, 2014

4 pages https://drive.google.com/file/d/0B27JIwgO5CB_U1Jnamc3UXgtc2c/edit

yes

112 Towards a Localised German Automatic Speech Recognition

Michael Stadtschnitzer, Christoph Schmidt, Daniel Stein

11. ITG Fachtagung Sprachkommunikation

Erlangen, Germany

September 24-26, 2014

ISBN 978-3-8007-3640-9

no

113 Describing and Contextualizing Events in TV News Show

José Luis Redondo García, Laurens De Vocht, Raphaël Troncy, Erik Mannens and Rik Van de Walle

In 2nd International Workshop on Social News on the Web (SNOW'14)

Seoul, South Korea

April 7, 2014 http://www.eurecom.fr/~troncy/Publications/Redondo_Troncy-snow14.pdf

Yes

114 Extracting Resources that Help Tell Events' Stories

Carlo Andrea Conte, Raphaël Troncy and Mor Naaman

In 1st International Workshop on Social Multimedia and Storytelling (SoMuS'14)

Glasgow, Scotland

April 1, 2014 http://www.eurecom.fr/~troncy/Publications/Conte_Naaman_Troncy-somus14.pdf

Yes

Page 55:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 55/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

115 Event-centric hyperlinking of socially contributed multimedia content

Benoit Huet In 1st International Workshop on Social Multimedia and Storytelling (SoMuS'14)

Glasgow, Scotland

April 1, 2014 http://www.eurecom.fr/publication/4267

no

116 Social Event Detection at MediaEval: a 3-year retrospect of tasks and results

Vasileios Mezaris, Georgios Petkos, Symeon Papadopoulos, Raphaël Troncy, Philipp Cimiano, Timo Reuter and Yiannis Kompatsiaris

In 1st International Workshop on Social Events in Web Multimedia (SEWM'14)

Glasgow, Scotland

April 1, 2014 http://mklab2.iti.gr/sewm14/wp-content/uploads/2014/03/SEWM_2014_Proceedings.pdf

yes

117 Interaction Design and User Needs for TV Broadcasts Enriched with Linked Open Data

Pérez Romero L., Baltussen L., Van Leeuwen P., Hildebrand M., Hardman L., Nixon L.

in VII International Workshop on Personalized Access to Cultural Heritage (PATCH 2014), Intelligent User Interfaces Conference

Haifa, Israel February 24, 2014

https://patch2014.files.wordpress.com/2012/07/submission-11-version-of-feb-10-20_29.pdf

yes

118 Video Event Understanding

Nikolaos Gkalelis, Vasileios Mezaris, Michail Dimopoulos, Ioannis Kompatsiaris

Encyclopedia of Information Science and Technology

IGI Global 2014 pp. 2199-2207

DOI: 10.4018/978-1-4666-5888-2.ch213

No

119 Event representation and visualization from social media

Xueliang Liu and Benoît Huet PCM 2013, 14th Pacific-Rim Conference on Multimedia

Nanjing, China

December 13-16, 2013

http://dx.doi.org/10.1007/978-3-319-04117-9_24

no

120 Television meets the Web: a Multimedia Hypervideo Experience

José Luis Redondo García, Raphaël Troncy

In ISWC 2013 Doctoral Consortium

Sydney, Australia.

October 20, 2013

48-55 http://www.eurecom.fr/~troncy/Publications/Redondo_Troncy-iswc13.pdf

Yes

121 Ethical and Social aspects of Internet Search in Historical Perspective

Miroslav Vacura In Proceedings of IDIMT - Interdisciplinary Information and Management Talks 2013

Sept. 11–13, 2013

Trauner-Verlag.

Prague, Czech Republic

September 2013

pp. 289–296

http://www.idimt.org/sites/default/files/IDIMT_2013_proceedings_final.pdf

yes

Page 56:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 56/64

NO Title Main author Title of the periodical or the series

Number, date or

frequency Publisher Place of

publication Year of

publication Relevant

pages

Permanent identifiers10 (if available)

Is/Will open access11

provided to this

publication?

122 Entityclassifier.eu: Real-time Classification of Entities in Text with Wikipedia

Milan Dojchinovski, Tomáš Kliegr

In ECML/PKDD (Demo paper)

LNCS 8190 Springer- Verlag

September 2013

pp. 654–658

http://link.springer.com/chapter/10.1007%2F978-3-642-40994-3_48

yes

123 Automatic Parameter Tuning and Extended Training Material: Recent Advances in the Fraunhofer Speech Recognition System

Jochen Schwenninger, Daniel Stein, Michael Stadtschnitzer

Proc. Workshop Audiosignal- und Sprachverarbeitung

Koblenz, Germany

September 2013

8 pages no

124 LinkedTV - Cross Media beim RBB

de Abreu Pereira, N., Think Cross - Change Media: Eine Standortbestimmung im Jahr 2012

Michitsch, Chr. et al. (Ed.)

Magdeburg 2012 pp. 36-41 ISBN 978-38-482-2378-7 http://www.ma-crossmedia.de/index.php?f=Konferenz&s=Konferenzband2012

yes

Page 57:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 57/64

In addition to scientific publications, the LinkedTV team has strived to bring the project to the awareness of numerous target groups, including industry players, the scientific community, standardisation groups, media, and the general public. From the start of the project, featuring the Linked Television vision at related international conferences and exhibitions has been an important part of the project’s dissemination activities. Table A2 gives a complete list of events in which LinkedTV has played a role. For certain highlights please refer to section 1.4.2.

Page 58:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 58/64

2.1.2 List of dissemination activities

Table 2: TABLE A2 - LIST OF DISSEMINATION ACTIVITIES

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

1 presentations UMONS Presentation at UMONS open day: numediart Documentary (Social Documentary with numediart demo videos content)

March 28, 2015 Mons, Belgium Civil Society 50 Belgium

2 Other CERTH 111 MPEG Meeting February 16-20, 2015

Geneva, Switzerland Scientific Community, Industry ~300 International

3 presentations S&V Ignite Talk at EuropeanaTech 2015 February 2015 Paris, France Industry 250 International

4 presentations UMONS Presentation of Social Documentary installation to UMONS cultural and industrial partners from Region Nord-Pas-De-Calais, Region Wallonne and Region Flandre

January 26, 2015 Mons, Belgium Other 25 Belgium, France

5 conference CERTH The 21st International Conference on Multimedia Modelling

January 5-7, 2015 Sydney, Australia Scientific Community ~100 International

6 posters MODUL LinkedTV_Culture_poster January 2015 used at EuropeanaTech Conference

Industry 100 Europe

7 Other S&V NewsReader hackathon January 2015 Amsterdam, The Netherlands

Scientific Community 50 International

8 exhibitions CWI Latin American Design Bienal November 25, 2014

Madrid, Spain Scientific community, Medias 200 International

9 workshops CERTH TRECVID 2014 Workshop November 10-12, 2014

Orlando, Florida, USA Scientific Community, Industry, Policy makers

~100 International

10 conference CERTH The 22nd ACM International Conference on Multimedia, ACM MM 2014

November 3-7, 2014

Orlando, Florida, USA Scientific Community, Industry ~400 International

11 Workshop CERTH ITI-CERTH participation to TRECVID 2014 Workshop

November 2014 Orlando, FL, USA Scientific Community, Industry, Policy makers

100 International

12 A drop down list allows choosing the dissemination activity: publications, conferences, workshops, web, press releases, flyers, articles published in the popular press, videos, media briefings, presentations, exhibitions, thesis, interviews, films, TV clips, posters, Other. 13 A drop down list allows choosing the type of public: Scientific Community (higher education, Research), Industry, Civil Society, Policy makers, Medias, Other ('multiple choices' is possible).

Page 59:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 59/64

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

12 presentations S&V Cross Media Café 'Uit het lab' November 2014 Amsterdam, The Netherlands

Industry 150 The Netherlands

13 presentations S&V, MODUL EUscreenXL 2014 conference October 30, 2014 Rome, Italy Medias 150 International

14 conference CERTH International Conference on Image Processing, ICIP 2014

October 27-30, 2014

Paris, France Scientific Community ~1500 International

15 workshops EURECOM, CERTH, Fraunhofer, UEP

MediaEval 2014 October 16-17, 2014

Barcelona, Spain Scientific Community ~100 International

16 presentations UMONS Presentation at Vice-Versa 1.0 workshop: Presentation of gesture recognition applied to art installation and TV control

October 16, 2014 Mons, Belgium Other 50 Belgium

17 exhibitions CWI CWI Open Day October 5, 2014 Amsterdam, The Netherlands

Other 50 Netherlands

18 video CWI LinkedTV News video October 1, 2014

19 exhibitions Many partners, RBB

IBC 2014 September 12-16, 2014

Amsterdam, The Netherlands

Other, Medias, Industry A potential of about

30 000

International

20 exhibitions RBB IFA 2014 September 5-10, 2014

Berlin, Germany Other, Medias, Industry 50 per day, around 250

all in all

Europe

21 web STI, NOTERIK, CONDAT, S&V, RBB

http://showcase.linkedtv.eu September 2014 Industry International

22 presentations UMONS Presentation demonstration of social documentary installation in Istanbul

August 23, 2014 Istanbul, Turkey Civil Society 20 Turkey

23 Other MODUL LinkedTV stress cubes for IBC 2014 August 2014

24 posters Fraunhofer LinkedTV roll-up for IBC 2014 August 2014

25 Other Fraunhofer LinkedTV mobile screen wipes for IBC 2014

August 2014

26 conferences UMONS Presentation at Interain 2014 July 9, 2014 Chicago, United States Scientific Community 50 International

27 press releases Fraunhofer Fernsehen und Internet miteinander verknüpft

July 1, 2014 http://www.fraunhofer.de/de/presse/presseinformationen/2014/Juli/Fernsehen_und_Internet.html

Industry, Scientific Community, Media

Germany

Page 60:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 60/64

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

28 press releases Fraunhofer Linking television and the Internet July 1, 2014 http://www.fraunhofer.de/en/press/research-news/2014/July/Linking_television.html

Industry, Scientific Community, Media

Europe

29 workshops UMONS Presentation at International Workshop on Interactive Content Consumption at TVX 2014

June 25, 2014 Newcastle upon Tyne, UK Scientific Community Europe

30 conferences CWI TVX 2014 June 25-27, 2014 Newcastle, UK Scientific community 200 International

31 posters MODUL LinkedTV Poster June 2014 shown at different events

32 conferences EURECOM, MODUL, CERTH

ESWC2014 May 25-29, 2014 Heraclio, Greece Scientific Community ~250 International

33 conferences CERTH IEEE International Conference on Acoustics, Speech, and Signal Processing

May 4-9, 2014 Florence, Italy Scientific Community ~2500 International

34 presentations EURECOM, CERTH ICMR'14 April 7-11, 2014 Glasgow, Scotland Scientific Community 200 International

35 conferences EURECOM, MODUL, CERTH

Speakers at the MediaMixer tutorial on Mixing Media on the Web at WWW2014 Conference

April 2014 Seoul, Korea Scientific Community 25 International

36 conferences EURECOM, MODUL, CERTH

Organizer of the Micropost NEEL challenge in the WWW2014 Conference

April 2014 Seoul, Korea Scientific Community 40 International

37 conferences EURECOM, MODUL, CERTH

Participants in the SNOW challenge in the WWW2014 Conference

April 2014 Seoul, Korea Scientific Community 25 International

38 presentations CWI BBC Connected Studios January 23, 2014 Belfast Scientific community, Industry, Medias

50 United Kingdom

39 conferences UMONS Presentation at the Multimedia Modeling Conference, MMM 2014

January 8, 2014 Diublin, Ireland Scientific Community 100 Europe

40 conferences CERTH, MODUL, EURECOM

MMM'14 and WMPA'14 January 6-10, 2014

Dublin, Ireland Scientific Community ~100 International

41 publications S&V "Future Media: LinkedTV en AXES" Trends en ontwikkelingen in crossmedia 2013-2014.

January 2014 Hilversum, The Netherlands Industry 750 books distributed 884 digital access statistics

The Netherlands

42 exhibitions Many partners, RBB

ICT 2013, Joint Exhibition November 6-8, 2013

Vilnius, Lithuania Scientific community ~5000 Europe

Page 61:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 61/64

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

43 Workshop CERTH ITI-CERTH participation to TRECVID 2013 Workshop

November 1, 2013

Gaithersburg, MD, USA Scientific Community, Industry, Policy makers

100 International

44 workshops CERTH TRECVID 2013 Workshop November 2013 Gaithersburg, Maryland, USA Scientific Community, Industry, Policy makers

~100 International

45 presentations CONDAT CSW - Corporate Semantic Web November 2013 Potsdam , Germany Scientific Community, Industry 35 Germany

46 exhibitions NOTERIK Bits&Chips 2013 Embedded Systems, booth

November 2013 's-Hertogenbosch, The Netherlands

Industry 250 The Netherlands

47 presentations CWI Eindhoven University November 2013 Eindhoven, The Netherlands Scientific community 30 The Netherlands

48 presentations S&V FIAT/IFTA 2013, presentation October 25-28, 2013

Dubai, United Arab Emirates Industry 200 International

49 workshops EURECOM Workshop on Event-based Media Integration and Processing

October 21-22, 2013

Barcelona, Spain Scientific Community 50 International

50 workshops EURECOM 2nd International Workshop on Socially-Aware Multimedia

October 21, 2013 Barcelona, Spain Scientific Community 75 International

51 workshops CERTH, EURECOM Social Event Detection 2013 (SED) @ MediaEval 2013

October 19, 2013 Barcelona, Spain Scientific Community ~100 International

52 workshops EURECOM, CERTH, Fraunhofer, UEP

Search and Hyperlinking Task @ MediaEval 2013

October 18, 2013 Barcelona, Spain Scientific Community ~100 International

53 presentations CONDAT Medientage 2013 October 16-18, 2013

Munich, Germany Industry ~300 Germany

54 thesis CWI "LinkedTV News: designing a second screen companion for web-enriched news broadcasts" PDEng Thesis, Eindhoven University of Technology

October 2013 Eindhoven, The Netherlands

55 presentations NOTERIK Mindtrek Festival 2013 October 1-3, 2013 Tampere, Finland Scientific Community, Industry 500 Europe

56 exhibitions CONDAT IBC 2013 September 17-19, 2013

Amsterdam, Netherlands Medias, Industry, Other A potential of about 30 000

Europe

57 conferences CERTH IEEE ICIP 2013 September 15-18, 2013

Melbourne, Australia Scientific Community ~1500 International

58 presentations CWI Interactive Knowledge Systems at UvA September 2013 Amsterdam, The Netherlands

Scientific community 30

Page 62:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 62/64

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

59 workshops UMONS, UEP Participation eNTERFACE workshop July 15 - August 9, 2013

Lisbon, Portugal Scientific Community 50 Europe

60 conferences UMONS Organisation INTETAIN 2013 July 3-5, 2013 Mons, Belgium Scientific Community 50 Europe

61 conferences CWI INTETAIN 2013 July 3-5, 2013 Mons, Belgium Scientific community 100 International

62 workshops STI, EURECOM, CERTH

FutureTV workshop @ EuroITV 2013 June 24, 2013 Como, Italy Scientific Community 40 International

63 workshops CWI LinkedTV FutureTV workshop June 24, 2013 Como, Italy Scientific community 30 International

64 presentations S&V, CWI, NOTERIK

Mediapark Jaar Congres 2013 June 20, 2013 Hilversum, The Netherlands Industry 250 The Netherlands

65 presentations S&V, CWI, NOTERIK

Volt! Dutch Technoogy Week June 3, 2013 Eindhoven, The Netherlands Industry 150 The Netherlands

66 conference NOTERIK "Enriched Multiscreen TV Experience" In 11th European Interactive TV Conference (EuroITV 2013), Demo Track

June 2013 Como, Italy Industry, Scientific Community 500 Europe

67 presentations EURECOM ESWC 2013 May 26-30, 2013 Montpelier, France Scientific Community 350 International

68 workshops EURECOM 2nd International Workshop on Real-Time Analysis and Mining of Social Streams (RAMSS 2013)

May 14, 2013 Rio de Janeiro, Brazil Scientific Community 30 International

69 workshops CERTH, EURECOM, S&V, STI, UMONS

1st Worldwide Web Workshop on Linked Media (LiME 2013)

May 13, 2013 Rio de Janeiro, Brazil Scientific Community ~100 International

70 workshops EURECOM 2nd International Workshop on Web of Linked Entities (WoLE 2013)

May 13, 2013 Rio de Janeiro, Brazil Scientific Community 40 International

71 conferences CERTH EURECOM International Conference on Multimedia Retrieval (ICMR 2013)

April 16-19, 2013 Dallas, Texas, USA Scientific Community ~150 International

72 presentations STI 1st Latin American Linked Data Meetup March 21, 2013 Cuenca, Ecuador Scientific Community 20 Latin American

73 flyers STI project factsheet March 2013 Scientific Community, Industry, Medias

Europe

74 flyers STI project postcard March 2013 Scientific Community, Industry, Medias

Europe

75 flyers STI project flyer March 2013 Scientific Community, Industry, Medias

Europe

76 presentations S&V iMMovator "Uit het Lab" February 12, 2013 Hilversum, The Netherlands Industry 150 The Netherlands

Page 63:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 63/64

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

77 workshops CERTH SMAP 2012 : 7th Semantic and Social Media Adaptation and Personalization Workshop

December 3-4, 2012

Luxembourg city, Luxembourg

Scientific Community ~50 International

78 workshops EURECOM 1st International Workshop on Web of Linked Entities (WoLE 2012)

November 11, 2012

Boston, USA Scientific Community 40 International

79 Workshop CERTH ITI-CERTH participation to TRECVID 2012 Workshop

November 2012 Gaithersburg, MD, USA Scientific Community, Industry, Policy makers

100 International

80 presentations CWI ICTDelta 2012 October 24, 2012 Roterdam, The Netherlands Industry 100 The Netherlands

81 conferences Fraunhofer, CERTH, EURECOM, UEP, RBB

NEM Summit 2012 October 16-18, 2012

Istanbul, Turkey Scientific Community ~400 International

82 workshops CWI HBB-NEXT Synchronisation Workshop (ICIN 2012)

October 11, 2012 Berlin, Germany Scientific Community 30 Europe

83 presentations CWI CWI in Bedrijf 2012 October 5, 2012 Amsterdam, The Netherlands

Industry 50 The Netherlands

84 workshops CERTH, EURECOM Social Event Detection 2012 (SED 2012) October 4-5, 2012 Pisa, Italy Scientific Community ~100 International

85 presentations EURECOM TeleTask Symposium October 2012 Potsdam, Germany Scientific Community International

86 flyers STI project newsletter October 2012 Scientific Community, Industry, Medias

200 Europe

87 workshops STI STUBA Workshop September 20-21, 2012

Bratislava, Slovakia Scientific Community 30 Europe

88 workshops CONDAT W3C Germany Office workshop on HTML5, Xinnovations conference

September 2012 Berlin, Germany Industry, Scientific Community 60 Germany, Europe

89 workshops STI, CERTH, EURECOM, CWI

3rd International Workshop on Future Television during the annual European conference on Interactive Television (EuroITV 2012)

July 4-6, 2012 Berlin, Germany Scientific Community 30 International

90 presentations STI, CWI, S&V, NOTERIK

EuroITV July 4-6, 2012 Berlin, Germany Industry, Scientific Community 500 Europe

91 conferences CERTH EURECOM International Conference on Multimedia Retrieval ICMR 2012

June 5-8, 2012 Hong Kong Scientific Community ~150 International

92 conference CWI "Antiques Interactive" Proc. EuroITV demo session

June 2012 Berlin, Germany Industry, Scientific Community 500 Europe

Page 64:  · Deliverable 9.3 Final Project Report. PROJECT FINAL REPORT . Grant Agreement number: 287911 . Project acronym: LinkedTV . Project title: Television Linked To The Web . Funding

Final Project Report D9.3

© LinkedTV Consortium, 2015 64/64

NO Type of activities12 Main leader Title Date/Period Place Type of audience13 Size of

audience Countries addressed

93 presentations RBB Cross Media Conference 2012, Think CROSS - Change Media 2012

March 30, 2012 Magdeburg, Germany Medias, Scientific Community 50 Germany

94 conferences STI SemTechBiz 2012 February 2012 Berlin, Germany Industry 80 International

95 web STI www.linkedtv.eu/feed/ November 2011

96 web STI twitter.com/linkedtv November 2011 220 followers

97 web STI https://youtube.com/user/LinkedTVeu November 2011

98 web STI www.linkedtv.eu November 2011

99 web STI slideshare.net/linkedtv November 2011

100 web NOTERIK http://noterik.github.io Industry, Civil Society International

101 web CWI http://linkedtv.project.cwi.nl