Overview of the MediaEval 2014 Visual Privacy Task

Visual Privacy Task Overview

Atta Badii, Touradj Ebrahimi, Christian Fedorczak, Pavel Korshunov,Tomas Piatrik, Volker Eiselein, Ahmed Al-Obaidi

Task Description

• To explore the possibilities to optimise the process of privacy filtering so as to:

1. obscure personal visual information effectively

whilst,

2. keeping as much as possible of the ‘useful’ information that would enable a human viewer to interpret the obscured video frame.  

Slide 2

Participants

VPT 2014: VPT 2014: 1111 registrations, registrations, 88 participants participants , ,

PEViD Dataset

21 annotated videos• ~20s• 25fps• 1920x1080• Indoor/Outdoor • Daytime/Evening • Featured actions include: Bag dropping, Pickpocketing, Fighting,

or simply Walking • Ground truth consists of: annotated People, Faces, Accessories,

Hair regions, Skin regions

Development Set = 9 videos

Evaluation Set = 12 videos

Privacy sensitive region

Sensitivity level

Skin Medium (M)Face High (H)Hair Low (L)Accessories Medium (M)Person Low (L)

Subjective Evaluation Streams

The privacy protected video clips were evaluated by each of three distinct communities of human evaluators:

i) Naive viewers (engaged via a crowdsourcing platform)• 290 workers responded, 230 were found to have provided

reliable responses

ii) Video-analytics technology and privacy protection solutions developers

• focus group consisted of (65) participants, (15) of them were females

iii) Surveillance monitoring staff and law enforcement staff• focus group comprised of (59) participants including (22)

females

Slide 5

Evaluation Setup

• Six (6) video clips were pre-selected from each submission and evaluated using the three (3) evaluation streams.

• A Questionnaire consisting of 12 questions had been carefully designed to examine aspects related to privacy, intelligibility, and pleasantness; this was used in stream 2 and 3.

• The First (5) questions were aimed at eliciting the opinions of the evaluators re the Contents of the viewed videos. The responses to these questions were considered with respect to the ground truth.

• The rest of the questions were aimed at eliciting the Subjective Opinions of the evaluators re the viewed videos.

• Stream 1 used a shortened version of the questionnaire with (7) questions in total due to crowdsourcing constraints.

Slide 6

UI-REF based criteria

• Privacy Protection Level – How adequate was the level of privacy protection achieved by the filter across all testing video clips?

• Level of Intelligibility – How much ‘useful’ information that was retained in the video frames after privacy filtering had been applied?

• Pleasantness of the resulting privacy filtered video frames in terms of their ‘aesthetic’ perceptual appeal to human viewers. How acceptable were any adverse aesthetic effects?

Perceived Effects, Side Effects, Cross-Effects, Affects

Slide 7

Stream 1 results

Slide 8

Stream 2 results

Slide 9

Stream 3 results

Slide 10

Median of the 3 streams

Slide 11

Thank You