Introduction (Motivation) · GPS information etc.) Visual + Metadata. 4 Benchmark Datasets: State-of-the-art Current datasets for event detection in images low number of images (e.g.,

1

Introduction(Motivation)

In2015,atotalof728millionsofpublicpictureswereuploadedtoFlickr

Suchlargeamountof user-generateddatamakesmultimediaindexingandretrievalamorechallengingtask

However,italsoopensnewopportunitiesfordevelopmentofnovelandmoreefficienttools

2

Introduction(Motivation)User-generated multimedia contents depictindividual experiences or collective activities

WhatisanEvent?

Arealworldhappening toWho?,What?,When?andWhere?

Aneventisplannedbypeopleattendedbypeopleandrelatedmediaarealsocapturedbypeople

Personalexperiences

Collectiveactivities

3

EventDetectioninImages:State-of-the-art

VisualInformation

Metadata(tags,GPSinformation

etc.)

Visual+Metadata

4

BenchmarkDatasets:State-of-the-art

Currentdatasetsfor

eventdetectioninimages

lownumberofimages(e.g.,EIMM[1],Cultural

eventrecognitiondatabase[3])

limitedvarietyofevents/eventclasses(e.g.,EiMM [2]andSED2013

database[2])

Unbalancedeventclasses(e.g., EiMM [1]andSED2013[2])

1. R.Mattivi etal..Exploitationoftimeconstraintsfor(sub-)eventrecognition.InProceedingsofthe2011jointACMworkshoponModelingandrepresentingevents,pages7(12).ACM,2011..

2. T.Reuteretal..Socialeventdetectionatmediaeval2013:Challenges,datasets,andevaluation.InMediaEval Workshop,2013..3. S.Escalera etal..ChaLearn LookingatPeople2015:ApparentAgeandCulturalEventRecognitionDatasetsandResults,ICCV2015

5

USED:AlargeScaleSocialEventDetectionDatasetAlargecollectionofimages

Covers14differenteventsclasses

AbalanceddatasetEqualnumberofimagesineachclass(35,000)

Event-classesinUSEDDataset

6

USED:AlargeScaleSocialEventDetectionDataset

DiversityincontentsIndoorVs.outdoorGrouppicturesVs.SingleportraitImagesofkey-momentsinaneventMulti-culturalOutliersandborderlinecasesaremanuallyremoved

Somesampleimagesfromweddingclass

7

USED:AlargeScaleSocialEventDetectionDataset

USED490,000 Eventrelated

imagesdepictinga widevarietyof

events

8

Comparisonswithstate-of-the-artdatasets

ExistingdatasetsforEventDetectionCulturalEventDetectionDatasetEiMMSED

DatasetName #Event-classes Total Images Minimagesinaclass

Max.images inaclass

EiMM 8 (socialevents) 13219 795 2253

SED 7 82213 342 71556

CulturalEvents 50 11776 180-200(Avg.) 180-200(Avg.)

USED 14 490000 35000 35000

Comparisons ofUSEDwithotherDatasets

9

ExperimentalValidationofUSED

DISCOVERINGEVENTSFROMSINGLEPICTURESUSINGACONVOLUTIONALNEURALNETWORK

10

Validation/ExperimentalSetup

Fine-tuningCNN

Classification

Pre-training

ParametersofaCNN(Alexnet)pre-trainedonImageNet dataset

[NIPS2012]

Fine-tunedonnewlycollecteddatasets

Reduced overalllearningrateIncreasedlearningrateof

newlayerMomentum=.9

WeightDecay=.0005

11

PreliminaryResultsDataset

USED

Event Type Accuracy EventType Accuracy

Concert 74.20% Conference 75.70%

Graduation 66.43% Exhibition 58.54%

Meeting 78.70% Fashion 65.43%

MountainTrip 67.00% Protest 74.58%

Picnic 54.42% Sports 72.24%

Sea-holiday 74.24% Theater 51.90%

Ski-holiday 48.00%

Wedding 51.00%

ResultsonUSEDdataset

DataAssemblageTrainingset=20,000imagesperclassValidationset=7000perclassTestset=7000imagesperclass

12

ComparisonsofaCNNtrainedonUSEDwithBaselineApproaches

ComparisonwithRosani etal.,[IEEETMM2015]

EiMMDataset SEDDatasetOurApproach 71.54 59.42BaselineApproach 38.8 31.15

0

10

20

30

40

50

60

70

80Ac

curacy(%

)

A.Rosani,G.Baoto,F.G.B.DeNatale,“EventMask:agame-basedframeworkforEvent-saliencyidentificationinImages”,IEEETransactionsonMultimedia2015

13

USED:ALarge-scaleSocialEventDetectionDataset

490,000 Event-relatedimages, 14differentevent-classes,35,000imagesper

class

ENJOY USED!