IMPROVED FACE TRACKING THANKS TO LOCAL FEATURES CORRESPONDENCE Alberto Piacenza, Fabrizio Guerrini, RiccardoLeonardi Department of Engineering Information – University of Brescia, Italy Dataset of YouTube shots with faces (average: 181 frames) FACE TRACKING ENHANCEMENT OVERVIEW Apply the face track enhancement stage SOLUTION Semantic description of the content in the Interactive Movietelling system [1] MOTIVATION Identify the frames in which a main character is present SPECIFIC CHALLENGE Use off-the-shelf tools for: 1) face detection 2) face recognition on the detected faces BASELINE SOLUTION 1) Imprecise or missed face detection 2) Face bounding box drifting PROBLEMS Character recognition is unreliable EFFECTS Flowchart of the operations involved in the creation of the enhanced face tracks. Output tracks: the frames of a small excerpt of one shot are presented. Blue rectangles: detected faces correctly identified in a given face track. but the face detection has failed to find the face in the in-between frames. Green rectangles: recovered faces for in- between frames thanks to the face tracks enhancement process. • Re-extract POI in the bounding box • Use KLT tracker to the next frame • Estimate the new bounding box using RANSAC • Use backward tracking as well : number of ground-truth objects in frame : number of detected objects in frame : -th ground-truth object : -th detected object Frame Detection Accuracy (FDA): Comparison with the CAMSHIFT algorithm EXPERIMENTAL RESULTS ADDITIONAL INFO Interactive Movietelling system: • Reference: [1] A. Piacenza, F. Guerrini, N. Adami, R. Leonardi, J. Porteous, J. Teutenberg, M. Cavazza, “Generating Story Variants with Constrained Video Recombination”, 19 th ACM Multimedia, pp. 223-232, 2011. • Link to example output clips: www.ing.unibs.it/alberto.piacenza/TrackWithPoints Acknowledgements: This work has been funded (in part) by the EC under grant Agreement IRIS (FP7-ICT-231824).