Face2mus 1437580648936

1

2

Guided by, Amol Joy (Assistant professor) ECE department

Date of Presentation 20 August 2015

Presented by, Ann Mary Thomas S7 ECE A Reg.no:12010492 Roll no: 121019

FACE 2 MUSIC

3

• Introduction• Objective• How to tract facial features in images• How to tract emotions in songs• FACE 2 MUSIC• Work flow for FACE 2 MUSIC• Experimental result• Image data base• Image classification• Audio data base• Audio classification• Over all performance of FACE 2 MUSIC• Conclusion• Reference

OUTLINE OF THE PRESENTATION

4

• smart phone has introduced mobile application.

• Used in therapeutics.

• Emotion sensitive music app would be of high value.

• Image-based emotion recognition.

INTRODUCTION

5

• the entire topic is divided into two sections: part A and part B. Part A consist of ; i. How to tract facial features in images ii. How to tract emotions in songs Part B consist of; i. Workflow for Face 2 Music ii. Experimental result iii. Overall performance

contd…

6

• Introduce a mobile application .

• sense the emotions of people for their better health.

• To provide better Human-machine interactions

OBJECTIVE

7

PART A

8

1.From extracted Action Units(AUs).

2.From Motion Units(MUs).

HOW TO TRACT FACIAL FEATURES IN IMAGES ?

9

• Developed by Ekman.

• Describes a specific feature in the face.

• It achieved accuracies ranging from 64.29% to 100%.

1.From extracted Action Units(AUs)

10

• Describes the direction of motion of feature and its concentration.

• Computationally expensive and person dependent.

• Hence, AU’s would be potentially better .

2.From Motion Units(MUs)

11

• Many theories developed to assign emotions or moods to

songs.

• Some of them are, 1. Hevner’s theory 2. Russell’s axes 3. Thayer’s axes

HOW TO TRACT EMOTIONS IN SONGS ?

12

• Some include tempo, loudness and harmony

• Low level sound descriptors and several open source libraries are available.

• Here, implemented a hybrid music and theme classifier.

Contd...

13

PART B

14

• It is an android application.

• It streams music from online radio stations .

• Emotion is deduced by emotion recognition techniques .

FACE 2 MUSIC

15

WORK FLOW FOR FACE 2 MUSIC

16

FACIAL FEATURES

17

• Tested on a Samsung Galaxy Tab 10.1 model.

• Server running on a 64 bit Intel core i5 processor.

• Developed using Android 3.1 API .

• Image processing and classification code was written in Matlab R2011b.

EXPERIMENTAL RESULTS

18

• Testing results of the audio and image classifiers are presented separately.

A. Image data base B. Image classification C. Audio database D. Audio classification E. Overall Face 2 Music performance

Contd...

19

• Image classifiers were trained on a home-made data base .

• Data base is composed of 120 images of 8 people.

• 5 images for each acted emotion per person.

• Images were classified in to 3 classes, i. Neutral ii. Sad iii. Happy.

1. IMAGE DATA BASE

20

SAMPLE IMAGES FROM THE DATA BASE

21

• Trained and tested on neural network and SVM classifiers.

• A grid search was performed to find the best classifier .

• Majority vote classifiers, given different feature sets.

• It is composed of 1 vs.1 SVM classifiers using RBF kernel.

2. IMAGE CLASSIFICATION

22

• Most of the misclassified images were from the sad class.

• Best classifiers were saved and used to classify the images .

• The feature set containing distances produced best results.

Contd...

23

24

• Classifier architectures were trained and tested on the Million song dataset subset.

• It included 10,000 instances and 4 features.

• Instances were clustered using k-means clustering algorithms.

3. AUDIO DATA BASE

25

• A neural network and an LM-SVM classifiers were trained.

• A grid search was performed to find optimal box constraint and kernel parameters.

• Training was validated using 5-fold, cross fold validation.

• Best models of various classifiers were saved on android tablet.

4. AUDIO CLASSIFICATION

26

5. OVERALL FACE 2 MUS PERFORMANCE

• The size of the application is 272KB.

• Uses 12 to 24 MB of RAM .

• Start up time is approximately 2 minutes.

• Time can vary based on the internet connection.

• To reduce start up time, run audio processing blocks on server.

27

28

• Proposed an automated online song streaming application .

• Emotions detected using facial feature identification, tracking and classification .

• Accuracy can be enhanced by more robust features.

• Real time performance can be improved.

CONCLUSION

29

• M. Walsh (2011, October 7). Mobile App Biz Soars: $12B by 2015.

• The Independent (2011, November 9). Mobile application

trends for 2012: the top ten applications. • J. Imam (2012, June 16). Young listeners opting to stream, not

own music.

• C.E. Guzzetta. “Effects of relaxation and music therapy on patients in a coronary care unit with presumptive acute myocardial infraction.” Heart & Lung: The journal of critical

care, vol.18, issue 6, p.609, 1989.

REFERENCES

THANK YOU

31