11 Introduction Recently, the mental and physical diseases caused by negative emotions and stress are increasing year by year. The physiological problems.

11

IntroductionIntroduction

Recently, the mental and physical diseases caused by negative emotions and stress are increasing year by year.

The physiological problems of 90 percent are related with mental factor.

Many emotion recognition techniques have been proposed.

Facial Expression Recognition:Using the relationship between the facial features for facial expression recognition.

Physiological Emotion Recognition:Using the physiological signals to recognize emotions.

22

Introduction- facial expression recognition Introduction- facial expression recognition

Most of automatic facial expression recognition systems have two characteristics:

Extract features in gray scale images.Recognize from general expression model.

Expression may be expressed differently by different people.

Therefore, we proposed a personalized facial expression recognition system combine with face recognition system and facial expression recognition system.

Expressionmodel #1

Expressionmodel #2

Expressionmodel #n

.

.

.

Personalized facial expression recognition system

Facial expressionrecognition

Facerecognition

Facial featureextraction

33

Introduction - physiological signalsIntroduction - physiological signals

Physiological reactions are non-autonomic nerves in physiology. The physiological reactions and the corresponding signals are hardly to control while emotions are excited.

The physiological reaction of emotion is generated similarly in different people.

Therefore, we proposed a emotion recognition system, combine with support vector regression and physiological signals.

44

Feature extraction: To extract features of face image. We extract two kinds of features: facial feature distances and facial edge features.

Face recognition: Use the method of Chang [9] to obtain the personal identity of user.

Face detection: Use the method of Chang[8] to extract face image in original image.

Original image: Use web camera to catch original image.

Expression recognition: Use radial-basis function neural network to recognize expression by personalized information.

Pre-processing: Face image normalization, pupil detection and Gabor transform are involved.

System overview: Facial Expression RecognitionSystem overview: Facial Expression Recognition

Original Image

Face Detection

Face Recognition

Expression 　Recognition

Result

Feature Extraction

Facial feature distances

Facial edge features

Pre-processing

55

Methods - Face detectionMethods - Face detection

Face detection ( 最佳論文獎 TAAI2006) Expression recognition depends on robust face detection and tracking.

We adopt an adaptive color space switching method proposed by Chang to detect face image.It can detect multiple faces and mark face regions automatically under complex background environment and variable lighting condition.The algorithm was validated under different human behavior and environmental variations such as camera motion, background change, object motion and brightness variation.

Samples of face detection (a) in color image; (b) in gray scale image; (c) in multiple face image

(a) (b) (c)

66

A Subject-dependent Facial Expression Recognition SystemA Subject-dependent Facial Expression Recognition System

Extracting significant facial features is important in the design and implementation of automatic facial expression systems

77

Methods - Feature extraction Methods - Feature extraction

Facial feature distances16 descriptive points were detected using empirical information in two color space (YCbCr and HSV).

17 feature distances are used to describe face changes. These distances are denominated D1, D2,..., D17.

88

Methods - Feature extractionMethods - Feature extraction

Facial edge featuresTo obtain the edge image, the average Gabor image is convoluted with a Sobel edge detection mask.On the edge image, 16 blocks are captured according to the detected facial feature points and compose the ”inner face region”.

where Bi(x,y) is the i-th block’s intensity of (x,y). bw and bh are the width and height of each block. Blocks is the number of blocks, we set as 16.

Blocks

i

bh

y

bw

x

ibh

y

bw

x

ii Blocksbwbh

yxB

bwbh

yxBF

1 0 00 0 **

,

*

, (1)

99

Methods - Face recognitionMethods - Face recognition

The personalized expression recognition is achieved by identifying a user’s face before expression recognition.

Chang’s method [5] was adopted. ( 佳作論文獎 TAAI2008)

However, in Chang’s method, a full face image, such as the extracted face images shown in Fig. (a), was used.

Since the background and hair significantly affect recognition, we used the inner face to identify the user, as show in Fig. (b).

(a) (b)

1010

Methods - Face recognitionMethods - Face recognition

Challenges in face recognition include illumination variation, pose variation, facial expression, aging, hair, and glasses.

Contributions of the proposed face recognition method

We used Gabor filters to obtain Gabor faces that have properties of scale normalization and grayscale equalization.

An AdaBoost committee machine is used to promote the recognition rate.

The Radial Basis Function Neural Network was adopted as the weak classifier.

The centers of RBFNN were adaptively selected using PCA.

A novel weight updating mechanism was applied to reduce the training time.

The proposed method has a high recognition rate and requires a short training time.

The proposed method can attenuate the influences of illumination, facial expression, and pose variations.

1111

Methods - Face recognition IntroductionMethods - Face recognition Introduction

Variations in illumination and facial

expression Pose variation

(a) (c) (e)

(b) (d) (f)

Variations in illumination, pose and facial expression seriously affect the detection of invariant salient features.

1212

Methods - Face recognition IntroductionMethods - Face recognition Introduction

There are two scenarios for face recognitionPure Face Recognition

Identify or verify a person from a digital image or a video frame.智慧數位宅 看臉色開門 (2008/05/01/ 蘋果日報 )

Integrated the RFID and face recognition for ID authentication.

電子投票 ( 國立雲林科技大學自由軟體研究中心 )2007/08/02~2007/08/06 台北世貿電腦應用展

U 化餐廳 ( 國立雲林科技大學自由軟體研究中心 )2008/07/30~ 2008/08/02 台北世貿電腦應用展

智慧生活空間人臉辨識門禁系統2008/11/29 ( 國立雲林科技大學校慶記者會 )

1313

智慧數位宅 看臉色開門 (2008/05/01/ 蘋果日報 )智慧數位宅 看臉色開門 (2008/05/01/ 蘋果日報 )

「數位住宅的概念讓房子有了智慧，屋主進門不須用鑰匙，只要人臉感應即可」

最近建築業者耗資 3000 萬元，在接待中心設立未來概念館，將進階導入一般住宅。 愈來愈多新建案規劃「數位住宅」，提升未來居住品質，除了可利用手機感應在社區內購物，還可以設定人員追蹤，從網路上寮解家人的所在位置。昨日遠雄建設（ 5522 ）聯合 12 家科技廠商，在林口「大未來」接待中心，成立「遠雄 2015 未來生活概念館」。

12 科技廠打造概念屋 遠雄企業團董事長趙藤雄說：「 2015 年的住宅科技可以在生活概念館體驗，未來也會逐步導入遠雄的建案中。」遠雄副理劉純信表示，未來生活概念館由國內外科技大廠聯手打造，包括三聯科技、立皓科技、永奕科技、台灣國際松下電工、台灣飛利浦照明事業部、資策會、富陽光電、傳技資訊、精誠資訊、滿景資訊、德凌資訊、韓國 S-Tec （台灣三星物產代理）等，共展示 17 項數位生活設施。有興趣的民眾都可以到「大未來」接待中心感受未來住宅科技。

電影場景在日常實現 遠雄 2015 未來生活概念館可提供的實境有 5 大主題，包括智慧型無人商店，住戶在購物時，用手機感應就能付款；社區數位信箱提供低溫宅配物品，住戶可以使用手機感應付費領取物品；住戶進入梯廳時，電腦會自動儲存人臉紀錄，可以不用鑰匙，就進入大門等等。這些如同在電影場景的科技，都可以運用在生活中。

遠雄建設在林口「大未來」接待中心，成立未來生活概念館，體驗 2015 年的住宅科技。

1414

Methods - Face recognitionMethods - Face recognition

Block diagram of the proposed method

1515

Gabor wavelet transformThe Gabor filter transforms an image into a fixed normalized size and eliminates the influence of illumination variance.

Gabor filter

Methods - Face recognitionMethods - Face recognition

22 2sk

Eight orientations （ = 0, /8,2 /8,….,7 /8 ）

Fiv

e s

ca

les

（s

= 0

,1, 2

, 3,4

）

2)(2

)(

2

2

,

2

222

222

),(

eeek

qpg qpikqpk

s

where

s = 0, 1, 2, 3, 4

= 0, /8, 2/8, ..., 7/8

1616

總結總結提出了一套結合人臉辨識的個人化人臉表情辨識系統。

藉由人臉偵測技術由輸入影像中擷取出目標人臉，再以人臉辨識技術來辨識出使用者的身分擷取使用者臉部的各個特徵依人臉辨識的結果與取出的臉部特徵組成特徵向量使用類神經網路辨識無表情、快樂、生氣、驚訝及悲傷等五種表情。實驗結果顯示提出的方法能夠正確地辨識人臉表情。

1717

Mel-Frequency Cepstrum CoefficientMel-Frequency Cepstrum Coefficient

The output of the filter bank by Y(m), the MFCCs are calculated as

where M is number of band-pass filter, c(k) is the kth coefficient of MFCC, P is number of MFCC coefficient, m is the index of band-pass filters.

PkM

mkmYkcM

m

,...,2,1 ,2

1coslog

1

(22)

1818

Linear Predictive Cepstrum CoefficientLinear Predictive Cepstrum Coefficient

In the Linear Predictive Cepstrum (LPC) analysis of audio each sample is predicted as linear weighted sum of the past p samples, where p represents the order of prediction.

where S(n) is the present sample; Ai is the ith linear combination coefficient.

The difference between the actual and the predicted sample value is termed as the prediction error.

Signal

Frame Segmentation Auto-correlation

Durbin algorithm LPC coefficient

LPCC coefficient

P

ii

inSAnS1

(23)

1919

Linear Predictive Cepstrum CoefficientLinear Predictive Cepstrum Coefficient

Using auto-correlation approach to reduce prediction error, and defined as

where x(n) is the original signal; R(k) is the auto-correlation function; p is the order of prediction.

Using the Durbin algorithm to acquire LPC coefficients, and using equation (25) to acquire LPCC coefficients.

where m is the LPC coefficient; cm is the LPCC coefficient.

1,...,1,0 ,1

0

PkknxnxkR

N

n

otherwise1

11

1

1

1

P

kkmk

m

kkmkm

m

cm

k

Pmcm

k

c

(24)

(25)

2020

The sequential floating forward selection algorithm is utilized to find discriminative features, and is a revised algorithm based on “sub-optimal” feature subset selection.

Sequential Floating Forward Selection Sequential Floating Forward Selection

Accuracy

Network