Workshopvin1 2018 Technology In Labour For Rebirth Of Hal 9000 Series

2018: Technology in Labour for Rebirth of HAL-9000 series

Some contributions of ETRO

2

The HAL-9000 series (1968)

2001: A Space Odyssey - Stanley Kubrick, Arthur C. Clarke

The spaceship discovery

HAL-9000

3

HAL-9000 Ambient Intelligence System (1968)

HAL-9000

Monitors its surroundings, the ship and its crew Analyses sensors, images and soundsConverses in natural spoken languageDesigned to assist the user

4

AMI-9000 Ambient Intelligence System (2018)

AMI-9000

Monitors its surroundings, the ship and its crew Analyses sensors, images and soundsConverses in natural spoken languageDesigned to assist the user

5

AMI-9000

AMI-9000 Ambient Intelligence System (2018)

Monitors its surroundings, the ship and its crew Analyses sensors, images and soundsConverses in natural spoken languageDesigned to assist the userIntelligence: expression/emotion/body language awareness

6

Current research @ ETRO-DSSP

Speech modificationtime scaling, intelligibility enhancement, automatic lip synchronization, voice conversion

Speech synthesisFlemish TTS, hierarchic TTS, AV TTS, expressive speech synthesis (emotions)

Noise suppression (single channel, contact microphones)robust speech recognition

Microphone array techniquesdistant recording, blind source separation

Speech disordersdiagnostics, remedy

7That’s Right

AIBOSBA

Emotion Recognition

Emotions in speechalter pitch, timing, voice quality and articulation.

Emotional speech rec.classify statistical measures of acoustic features into classes.

Two approachesSegment Based (SBA) & AIBO (Sony)

8

X-database study of emotion recognition

Obtained state-of-the art recognition scoresEmotion recognition = database dependentClassifiers can be learned on joined databaseUse of higher level features might helpUse arousal detection as case study

24%46%42%35%Baseline X-DB

23%53%45%54%X-DB

51%34%42%32%Baseline

54% H:67%Human 85%67%82%Literature

64%75%69%87%Best score

DanishBerlinBabyEarsKismet

Go to Demo

9

Current research @ ETRO-IRIS

USER NATURAL INTERACTION

-motion-speech

-expression

USER INTERACTION SYNTHESIS

- estimating the facial animation parameters

- face model adaptation

- mouth visualization

- emotion feature extraction

- audiovisual speech segmentation

- morphing a 3D head

- data-driven feedback

- animating an avatar

- detection & tracking of body and face

USER INTERACTION ANALYSIS

10

Facial Analysis & Synthesis

GesturesMotion estimationPose and structure variationsEye gaze and expressions

Multi-modal speechEnhancement by mouth images Animated avatar

and takes into account the natural face motions

11

Expression Analysis

Facial Action Coding Systema muscle-based method to measure facial movements w.r.t. Facial anatomy , widely used in Psychology

Each Action Units (AU) represents one visibly distinguishable facial change (46 AUs for facial appearances, 12 AUs for gaze direction and head pose).

Face expression = Co-occurrence of several AUsA parametric model combining several AU’shas been built for expression analysis

What is hidden behind a face expression?the temporal course of the muscle activities (intensity of muscle contraction/relaxation versus time). Information to recognise concealed emotions (e.g. deception). AUs are purely descriptive. FACS provides a dictionary to interpret the corresponding emotions

recognition/synthesis

Visual input

Face processing unit

Expression processing unit

AngerSurprise

12

AMI-9000 Ambient Intelligence System (2018)

Smart buildings (surveillance)Care for the elderly at home (assisting, security)Personalised personal assistants (understand the user)VIN: adapt according to the user’s state-of-mindEmbodied conversational agents for education