Top Banner
www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions, voice and body gestures English only, sorry
17

Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Mar 26, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

www.asc-inclusion.eu

A project to develop software to assist people with autism to recognise, understand and

express emotions through facial expressions, voice and body gestures

English only, sorry

Page 2: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

IntroductionIntroduction

• A project to develop interactive games to assist people with autism to recognise, understand and express emotions through facial expressions, voice and body gestures

• Primarily for children aged 5 to 10• Project by 7 organisations under a

European Commission funded 3-year cooperative research project (FP7 programme)

• First version will be available in late 2014 (in English)

www.asc-inclusion.eu

Page 3: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Autism & understanding Autism & understanding emotionsemotions• Children and adults with autism experience

significant difficulties recognising others’, as well as their own, emotions and mental states, including facial expressions, vocal intonation, gestures and body language, and their integration in context (Baron-Cohen, 1995; Hobson, 1993; Golan, Baron-Cohen & Golan, 2008).

www.ascinclusion.euwww.asc-

inclusion.eu

Page 4: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Autism & understanding Autism & understanding emotionsemotions• Individuals with autism also experience difficulties in

expressing their own emotions non-verbally, including the ability to:– Direct appropriate facial expressions to others

(Kasari et al, 1993)– Modulate their vocal intonation appropriately

when expressing emotion (McCann & Peppe, 2003)

– Use appropriate gestures and body language (Attwood, 1998).

– Integrate non-verbal communicative cues with speech (De Marchena & Eigsti, 2010).

• Understanding and expression of emotions is not learned spontaneously by people with autism, and therefore needs to be taught explicitly.

• Without these skills, the risk of social exclusion for individuals with autism is significantly increased.

www.asc-inclusion.eu

Page 5: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Systematic strengths of people Systematic strengths of people with autismwith autism

Individuals with autism are often hyper-attentive to detail and prefer predictable, rule-based environments and systems.

Within such environments, individuals with autism show good, and sometimes even superior systemising skills, compared to the general population (Baron-Cohen, 2003).

If provided with a system of emotions, it is plausible that systemising skills could be harnessed to help individuals with autism learn to recognise emotions (Golan & Baron-Cohen, 2006).

The computer provides individuals with autism with such a predictable, rule-based environment, enabling them to capitalise on their systemising skills.

Computerised socio-emotional training in autism has been shown to change behavior, as well as brain functioning (Bölte et al., 2006)

www.asc-inclusion.eu

Page 6: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Need for a new software Need for a new software programprogramStudies attempting to teach emotion and

mental state recognition to people with autism have shown mixed results, reporting limited generalisation from taught curriculum to situations not included in the training program.

Training of individuals with autism requires facilitation by trained specialists, which is not always available, and can be quite expensive.

In view of the growing prevalence of autism, there is an urgent need to seek new innovative methods for supporting integration of children with autism into society.

Information & communication technologies (ICT), may provide an easily available solution. www.asc-

inclusion.eu

Page 7: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

How the software will How the software will workworkThe ASC-Inclusion software will:•Analyse the children’s facial expressions, vocal intonations and gestures (using standard microphones and webcams);•Train children to recognise their own, and others’, facial expressions, tone-of-voice and body gestures through interactive games, text communication, animations, video and audio clips;

www.ascinclusion.euwww.asc-

inclusion.eu

Page 8: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

How the software will How the software will workworkThe ASC-Inclusion software will:•Personalise settings according to children’s individual needs;•Support professionals, parents and carers with professional information, reports on children’s progress and forums to interact with other professionals and carers.

www.ascinclusion.euwww.asc-

inclusion.eu

Page 9: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

How the software will How the software will workworkThe program will consist of 3 state-of-the-art subsystems for: •Facial expressions•Vocal intonation•Body gesturesThe subsystems will be integrated together in an online virtual world platform to provide the user with a comprehensive and motivating learning experience.

www.ascinclusion.euwww.asc-

inclusion.eu

Page 10: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

How the software will How the software will workworkFacial expressions

The ASC-Inclusion software will develop this existing system to recognise the user’s facial expressions.

www.ascinclusion.euwww.asc-

inclusion.eu

Example video of facial expression

recognition

Page 11: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

How the software will How the software will workworkVocal intonation

The ASC-Inclusion software will develop this existing system to recognise vocal intonation and adapt it for a learning oriented environment. (Note the unfriendly character used in this example video will NOT be used in the ASC-Inclusion software, only the technology behind it that recognises emotions in the user’s tone of voice).

www.ascinclusion.euwww.asc-

inclusion.eu

Example video of vocal intonation

recognition

Page 12: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

How the software will How the software will workworkBody gestures

The ASC-Inclusion software will develop this existing system to recognise more subtle gestures of the head, shoulders and hands to express emotions.

www.ascinclusion.eu

Example video of basic body gesture recognition

www.asc-inclusion.eu

Page 13: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Involvement of people with Involvement of people with autismautism

www.asc-inclusion.eu

• Involvement in all stages of software development

• Direct & indirect involvement (people with autism, as well as their teachers, parents, carers, and professionals)

• Planning – Information & advice from autism organisations

• User requirements – Research during early phase of software development to define the needs of people with autism

• Testing – People with autism directly involved in testing the software in UK & Israel

• Evaluation – Intermediaries (carers, social workers, psychologists & doctors) involved in evaluating the usefulness of the software for people with autism

Page 14: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Status and milestones Status and milestones

MilestonesEstimated start point

Project kick off November 2011

First version of the basic systems end of year 1 (2012)

Experiments with system startend of year 2 (2013)

First integrated system end of year 2 (2013)

Final project results end of project (2014)

www.ascinclusion.euwww.asc-

inclusion.eu

Page 15: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

Project partnersProject partners

www.ascinclusion.euwww.asc-

inclusion.eu

Page 16: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

More information & More information & updatesupdates

www.ascinclusion.euwww.asc-

inclusion.eu

Website: www.asc-inclusion.eu Join our mailing list / contact us:

[email protected]

Nikki Sullings
New slide added here... think it would be a good idea to always acknowledge the EC and the funding grant we received from them for thsi project :)
Page 17: Www.asc-inclusion.eu A project to develop software to assist people with autism to recognise, understand and express emotions through facial expressions,

AcknowledgementsAcknowledgements

The research leading to this software product has received funding from the European Community’s Seventh Framework Programme (FP7, 2007-2013), under grant agreement n° 289021.

www.ascinclusion.euwww.asc-

inclusion.eu

Nikki Sullings
New slide added here... think it would be a good idea to always acknowledge the EC and the funding grant we received from them for thsi project :)