Assessing Engagement in an Emotionally-Adaptive Applied Game

Assessing Engagement in an Emotionally-Adaptive

Applied GameBoyan Bontchev and Dessislava Vassileva

Dep. of Software Technologies,

Fac. of Mathematics and Informatics, Sofia Univ., Bulgaria

TEEM-2016

Salamanca, November 2, 2016

Assessing Engagement in an Emotionally-Adaptive Applied Game

2 Player-centric adaptation

Player-centric adaptation in video games needs to answer some questions: Who – player character structural and behavioral

changes => requires player modelling How – how adaptation will be realized => requires

adaptive loop design What – which game features can be adapted =>

requires adaptive gameplay design Why – advantages of game adaptation => requires

analysis of outcomes in playability and learning results

02/11/2016


3 The ADAPTIMES FP7 project ADAPTIve player-centric serious video gaMES ADAPTIMES aims at investigating how

cognitive abilities (the cognitive part), psycho-emotional status (the affective part) and playing style (the conative part – that of “desire, volition, and

striving”)

can be used in a holistic approach for efficient and effective player-centric adaptation

02/11/2016


4 Adaptation Model

02/11/2016

Game dynamics

Gam

e m

echa

nics

Gam

e aesthetics

Player

Player style metrics

Perfo

rman

ce

indi

cato

rs

Emotional

indicatorsPlaying styles

Performance and efficiency

(abilities, skills and knowledge)

Emotions(flow, immersion &

intrinsic motivation)


5 Measuring process

Measuring player’s performance – by tracking player’s results, incl. time, task difficulties and efficiency

Measuring playing styles – based on tracking specific gaming metrics describing player’s style of playing

Measuring player’s affect (emotions and arousal):By means of analysis of psychophysiology signals

(arousal)By means of visual expressions analysis (emotional

states)

02/11/2016


6 Affect inferred by psychophysiology signals EDA (Electro-Dermal Activity) Custom EDA meter – custom hardware device and

software (H2020 RAGE asset developed by SU)

02/11/2016


7 Emotions inferred by visual expressions analysis Custom emotion recognizer using Affectiva SDK Recognizes: sadness, disgust, anger, surprise, fear, joy;

engagement, attention (fixation), eye closures (saccades)

02/11/2016


8 Workflow of emotion-based game dynamic adaptation control

02/11/2016

Implicit EmotionalIndicators

(via sensors & other devices)Facial expr.

GSRHeart Rhythm

ECGEEG

Pressure…

ChangeProcedures

TimeVelocity

AccelerationRouteSize

Defeat range Luminosity

Contrast…

Real-TimeExplicitPlayer

Feedbackthrough

AdaptationControl Panel

Analysis

Change

Visuali-zation

Triggering Events

on Behavior Templates


9 Emotional Game Adaptation 1/2

02/11/2016

Adaptation of flying objects’ velocity, puzzle difficulty, and ambient light for lower phasic arousal


10 Emotional Game Adaptation 2/2

02/11/2016

Adaptation of flying objects’ velocity, puzzle difficulty, and ambient light for higher phasic arousal


11 Box plots of player’s average emotions inferred by facial expressions

02/11/2016


12 Box plots of measured GEngQ terms

02/11/2016

GEngQ = Game Engagement Questionnaire (Brockmyer et al., 2009)


13 Player´s joy, engagement, attention, eye closure, and SCR (skin conductance response)

02/11/2016

Successful shot Unsuccessful shot


14 Correlations of psychological states (by facial expressions) and arousal with GEngQ terms

02/11/2016

Game Engagement Questionnaire (GEngQ)

State / arousal

Presen-ce

Absorp-tion Flow Immer-

sionEngage-

ment

Atten-tion 0.3878 0.0645 0.1779 0.0139 0.1978

Enga-gement 0.0087 -0.0510 0.1870 -0.0532 0.0221

Eye closure 0.3818 0.0885 0.3711 0.1737 0.1554

SCLlevel 0.1341 -0.1312 0.0459 -0.1067 -0.0318

SCRlevel 0.0119 -0.0558 -0.2177 0.1042 -0.0674


15 Correlations of psychological states and arousal with emotions inferred by facial expressions

02/11/2016

Emotions

State / arousal

Sad-ness Disgust Anger Surp-

rise Fear Joy

Atten-tion -0.0226 0.3712 0.2329 0.1835 -0.1677 0.2040

Enga-gement 0.0782 0.7049 0.1653 0.4296 0.1013 0.8346

Eye closure -0.1257 0.2293 0.0918 0.4657 0.1713 0.3196

SCLlevel 0.0226 -0.1484 -0.0759 0.3873 0.3007 -0.0027

SCRlevel -0.0275 0.3616 -0.0307 0.1285 -0.0959 0.4008

Conclusions 1/2 Statistically significant correlations found:

attention and presenceeye closure and presenceeye closure and flowengagement and surpriseengagement and joySCL arousal level and surpriseSCR and joyeye closure & average efficiencySCL & relative time

Facial expressions (emotions) are complementary to EDA (arousal)

16

02/11/2016Assessing Engagement in an Emotionally-Adaptive Applied Game

Conclusions 2/2

Player-centric adaptive game play possesses essential advantages compared to the non-adaptive gameplay

The synergy of using player´s performance, emotions, and playing styles is very promising

All they are identified at run time and used for adapting game mechanics, dynamics and audio-visual content, in a dynamic and implicit way

More experiments are needed for revealing how such emotion-based adaptations affect game experience and playability, learning outcomes and consequences of flow

17



18 Future work Emotionally-adaptive games and other software app’s Dynamic adaptation control Software app’s using user affectation:

HRVTEMP….

Web-based platforms for generation of educational games targeted to non-ICT people

Semantic content management in applied games

02/11/2016

Thank you for your attention! Discussion

Videos at: https://www.youtube.com/watch?v=rSzM2RrceNw&index=1&list=PLs4ZK9XAjRxpROiCNQ1kdDR8SFpH-itmY

More info at: http://adaptimes.eu/

19


https://www.youtube.com/watch?v=rSzM2RrceNw&index=1&list=PLs4ZK9XAjRxpROiCNQ1kdDR8SFpH-itmY



http://adaptimes.eu/

http://adaptimes.eu/

Assessing Engagement in an Emotionally-Adaptive Applied Game

Engineering