Optimizing Motor Imagery Neurofeedback through the Use of Multimodal Immersive Virtual Reality and Motor Priming

Optimizing Motor Imagery Neurofeedback through the Use of Multimodal Immersive Virtual Reality and Motor Priming

Athanasios (Thanos) Vourvopoulos - John Edison Muñoz Cardona - Sergi Bermudez i Badia

University of Madeira / M-ITI

ICVR 20152

• So far, the development of Brain–Computer Interfaces (BCIs) that translate brain activity into control signals in computers or external devices provide new strategies to overcome stroke-related motor limitations

Interfacing the Brain with the Computer

Signal Processing

Signal Acquisition

End Effector

Control SignalRaw EEG

Motor Imagery

ICVR 20153

Why Is Motor Imagery (MI) Important?

• MI is relying on the same brain systems that would be used for actual performance of the task (Miller

et al., 2010).

• Repeated practice of MI can induce plasticity changes in the brain (Jackson et al., 2001)

• Combination of MI and BCI could augment rehabilitation gains (Ang et al., 2011)

Miller, K. J., Schalk, G., Fetz, E. E., Nijs, M. den, Ojemann, J. G., & Rao, R. P. N. (2010). Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proceedings of the National Academy of Sciences of the United States of America, 107(9),4430-5.

Movement Imagery

How can Motor Priming (MP) be utilized for neurorehabilitation?

ICVR 20154

• Motor priming is a possible way to facilitate motor learning (Stoykov, ME et al., 2015)

• Priming of the motorcortex is associated with changes in neuroplasticity that are associated with improvements in motor performance (Stinear, CM et al., 2008)

• So far, MP has not been tested in BCI training

ICVR 20155

BCI’s in Stroke Rehabilitation

Closed-Loop Robotic Control for Stroke Rehabilitation,Max Planck Institute for Intelligent Systems

Inclusion of patients with limited/no active movement

Boosts motor imagery practice during stroke recovery

Functional and structural plasticity and recovery

Long and repetitive training resulting in user fatigue

No standardized and accepted treatment for the use of BCIs

Little is known about how a BCI may affect brain plasticity through sensori-motor cortex oscillations

ICVR 20156

How Can we Optimize Training

Maximize the engagement of:1. Users

2. Sensory-motor networks

Development of multimodal feedback, in an immersive VR environment

ICVR 20157

Methodology - BCI Training Task Design

(a) Feedback level

(b) Instructions level (c) Task level

Incorporated all the necessary properties of a good instructional design (Lotte et al. 2013)

ICVR 20158

Methodology - Experimental Setup

Clear Goals and FeedbackPositive reinforcementChallenging task Immersive VR

ICVR 20159

Methodology - Experimental Setup

EEG Signal Acquisition

HMD

Hand Tracking

Stereo Sound

g.MOBIlab+Oculus Rift (DK1)Leap Motion

ICVR 201510

Methodology - Experimental Conditions

VR-MP:Virtual Reality-Motor

Priming

VR:Virtual Reality

Control : Standard Motor

Imagery

ICVR 201511

Methodology - Experimental Design

VR-MP:Virtual Reality-Motor Priming

ICVR 201512

Methodology - Experimental Design

VR:Virtual Reality

ICVR 201513

Methodology - Experimental Design

Control : Standard Motor

Imagery

ICVR 201514

Methodology - Experimental Design

MI-BCI Training

MI-BCI Session

QuestionnairesRest

8 min 8 min15 min 10 min

Equipment Setup

10-15 min

Motor Priming

8 minutes

8 min

1 Condition/Day per User

ICVR 201515

• 9 healthy participants (1 female)• Mean Age: 27 ± 2• 1 Left handed• Voluntary sample• No previous known neurological disorder• No previous experience in BCIs

Methodology - Participants

ICVR 201516

Methodology - Data

QuestionnairesClassifier Score % (LDA)

Raw EEG (Rhythms)

Presence Questionnaire

(PQ)

Kinaesthetic Imagery (KI)

NASA TLX: workload

Alpha (8 - 12 Hz)

Beta (12 - 30 Hz)

Theta (4 - 7 Hz)

Gamma (25- 100 Hz)

Methodology: Extracting the EEG Rhythms

17

Rhythms

Seconds

Alpha

Beta

Theta

Gamma

Raw EEG

ECG

EMG

7 - 14 Hz

15 - 30 Hz

4 - 7 Hz

30 - 100 Hz

ICVR 201518

Results: Do conditions modulate brain activity patterns?

Is associated with• Drowsiness• Concentration• Visual fixation (J. M. Stern, 2005)

VR-MP VR Control

Mea

n Po

wer

(dB)

Alpha (α)

ICVR 201519

Results: Do conditions modulate brain activity patterns?

Mea

n Po

wer

(dB) Is associated with

• Motor behavior• Active thinking• Active attention (S. Sanei, J. A. Chambers, 2008)

Beta (β)

VR-MP VR Control

ICVR 201520

Results: Do conditions modulate brain activity patterns?

Mea

n Po

wer

(dB) Is associated with

• meditative, relaxed and creative states (S. Sanei, J. A. Chambers, 2008)

VR-MP VR Control

Theta (θ)

ICVR 201521

Results: Do conditions modulate brain activity patterns?

Mea

n Po

wer

(dB) Is associated with

• visual, auditory, somatic and olfactory perception

• Attention (J. T. Cacioppo et al., 2007)

Gamma (γ)

VR-MP VR Control

ICVR 201522

Results: Do conditions modulate brain activity patterns?

ICVR 201523

Results: Do different conditions improve MI-BCI training performance?

MI-BCI Calibration Performance

• VR-MP provides the highest performance

• There is a trend in favor of multimodal setups

• No significant differences between groups

ICVR 201524

Results: Is subjective experience modulated by condition?

NASA TLX : Reported workload during BCI

Mean Difference = 17.556 Std. Error = 3.575 Sig: 0.001

TLX questions

*Mental Demand (M=12, STD=5)Physical demand (M=7, STD=5)*Temporal demand (M=7, STD=3)Performance (M=10, STD=4)Effort (M=12, STD=4)*Frustration (M=11, STD=4)

* Significant difference

VR-MP VR Control

ICVR 201525

Results: How realistic was the multimodal VR simulation?

70%Realism (M=73, STD=8)

Self-evaluation of performance (M=83, STD=9) Sounds (M=79, STD=12)

Quality of the interface (M=58 ,STD=13 )

Possibility to act (M=77 ,STD=14 )

Presence Questionnaire groups in % (perceived sense of presence during BCI, Cond A)

ICVR 201526

Results: Relationship between subjective experience and brain activity

TLX: • Mental Demand

• Temporal Demand

• Kinaesthetic Imagery Kinaesthetic Imagery

TLX: • Effort• Frustration• Physical Demand

PQ: • Quality of Interface

TLX:• Physical Demand

PQ: • Realism• Sounds

ICVR 201527

A trend suggesting that multimodal VR feedback and motor priming could increase training performance

Relationship between kinesthetic imagery and Beta band could play an important role as inclusion criteria in neurorehabilitation through MI-BCI paradigms

Discussion

ICVR 201528

VR-MP is more mentally demanding task engaging additional neural circuits than in the other 2

conditions

Significant contributions of the VR-MP to the engagement of alpha and beta bands, related with MI practice

increased cortical activation in the affected somatosensory and motor areas

Discussion

ICVR 201529

Extend the study with more participants

Design of a complete MI-BCI game with integrated task-related training

Clinical Validation with stroke survivors

Future Work

ICVR 201530

Thank You !

http://neurorehabilitation.m-iti.org/