Brain Computer Interface in BMI

Papavasileiou-1

CSE5810

Brain Computer Interface in BMIBrain Computer Interface in BMI

Ioannis Papavasileiou Computer Science & Engineering Department

The University of Connecticut371 Fairfield Road, Unit 4155

Storrs, CT 06269-2155

[email protected]

Papavasileiou-2

CSE5810

What is BCI?What is BCI? BCI is:BCI is:

System that allows direct communication pathway between human brain and computer

It consists of data acquisition devices, and appropriate algorithms

How is it used in BMI:How is it used in BMI: Clinical research Disease-condition detection and treatment Human computer interfaces for

Control Emotions detection Text input - communication

Papavasileiou-3

CSE5810

Research areas involvedResearch areas involved Computer scienceComputer science

Data mining Machine learning Human computer interaction

NeuroscienceNeuroscience Cognitive scienceCognitive science EngineeringEngineering

Papavasileiou-4

CSE5810

Key challengesKey challenges Technology-related:Technology-related:

Sensor quality – low SNR Supervised learning – “curse of dimensionality” System usability Real-time constraints Non-invasive EEG information transfer rate is

approx. 1 order of magn. lower People-relatedPeople-related

People are not always familiar with technology Preparation – training phases are not fun! Concentration, attention consciousness levels Task difficulty

Papavasileiou-5

CSE5810

BCI componentsBCI components Data acquisitionData acquisition

Electroencephalography (EEG) Electrical activity recording Invasive or not

Functional Near Infrared Spectroscopy (fNIRS) Recording of infrared light reflections of the brain

Functional magnetic resonance imaging (FMRI) Detection of changes in blood flow

Data AnalysisData Analysis Data mining & machine learning

Decision makingDecision making Output & ControlOutput & Control

HCI

Papavasileiou-6

CSE5810

Typical BCI architectureTypical BCI architecture

Papavasileiou-7

CSE5810

Electroencephalography (EEG)Electroencephalography (EEG) What is it:What is it:

Recoding of the electrical activity of the brain

Types:Types: Invasive Non-invasive

Properties:Properties: High temporal resolution Low spatial resolution Scalp acts as filter!

Papavasileiou-8

CSE5810

International 10-20 standardInternational 10-20 standard Electrodes located at the scalp at predefined Electrodes located at the scalp at predefined

positionspositions Number of electrodes can varyNumber of electrodes can vary

Papavasileiou-9

CSE5810

The EEG wavesThe EEG waves Alpha – Alpha –

occipitallyoccipitally Beta – frontally Beta – frontally

and parietallyand parietally Theta – children, Theta – children,

sleeping adultssleeping adults Delta – infants, Delta – infants,

sleeping adultssleeping adults

Papavasileiou-10

CSE5810

fMRIfMRI Functional magnetic resonance imagingFunctional magnetic resonance imaging Fact:Fact:

Cerebral blood flow and neuronal activation coupled

Detection of blood flow changesDetection of blood flow changes Use of magnetic fieldsUse of magnetic fields High spatial resolutionHigh spatial resolution Low temporal resolutionLow temporal resolution Clinical use:Clinical use:

Assess risky brain surgery Study brain functions

Normal Diseased Injured

Map functional areas of the brain

Papavasileiou-11

CSE5810

fNIRSfNIRS Functional Near Infrared SpectroscopyFunctional Near Infrared Spectroscopy Project near infrared light into the brain from the scalpProject near infrared light into the brain from the scalp Measure changes in the reflection of the light due toMeasure changes in the reflection of the light due to

Oxygen levels associated with brain activity Result absorption and scattering of the light

photons Used to build maps of brain activity

High spatial resolution High spatial resolution <1 cm

Lower temporal resolutionLower temporal resolution >2-5 seconds

Papavasileiou-12

CSE5810

BMI & clinical applicationsBMI & clinical applications Diagnose:Diagnose:

Epilepsy – seizures Brain-death Alzheimer’s disease Physical or mental problems

Study of:Study of: Problems with loss of consciousness Schizophrenia (reduced Delta waves during sleep)

Find location of:Find location of: Tumor Infection bleeding

Source: http://www.webmd.com/, http://www.nlm.nih.gov

Papavasileiou-13

CSE5810

Sleep disorders & mental tasksSleep disorders & mental tasks Sleep disorders studySleep disorders study

Insomnia Hypersomnia Circadian rhythm disorders Parasomnia (disruptions in slow sleep waves)

Mental tasks monitoringMental tasks monitoring Mathematical operations Counting Etc.

Papavasileiou-14

CSE5810

NeurofeedbackNeurofeedback Applications in

Autistic Spectrum Disorder (ASD) Anxiety Depression Personality Mood Nervous system

Self control

Papavasileiou-15

CSE5810

Feedback EEG-BCI architectureFeedback EEG-BCI architecture

Papavasileiou-16

CSE5810

Typical data analysis processTypical data analysis process Data acquisition and segmentationData acquisition and segmentation

Preprocessing Removal of artifacts Facial muscle activity External sources, like power lines

Feature extractionFeature extraction Typically sliding window Time-frequency features Latency introduced

Papavasileiou-17

CSE5810

Feature extractionFeature extraction Model-based methodsModel-based methods

Require selection of the model order FFT (Fast Fourier Transform) – based methodsFFT (Fast Fourier Transform) – based methods

Apply a smoothing window Features used:Features used:

Specific frequency band power Band-pass filtering and squaring Autoregressive spectral analysis

Many times a feature selection or projection is done to Many times a feature selection or projection is done to reduce the huge feature vectorsreduce the huge feature vectors

Papavasileiou-18

CSE5810

Data ClassificationData Classification Typical classifiers usedTypical classifiers used

Artificial Neural Networks (ANN) Linear Discriminant analysis (LDA) Support Vector Machines (SVM) Bayesian classifier Hidden Markov Models (HMM) K-nearest neighbor (KNN)

Parameters for each classifier can affect the Parameters for each classifier can affect the performanceperformance # of hidden units in ANN # of supporting vectors for SVMs Etc.

Papavasileiou-19

CSE5810

Human computer interactionHuman computer interaction BCIs are considered to be means of communication and BCIs are considered to be means of communication and

control for their userscontrol for their users HCI community defines three types:HCI community defines three types:

Active BCIs Consciously controlled by the user E.g. sensorimotor imagery (multi-valued control signal)

Reactive BCIs Output derived from reaction to external stimulation Like P300 spellers

Passive BCIs Output is related to arbitrary brain activity E.g. memory load, emotional state, surprise, etc.

Used in assistive technologies and rehabilitation Used in assistive technologies and rehabilitation therapiestherapies

Papavasileiou-20

CSE5810

BCI & Assistive TechnologiesBCI & Assistive Technologies Communication systemsCommunication systems

Basic yes/no Character spellers Virtual keyboards

ControlControl Movement imagination

Cursor Wheelchairs Artificial limbs & prosthesis

Automation in smart environments Current BCI systems have at most 10-25 bits/minute Current BCI systems have at most 10-25 bits/minute

maximum information transfer ratesmaximum information transfer rates It can be valuable for those with severe disabilities

Papavasileiou-21

CSE5810

P300 spellersP300 spellers Most typical reactive BCIMost typical reactive BCI 3-4 characters / min with 95% success3-4 characters / min with 95% success

Papavasileiou-22

CSE5810

P300 waveP300 wave Event related potential (ERP)Event related potential (ERP) Elicited in the process of decision makingElicited in the process of decision making Occurs when person reacts to stimulusOccurs when person reacts to stimulus Characteristics:Characteristics:

Positive deflection in voltage Latency 250 to 500 ms

Typically 300 ms Close to the parietal lobe in the brain Averaging over multiple records required

Papavasileiou-23

CSE5810

Other ERP usesOther ERP uses Lie detectionLie detection

Increased legal permissibility Compared to other methods

ERP abnormalities related to conditions such as:ERP abnormalities related to conditions such as: Parkinson’s Stroke Head injuries And others

Typical ERP paradigmsTypical ERP paradigms Event related synchronization (ERS) Event related de-synchronization (ERD)

Papavasileiou-24

CSE5810

Other Control BCI paradigmsOther Control BCI paradigms Lateralized readiness potentialLateralized readiness potential

Game control 1~2 seconds latency Negative shift in EEG develops before actual

movement onset Steady-state visually evoked potentials (SSVEPs)Steady-state visually evoked potentials (SSVEPs) Slow cortical potential (SCP)Slow cortical potential (SCP)

Imaged movements affect mu-rhythms They shift polarity (+ or -) of SCP

Sensorimotor cortex rhythms (SMR)Sensorimotor cortex rhythms (SMR) EMGEMG

Papavasileiou-25

CSE5810

SCP & SMR vs P300SCP & SMR vs P300 Typically SCP and SMR BCIs require significant Typically SCP and SMR BCIs require significant

training to gain sufficient controltraining to gain sufficient control In contrast P300 BCIs require less as they record In contrast P300 BCIs require less as they record

response to stimuliresponse to stimuli However, they require some sort of stimuli like

visual (monitor always in place) or audio Also SCP BCIs have longer response timesAlso SCP BCIs have longer response times

Papavasileiou-26

CSE5810

Binary speller controlBinary speller control User imagines movement of cursorUser imagines movement of cursor

Typically hand movement The goal is to select a characterThe goal is to select a character

Papavasileiou-27

CSE5810

Wheel chair controlWheel chair control All the mentioned BCI All the mentioned BCI

paradigms have been paradigms have been applied to wheelchair applied to wheelchair controlcontrol

Either using a monitor for Either using a monitor for feedbackfeedback

Or active paradigms as Or active paradigms as sensorimotor imagery sensorimotor imagery (SMR)(SMR)

Similar approaches have Similar approaches have been applied to roboticsbeen applied to robotics Artificial limbs etc

Papavasileiou-28

CSE5810

Environment controlEnvironment control BCIs used by disabled to improve quality of lifeBCIs used by disabled to improve quality of life Operation of devices likeOperation of devices like

Lights TV Stereo sets Motorized beds Doors Etc

Typically use of P300, SMR and EMG related BCIsTypically use of P300, SMR and EMG related BCIs

Papavasileiou-29

CSE5810

EMG-based human-robot interface exampleEMG-based human-robot interface example Motion prediction based on hand positionMotion prediction based on hand position EMG pattern classification as control commandEMG pattern classification as control command Combination of both yields motion command to Combination of both yields motion command to

prosthetic handprosthetic hand

Papavasileiou-30

CSE5810

Emotions detectionEmotions detection Use of facial expressions to imply user emotionsUse of facial expressions to imply user emotions ERD/ERS based BCIsERD/ERS based BCIs

Emotional state can change the asymmetry of the frontal alpha

P300 - SSVEPP300 - SSVEP Emotional state can change the amplitude of the

signal from 200ms after stimulus presentation

Papavasileiou-31

CSE5810

BCIs for recreationBCIs for recreation GamesGames

EPOC headset Mindset

Virtual realityVirtual reality Outputs of a BCI areShown virtual environment

Creative ExpressionCreative Expression Music

Generated form EEG signals Visual art

Painting for artists who are locked in as a result of ALS – amyotrophic lateral sclerosis

Papavasileiou-32

CSE5810

Security and EEGSecurity and EEG EEG has been used in user authenticationEEG has been used in user authentication Every brain is differentEvery brain is different Different characteristics of EEG waves are used in Different characteristics of EEG waves are used in

user authenticationuser authentication ProsPros

User has nothing to remember Harmless Automatically applied

ConsCons User has to wear an EEG headset Accuracy is still not 100% Still not used in practice

Papavasileiou-33

CSE5810

Thank you!Thank you!