Graz-Brain-Computer Interface: State of Research By Hyun Sang Suh.

Graz-Brain-Computer Interface: Graz-Brain-Computer Interface: State of ResearchState of Research

ByHyun Sang Suh

Overview: BCI systemsOverview: BCI systems Overview: BCI systemsOverview: BCI systems

The user performs a certain task, which has a distinct EEG signature

The user performs a certain task, which has a distinct EEG signature

The specific features are extracted from the EEG

The specific features are extracted from the EEG

A pattern classification system uses these EEG features to determine which task the user performed

A pattern classification system uses these EEG features to determine which task the user performed

The BCI presents feedback to the user, and forms a message or command

The BCI presents feedback to the user, and forms a message or command

Motor execution vs. Movement imaginationMotor execution vs. Movement imaginationMotor execution vs. Movement imaginationMotor execution vs. Movement imagination

Imagination

Execution

ERD ERS

500mstime

Subject 1, g3 Subject 2, f4

How can we discriminate four motor How can we discriminate four motor imagery tasks?imagery tasks?

How can we discriminate four motor How can we discriminate four motor imagery tasks?imagery tasks?

TongueTongue

Left HandLeft Hand

Right handRight hand

FootFoot

The mu-wave BCIThe mu-wave BCI The mu-wave BCIThe mu-wave BCI Mu wave activity occurs around roughly 12 Hz.

Alpha waves are strongest over the visual areas in the occipital lobe, But mu waves are strongest over the motor areas in the frontal lobe.

Mu activity changes as people perform or imagine movement. You have ERD/ ERS patterns depending on the motor imagery tasks

Mu wave activity occurs around roughly 12 Hz.

Alpha waves are strongest over the visual areas in the occipital lobe, But mu waves are strongest over the motor areas in the frontal lobe.

Mu activity changes as people perform or imagine movement. You have ERD/ ERS patterns depending on the motor imagery tasks

Time

Subjects and experimental paradigmSubjects and experimental paradigm Subjects and experimental paradigmSubjects and experimental paradigm

Participants: Six female and three male healthy right-handed subjects.

Remain relaxed and avoid any motion during experiment. Imagine the experience of movement (kinesthetic, MIK). The arrow pointing represent one of the four different

tasks (left hand, right hand, both feet and tongue). EEG signal were recorded from 60 electrodes referenced

to the left mastoid.

Participants: Six female and three male healthy right-handed subjects.

Remain relaxed and avoid any motion during experiment. Imagine the experience of movement (kinesthetic, MIK). The arrow pointing represent one of the four different

tasks (left hand, right hand, both feet and tongue). EEG signal were recorded from 60 electrodes referenced

to the left mastoid.

Quantification of ERD/ ERSQuantification of ERD/ ERS Quantification of ERD/ ERSQuantification of ERD/ ERS

First, band-pass filtering of each trial. Second, squaring of samples (with smoothing) Third, averaging of N trials.

The ERD/ ERS pattern is defined as the percentage power decrease (ERD) or power increase (ERS) comparison to one-second reference interval (0.5-1.5 sec).

First, band-pass filtering of each trial. Second, squaring of samples (with smoothing) Third, averaging of N trials.

The ERD/ ERS pattern is defined as the percentage power decrease (ERD) or power increase (ERS) comparison to one-second reference interval (0.5-1.5 sec).

( )( ) B ref

ref

P t PERD t

P

Kappa coefficient and ITVKappa coefficient and ITV Kappa coefficient and ITVKappa coefficient and ITV

Kappa coefficient - To measure distinctiveness

Kappa coefficient - To measure distinctiveness

1

11

acc n

n

Where acc is the accuracy derived by confusion matrix, n is the number of classes

Where acc is the accuracy derived by confusion matrix, n is the number of classes

Intertask variability (ITV) - standard deviation of averaged ERD/ ERS

Intertask variability (ITV) - standard deviation of averaged ERD/ ERS

Frequencies and band power changesFrequencies and band power changes Frequencies and band power changesFrequencies and band power changes

Time-frequency maps displaying ERD/ ERSTime-frequency maps displaying ERD/ ERS Time-frequency maps displaying ERD/ ERSTime-frequency maps displaying ERD/ ERS

time

Maps displaying the topographical Maps displaying the topographical distribution of averaged band powerdistribution of averaged band power

Maps displaying the topographical Maps displaying the topographical distribution of averaged band powerdistribution of averaged band power

High ITVHigh ITV

Low ITVLow ITV

Intertask variability: ITVIntertask variability: ITV

Brainloop Interface for GoogleBrainloop Interface for GoogleBrainloop Interface for GoogleBrainloop Interface for Google

R. Scherer, G. Pfurtscheller. The self-paced Graz brain-computer interface: methods and applications. Computational Intelligence and Neuroscience 2007, 79825, 2007.

Mu vs. P300 BCIsMu vs. P300 BCIsMu vs. P300 BCIsMu vs. P300 BCIs

Requiring training

Work in real-time

2D control possible

Continuous control

Affected by movement

Requiring training

Work in real-time

2D control possible

Continuous control

Affected by movement

Requiring no training

Require averaging

1D control only

Discrete control

Affected by distraction

Requiring no training

Require averaging

1D control only

Discrete control

Affected by distraction

Mu BCIMu BCI P300 BCIP300 BCI

Phase Synchronization FeaturesPhase Synchronization FeaturesPhase Synchronization FeaturesPhase Synchronization Features

Currently, BCIs system is not considered the relationships between EEG signals measure at different electrode recording.

We can obtain the additional information from this relationships.

Phase Locking value (PLV) is one of the method to quantify such relationships.

The PLV can measure the level of phase synchronization between pairs of EEG signals.

The PLV value of 1 means that the two channels are highly synchronized, whereas a value of 0 means no phase synchronization.

Currently, BCIs system is not considered the relationships between EEG signals measure at different electrode recording.

We can obtain the additional information from this relationships.

Phase Locking value (PLV) is one of the method to quantify such relationships.

The PLV can measure the level of phase synchronization between pairs of EEG signals.

The PLV value of 1 means that the two channels are highly synchronized, whereas a value of 0 means no phase synchronization.

Phase Synchronization FeaturesPhase Synchronization FeaturesPhase Synchronization FeaturesPhase Synchronization Features

BCI Applications BCI Applications BCI Applications BCI Applications

Patient with Spinal Cord InjuryPatient with Spinal Cord InjuryPatient with Spinal Cord InjuryPatient with Spinal Cord Injury

Spinal Cord Injury (SCI)

- Damage or trauma to the spinal cord that result in a loss or impaired function

- The effects of SCI depend on type of injury (i.e, a car accident, falls, sports injuries, or a disease)

Spinal Cord Injury (SCI)

- Damage or trauma to the spinal cord that result in a loss or impaired function

- The effects of SCI depend on type of injury (i.e, a car accident, falls, sports injuries, or a disease)

Restoration of hand movement in SCI patientRestoration of hand movement in SCI patientRestoration of hand movement in SCI patientRestoration of hand movement in SCI patient

Functional Electrical StimulationFunctional Electrical StimulationFunctional Electrical StimulationFunctional Electrical Stimulation

BCI controlled FESBCI controlled FESBCI controlled FESBCI controlled FES

G. Pfurtscheller, G. R. Müller, J. Pfurtscheller, H. J. Gerner, Rüdiger Rupp. 'Thought'-control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia. Neuroscience Letters 351, 33-36, 2003. .

What is the Neuroprosthese?What is the Neuroprosthese?What is the Neuroprosthese?What is the Neuroprosthese?

It is a device which replaces nerve function lost as a result of disease or injury.

The neuroprosthetics can act as a bridge between functioning elements of the nervous system and damaged nerves.

It can be used in the spinal cord to allow standing in paraplegics.

It is a device which replaces nerve function lost as a result of disease or injury.

The neuroprosthetics can act as a bridge between functioning elements of the nervous system and damaged nerves.

It can be used in the spinal cord to allow standing in paraplegics.

Hand prosthesesHand prostheses

AUDITORY PROSTHETICSAUDITORY PROSTHETICSAUDITORY PROSTHETICSAUDITORY PROSTHETICS most successful example of sensory prosthetic is the cochlear

implant. lack the cochlear hair cells that transduce sound into neural

activity. Extended to direct stimulation of the brainstem for those with

dysfunctional cochlear nerves.

most successful example of sensory prosthetic is the cochlear implant.

lack the cochlear hair cells that transduce sound into neural activity.

Extended to direct stimulation of the brainstem for those with dysfunctional cochlear nerves.

VISUAL PROSTHETICSVISUAL PROSTHETICSVISUAL PROSTHETICSVISUAL PROSTHETICS

The device uses electrical signals to bypass dead photoreceptors and stimulate remaining viable cells of the retina.

Images come from the external video camera worn behind the patient’s glasses.

The images are transmitted through a computer to electrodes attached to the retina

Reproduce the visual image in the occipital lobe.

The device uses electrical signals to bypass dead photoreceptors and stimulate remaining viable cells of the retina.

Images come from the external video camera worn behind the patient’s glasses.

The images are transmitted through a computer to electrodes attached to the retina

Reproduce the visual image in the occipital lobe.

BCI controlled NeuroprostheseBCI controlled NeuroprostheseBCI controlled NeuroprostheseBCI controlled Neuroprosthese

The BCI system is implanted his right hand and arm

Detect brain pattern (ERD/ ERS) of left hand foot imagery movement

Provide two graps patterns

The BCI system is implanted his right hand and arm

Detect brain pattern (ERD/ ERS) of left hand foot imagery movement

Provide two graps patterns

BCI controlled NeuroprosthesisBCI controlled NeuroprosthesisBCI controlled NeuroprosthesisBCI controlled Neuroprosthesis

G. R. Müller-Putz, R. Scherer, G. Pfurtscheller, R. Rupp. EEG-based neuroprosthesis control: a step towards clinical practice. Neuroscience Letters 382, 169-174, 2005.

BCI controlled GameBCI controlled GameBCI controlled GameBCI controlled Game

Thank you for your attention