ESCUELA T ´ ECNICA SUPERIOR DE INGENIER ´ IA Y SISTEMAS DE TELECOMUNICACI ´ ON PROYECTO FIN DE GRADO T ´ ITULO: BCI based FES system for stroke neurorehabilitation: Comparison of SBCSP and CSSBP algorithms AUTOR: ´ Angel Post Vinuesa TITULACI ´ ON: Grado en Ingenier´ ıa de Sistemas de Telecomunicaci´ on TUTOR: Sadasivan Puthusserypady UNIVERSIDAD: Technical University of Denmark (DTU) CENTRO: Department of Electrical Engineering (BME) PA ´ IS: Dinamarca Fecha de lectura: 04 de Julio de 2016 Calificaci´ on: El coordinador de Movilidad,
142
Embed
ESCUELA TECNICA SUPERIOR DE INGENIER IA Y SISTEMAS DE ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ESCUELA TECNICA SUPERIOR DE
INGENIERIA Y SISTEMAS DE
TELECOMUNICACION
PROYECTO FIN DE GRADO
TITULO: BCI based FES system for stroke neurorehabilitation:Comparison of SBCSP and CSSBP algorithms
AUTOR: Angel Post Vinuesa
TITULACION: Grado en Ingenierıa de Sistemas de Telecomunicacion
TUTOR: Sadasivan Puthusserypady
UNIVERSIDAD: Technical University of Denmark (DTU)
CENTRO: Department of Electrical Engineering (BME)
PAIS: Dinamarca
Fecha de lectura: 04 de Julio de 2016
Calificacion:
El coordinador de Movilidad,
Resumen
Muchas personas experimentan debilidad muscular o paralisis despues de un accidente
cerebrovascular, que puede afectar a su movilidad y equilibrio, por lo general en un lado
de su cuerpo o de un solo brazo o una pierna.
Un sistema de estimulacion electrica funcional (FES) se puede utilizar para recuperar
las capacidades motoras, utilizando las corrientes electricas para activar los nervios que
inervan las extremidades afectadas.
Este dispositivo FES es controlado por la interfaz cerebro-ordenador (BCI), que propor-
ciona un sistema de comunicacion entre el cerebro humano y los dispositivos externos,
utilizando las senales EEG de los pacientes. Diferentes actividades imaginarias, como el
movimiento de las extremidades, se pueden clasificar en base a los cambios producidos
en las bandas de frecuencia µ y β y sus distribuciones espaciales.
Con respecto a los patrones topograficos de modulaciones de ritmo cerebrales, el algo-
ritmo de patrones espaciales comunes (CSP) ha demostrado ser muy util para extraer
filtros espaciales discriminativos de sujetos especıficos. Sin embargo, el CSP es limitado
en muchas situaciones y no esta optimizado para el problema de clasificacion EEG. Para
superar esta limitacion, vamos a utilizar un metodo alternativo basado en un algoritmo
“Sub-band Common Spatial Pattern” (SBCSP) y la integracion de resultados con el
metodo “score fusion”.
Al mismo tiempo, utilizaremos otro metodo, el algoritmo “Common Spatial-Spectral
Boosting Pattern” (CSSBP), y compararemos los resultados de ambos metodos.
El objetivo de este proyecto es disenar un sistema FES controlado por la interfaz BCI
para mejorar las habilidades motoras de los dedos de una mano para los pacientes de-
spues de un accidente cerebrovascular, la comprension de los sistemas BCI y las senales de
EEG utilizados, la aplicacion de un algoritmo de SBCSP, ası como algoritmo de CSSBP
para la extraccion y clasificacion en MATLAB, y la realizacion de una evaluacion final.
Abstract
Many people experience muscle weakness or paralysis after a stroke, which can affect
their mobility and balance, usually on one side of their body or in just one arm or leg.
A functional electrical stimulation (FES) system can be used to regain motor skills, us-
ing electrical currents to activate nerves innervating the affected extremities. This FES
device is controlled by the Brain-computer interface (BCI), which provides a communi-
cation system between human brain and external devices, using the EEG signals of the
patients.
Different imaginary activities, like limb movement, can be classified based on the changes
in µ and β rhythms and their spatial distributions. With respect to the topographic
patterns of brain rhythm modulations, the Common Spatial Patterns (CSP) algorithm
has proven to be very useful to extract subject-specific, discriminative spatial filters.
However, CSP is limited in many situations and it is not optimized for the EEG classi-
fication problem. To overcome this limitation, we will use an alternative method based
on Sub-band CSP (SBCSP) and score fusion.
This method will be compared with the common spatial-spectral boosting pattern (CSSBP)
algorithm. The aim of this project is to use a BCI controlled FES system to improve
the motor skills in the fingers of one hand for post-stroke patients, understanding the
BCI systems and EEG signals used, implementing a SBCSP algorithm as well as CSSBP
algorithm for the extraction and classification in MATLAB, and do a final evaluation.
Acknowledgements
I would like to extend my appreciation and gratitude for the help and support to the
persons who have contributed in the making this study possible.
Sadasivan Puthusserypady, my supervisor for his help, patience and guidance during the
study.
Helle K.Iversen, for the opportunity to collaborate with Glostrup Hospital and her guid-
ance during the project.
All the students who have contributed in this project and all the volunteer subjects.
My family, thank you for the opportunity of being here, encouraging me in all of my
pursuits and inspiring me to follow my dreams.
My friends, thank you for listening, offering me advice, supporting me through this
entire process and also for all the unforgettable moments during the year.
A heartfelt thanks goes out to Marıa for all your love, support, patience and under-
2.17 Upper panel: Superimposed band power time courses computed for threedifferent frequency bands (10–12 Hz, 14–18 Hz, and 36–40 Hz) from EEGtrials recorded from electrode position C3 during right index finger lifting.EEG data are triggered with respect to movement-offset (vertical line att = 0 s). Lower panel: Examples of ongoing EEG recorded during rightfinger movement. Movement-onset at t = 0 s. [12] . . . . . . . . . . . . . 17
According to the World Health Organization, 15 million people suffer stroke worldwide
each year. Of these, 5 million die, and another 5 million are permanently disabled.
A stroke occurs when the blood supply to a part of the brain in suddenly cut off.
Consequently, the cells in this area become damaged or die, which can cause severe
symptoms or death.
The effects vary depend on which part of the brain is injured, and how severely. Some of
them can be sudden weakness (that includes paralysis of one side of the body, losses of
movements in one hand, leg or both, weakness and twisting of one side of the face. . . ),
problems with balance or difficulty with seeing or speaking.
Stroke is a leading cause of serious long-term disability. The sudden nature of stroke
means that sufferers and their families cannot prepare the tremendous blow to their lives
and how it affects their quality of life.
Recovery of the motor skills in after stroke patients is crucial in order to improve daily
living activities, and there are several studies about this issue, most of them about brain
computer interface (BCI) systems based on electroencephalographic (EEG) signals.
A Brain computer interface (BCI) is a device that responds to neural processes from
the brain to provide a communication between the brain and an external device. This
technology has been incredibly developed in the last years, focus on improve the lives of
people with neurological disorders, most of them in post-stroke patients.
To regain motor skills in patients, one of the most effective ways is motor imagery. In
the imagination of limb movement, suppression of EEG signals happens in the specific
region of the motor and somatosensory cortex due to loss of synchrony in µ and β bands,
classically defined in the 12-16Hz and 18-24Hz respectively, is termed event-related de-
synchronization (ERD).
1
Introduction 2
This brain rhythm can be used to control as a control signal for assistive devices to regain
motor skills. One effective system that can be controlled by a BCI is the functional
electrical stimulation (FES) system. FES is a technique that use electrical currents
to active nerves innervating extremities affected by paralysis caused by stroke or other
disorders.
In this project we address a solution for these post stroke paralysis disorders. Particu-
larly, we design a BCI controlled FES system to improve the motor skills in the fingers of
one hand for post-stroke patients. This system combines the FES stimulations and men-
tal imagery, reconstructing the neuron-circuit between paralysis limbs and corresponding
pathological brain area of the subject.
In this system, there are many techniques for the extraction and classification of the EEG
signals. In the ERD brain rhythm, one of the main problems extracting and analyzing
this signals, is that the frequency bands varies from subjects. Common Spatial Patterns
(CSP) algorithm has proven to be very useful in extracting ERD. However, it can be only
applied to the informative frequency band, which are not the same in all the subjects.
To solve this limitation, the Common Spatio-Spectral Patter (CSSP) and the Common
Sparse Spectral Pattern (CSSSP) were proposed, in which a spatial and spectral filter
are optimized to solve this problem, but the flexibility of the frequency filters is still
limited, and are not really effective when it is used with people suffering neurological
diseases.
To solve this limitation, we use a method based on Sub-band CSP (SBCSP) and score
fusion, decomposing the EEG signals into sub-bands using a filter bank, deriving the
final decision from fusion of the score from each sub-band.
In addition, we compare it with an adaptive boosting algorithm to perform autonomous
selection of key channels and frequency band.
Finally, we evaluate the system into two different ways:
- From some databases that we can find in BNCI database.
- With data recorded from 12 subjects.
The aim is that the system works properly, getting the best results as possible, under-
standing all the processes, comparing the results between the two proposed algorithms
and obtaining a conclusion.
Chapter 2
Background
2.1 The brain
The brain is the body’s control centre, managing about everything we do. Inside our
heads, is an organ of 1,5 kg of weight consisting on billions of tiny cells, being the most
complex organ of the body. It controls body activities, ranging from heart rate and
sexual function to emotion, learning, and memory. [19, 20]
2.1.1 Mapping the brain
We can divide the brain into 3 different parts: The fore-brain, the mid-brain and the
hind-brain. [19]
Figure 2.1: Brain divided into 3 parts: Fore-brain, Mid-brain and Hind-brain. [3]
3
Background 4
The fore-brain is credited with the highest intellectual functions. There we can find
the Cerebrum, the largest part of the human brain. Interpreting touch, vision, hearing,
speech, emotions, learning, reasoning and fine control of movement are the higher func-
tions of the Cerebrum.
The cerebrum is composed of right and left hemispheres connected by the corpus callo-
sum, controlling each hemisphere the opposite side of the body, and the surface of the
cerebrum has a folded appearance called the cortex, referred as grey matter because of
his grey colour, contains about 70% of the 100 billion nerve cells.
The cerebral cortex can be divided into different lobes [19].
Figure 2.2: Lobes of the cerebral cortex [4]
Background 5
Each lobe has different functions, but each lobe does not function alone.
Lobe Functions
Frontal Lobe Initiating and coordinating motor movements.
Higher cognitive skills.
Personality, emotions.
Speech: Speaking and writing.
Parietal lobe Sensory processes.
Language.
Interprets signals from vision, hearing, motor, sensory and memory.
Spatial and visual perception.
Occipital lobe Process visual information
Temporal lobe Process auditory information.
Memory.
Hearing.
Sequencing and organization.
Table 2.1: Functions of the lobes from the cerebral cortex
The fore-brain has also other parts: the basal ganglia (cerebral nuclei deep in the cerebral
cortex, coordinating muscle movements), the thalamus (prioritize the information and
send it to receive from the cerebral cortex) and the hypothalamus (control of behaviours
and regulate body temperature, blood pressure, emotions and hormones).
The mid-brain has the function of visual and auditory reflexes and relaying this infor-
mation to the hypothalamus.
Finally, the hind-brain is composed by the pons, medulla oblongata, cerebellum and
spinal cord. The pons and medulla oblongata control respiration, heart rhythms and
blood glucose, the cerebellum coordinate muscle movements, maintain posture and bal-
ance and the spinal cord, the via that receive sensory information from all parts of the
body. [5, 19, 20]
Background 6
2.1.2 Neurons and the action potential
Neurons are the basic working units of the brain, transmitting information, cooperating
and competing with each other in regulating the overall state of the nervous system.
All the neurons consist of a cell body, dendrites, an axon and a synaptic terminals. [20]
Figure 2.3: Parts of a neuron: Dendrites (receiving), cell body (integrating), axonand synapse (transmitting). [5]
Chemical signals received in the dendrites from the axons that contact them are trans-
formed into electrical signals, and make a decision about whether to pass with the other
electrical signals from all the other synapses. Electrical potentials travel from dendrites
to synapses and repeat the process. [5]
To communicate from one neuron to another, the axons of neurons transmit electrical
pulses called action potentials, because of the ion-channels that are contained in the
axon-al membrane, that can open and close to let through electrically charged ions.
The flow of ions creates an electrical current that produces voltage changes across the
neuron’s cell membrane. [5, 19]
Figure 2.4: Neuronal communication.[6]
Background 7
2.1.3 Brain disorders and stroke
More than 1000 disorders of the brain and nervous system result in more hospitalizations
than other disease group. [19]
When the brain is damaged, the results can be devastating.
We can divide the Brain disorders in four categories: Brain injuries, brain tumours,
neurodegenerative diseases and mental disorders. [21]
Brain injuries are often caused by blunt trauma, damaging brain tissue, neurons and
nerves, affecting to the main abilities of the person. Some of this brain injuries are
haematoma, contusions, cerebral oedema or strokes. There can be treated by medication,
rehabilitation or brain surgery. [21]
A stroke occurs when blood flow to an area in the brain is cut off and the brain cells,
deprived of the oxygen and glucose needed to survive, die, so some abilities controlled
by this cells are lost. There are two types of strokes: Ischemic (clot) and haemorrhagic
(bleed). Ischemic stroke is caused by a blockage cutting off the blood supply to the
brain and haemorrhagic when a blood vessel bursts within or on the surface of the
brain. [7, 18, 22, 23]
Figure 2.5: Hemorrhagic and Ischemic Stroke.[7]
Background 8
Stroke causes a greater range of disabilities than any other condition. According to the
World Health Organization, 15 million people suffer stroke worldwide each year. Of
these, 5 millions die, and another 5 million are permanently disabled [18, 22]
Difficulty % of people affected
Upper limb/arm weakness 77%
Lower limb/leg weakness 72%
Visual problems 60%
Facial weakness 54%
Slurred speech 50%
Bladder control 50%
Swallowing 45%
Aphasia 33%
Depression 33%
Bowel control 33%
Dementia 30%
Inattention/neglect 28%
Emotionalism within six-months 20%
Emotionalism post six-months 10%
Table 2.2: Effects of stroke. [18]
A transient ischemic attack (TIA) is like a mini-stroke, where symptoms resolve within
24 hours.
There are many symptoms of stroke, like sudden numbness or weakness of some part of
the body, sudden confusion, trouble speaking or understanding, sudden trouble seeing in
the eyes, sudden trouble walking, loss of balance, or sudden severe headache. [18, 23, 24]
Hemiparesis is a very common disease after a stroke, and it consists in some degree
of trouble moving one side or suffer for weakness on one side of their bodies. People
with hemiparesis may have trouble moving their arms and legs, walking and loss of
balance. Right-sided hemiparesis occurs when the left side of the brain is injured, and it
affects to language and speaking. On the other hand, left-sided hemiparesis occurs when
the right side of the brain is injured and affects to the memory, attention, non-verbal
communication and how to learn. Then, damage in the lower part of the brain affect
the movement, and is called ataxia. [18, 23, 24]
There are two types of hemiparesis: pure motor hemiparesis (face, arm and leg weakness)
and ataxic hemiparesis syndrome (weakness or clumsiness on one side of the body). [24].
Background 9
Strokes are life-changing events that can affect a person physically and emotionally. The
goal of stroke rehabilitation is to help to relearn skill that were lost. There exist reha-
bilitating activities to have part or fully recover like speech therapy, physical therapy,
occupational therapy, join a support group with common mental health and also support
from friends and family.
Furthermore there are Technology-assisted physical activities: Functional electrical stim-
ulation, robotic technology, wireless technology, virtual reality and noninvasive brain
stimulation. Finally, there are experimental therapies, like biological (stem cells) or
alternative medicine (massages, acupuncture. . . ). [22]
Brain tumors can be also very dangerous. There can be malignant or benign, and the
type of treatment depends or many factors, and can be surgery, chemotherapy and
radiation therapy.
Neurodegenerative diseases cause your brain and nerves to deteriorate over time. Some
of them are Alzheimer, Huntington, amyotrophic, Parkinson and dementia. All of them
have permanent damage but can be controlled by some treatments like medication.
Finally, mental disorders like anxiety, depression and schizophrenia are so common, and
there could be treated by medication or therapy. [19, 21]
Background 10
2.2 BCI
For many years people have dreamed and speculated with the possibility of communicate
and control the human brain with computer or robots. Over the past twenty five years,
this dream has been brought to fruition by several researches and scientific programs,
being nowadays one of the fastest-growing areas of scientific research. This technology
is called Brain-Computer Interface (BCI). [10, 25, 26]
2.2.1 BCI definition
A brain-computer interface is a device that responds to neural processes from the brain
to provide a direct communication pathway between the brain and an external device
without the use of the normal neuromuscular pathways. [27] So the possibility that
BCI allows a person to communicate with or control the external world without using
common neuromuscular pathways brings hope to persons who suffers from neurological
disorders, such severe motor disabilities, amyotrophic lateral sclerosis, spinal cord injury
or stroke. [26]
A BCI could be defined as a system that measures and analyses brain signals and converts
them in real-time into outputs that do not depend on the normal output pathways of
peripheral nerves and muscles. [26]
Figure 2.6: BCI system.
Background 11
We can distinguish between two types: dependent and independent BCIs. A dependent
BCI does not use the brain’s normal output pathways to carry the message, but activity
in this pathways is needed to generate the brain activity that does carry it, like for
example a systems that uses visual evoked potentials (VEPs) to detect gaze direction.
In contrast, an independent BCI does not depend in any way on the brain’s normal
output pathways, the message is not carried by muscles and nerves, being activity in
these pathways not needed to generate the brain activity that does carries the message,
like for example a system that uses a P300 evoked potentials produced by the user with
his intent. [10, 26]
A variety of neurophysiologic signals reflecting in-vivo brain activities might be recorder
and used to drive a BCI. Two types of brain activities may be monitored: electro-
physiological and hemodynamic Electro-physiological activity is generated by electro-
chemical transmitters exchanging information between the neurons, and hemodynamic
response is a process in which the blood releases glucose to active neurons at a greater
rate than in the area of inactive neurons. [26, 28]
A neuroimaging modality classification is summarized in the table below, including elec-
trophysiological methods such as electroencephalography, electrocorticography, magne-
toencephalography, and electrical signal acquisition in single neurons, and metabolic
methods such as functional magnetic resonance and near infrared spectroscopy. [28]
Neuro-
imaging
method
Activity
measured
Measu-
rement
Temporal
resolution
Spatial
resolution
Risk Portability
EEG Electrical Direct 0.05 s 10 mm Non-invasive Portable
MEG Magnetic Direct 0.05 s 5 mm Non-invasive Non-Portable
ECoG Electrical Direct 0.003 s 1 mm Invasive Portable
Intracortical
neuron
recording
Electrical Direct 0.003 s 0.05-0.5 mm Invasive Portable
fMRI Metabolic Indirect 1 s 10 mm Non-invasive Non-Portable
NIRS Metabolic Indirect 1 s 10 mm Non-invasive Portable
Table 2.3: Neuroimaging methods.[18]
Background 12
2.2.2 Electroencephalography
Electroencephalography is the neurophysiologic measurement of the electrical activity
of the brain using electrodes placed on the scalp. The resulting traces are known as
electroencephalogram (EEG) and they represent an electrical signal (post-synaptic po-
tentials) from a large number of neurons. [9]
EEG methods are based in electro-physiological signals, and can function in most en-
vironments, requiring relatively simple and inexpensive equipment, and being a good
solution in several researches and BCI systems. It is considered a non-invasive method
because their signals are recorder from the scalp. EEG is recording of electrical ac-
tivity from the scalp produced by firing of neurons from millions of neuron of similar
orientation (radial) with in the brain being an outcome of oscillations of this neuronal
assemblies which occurs at different frequencies. [10, 26, 29]
1. DELTA - 3 Hz and less (deep sleep, when awake pathological). Can be extracted
from the frontal lobe.
Figure 2.7: Delta frequency[8, 9]
2. THETA - 3.5 - 7.5 Hz (creativity, falling asleep). Can be extracted from the central
and occipital lobes.
Figure 2.8: Theta frequency[8, 9]
Background 13
3. ALPHA - 8 - 13 Hz (relaxation, closed eyes). Can be extracted from the occipital
lobe.
Figure 2.9: Alpha frequency[8, 9]
4. MU - 8 - 13 Hz (immobility). Can be extracted from the frontal lobe.
Figure 2.10: Mu frequency[8, 9]
5. BETA - 14 - 30 Hz and more (concentration, logical and analytical thinking, fidget).
Can be extracted from the frontal and temporal lobes..
Figure 2.11: Beta frequency[8, 9]
6. GAMMA - greater than 30 Hz (simultaneous processes). Can be extracted from
the frontal lobe.
Figure 2.12: Gamma frequency[8, 9]
Background 14
It is well known that each frequency band has their characteristics. For examples, alpha
waves are increased when person feel comfortable and during closed his or her eyes and if
he or she opened eyes then, the waves are decreased. Specially, mu waves are decreased
during the imaginary movement. In case β-waves, these are increased during mental
activities. In case gamma waves, these are increased when he concentrate on something.
[8, 30]
Changes in the power spectrum of different frequency bands before during or after an
event reflects changes in firing pattern of neurons (group). When there exist a decrease
in power in a frequency band, it is called “Event related desynchronisation” (ERD). On
the other hand, when there exist an increase in power in a frequency band it is called
“Event related synchronization” (ERS). [8]
Figure 2.13: ERD and ERS phenomena [10]
EEG is recorder by electrodes, being placed over the scalp and commonly based on
the International 10-20 system, which has been standardized by the American Electro
encephalographic Society. This system uses two reference points in the head to define
the electrode location. One is the nasion (at the top of the nose) and the inion (found
in the bony lump at the base of the skull). The transverse and median planes divide the
skull from these two points. The electrode locations are determined by marking these
planes at intervals of 10% and 20% .The letters in each location corresponds to specific
brain regions in such a way that A represents the ear lobe, C the central region, Pg the
nasopharyngeal, P the parietal, F the frontal, Fp the frontal polar, and O the occipital
area. [11]
Background 15
Figure 2.14: Electrode placement over scalp.[11]
BCIs fall into 5 groups based on the electrophysiological signals they use, some recorded
from the scalp: visual evoked potentials, slow cortical potentials, P300 evoked potentials,
µ- or β-rhythms, and other by implanted electrodes: cortical neuronal activity. [10, 26]
1. Visual evoked potentials (VEP):
The VEP-based communication systems depend on the user’s ability to control
gaze direction, they are considered as dependent BCI systems. They consists in
record the VEP from the scalp over visual cortex to determine the direction of the
eye gaze to be able to move a cursor, write a letter or other different functions
from several researches. [10]
2. Slow cortical potentials (SCPs):
Among the lowest frequency of the scalp recorder EEG signals, are the SCPs.
Negative ones are associated with movement and cortical activations, and positive
ones with reduced cortical activation. Users learn to control this signals for exam-
ple to move a cursor with two targets (top and bottom), or to write a document
by selecting letters in two-choice selections. [10]
Background 16
Figure 2.15: SCP BCI. [10]
3. P300 evoked potentials:
The P300 wave is a measurable direct reaction of the brain to a certain sensory,
cognitive or mechanical stimulus, typically evoke in the EEG over parietal cortex
a positive peak at about 300ms. It does not requires an initial training for the
user, and can be used for example to determine a choice by flashing different row
o columns, and counting how many times it appears [10].
Figure 2.16: P300 BCI.[10]
Background 17
4. Mu/Beta rhythms and other activity from sensorimotor cortex:
Prominent electrophysiological features associated with the brain’s normal motor
output channels are µ- and β-rhythms. The rhythms are synchronized when no
sensory inputs or motor outputs being processed. Movement or movement prepa-
ration results in a desynchronisation of the µ- and β-rhythms, referred to as ERD.
The, ERS occurs after movement when the rhythms synchronize again. This ERD
and ERS phenomenon in µ- and β-rhythms occur during imagined movements as
well, making them suitable for paralysed individuals [31].
Figure 2.17: Upper panel: Superimposed band power time courses computed for threedifferent frequency bands (10–12 Hz, 14–18 Hz, and 36–40 Hz) from EEG trials recordedfrom electrode position C3 during right index finger lifting. EEG data are triggeredwith respect to movement-offset (vertical line at t = 0 s). Lower panel: Examples ofongoing EEG recorded during right finger movement. Movement-onset at t = 0 s. [12]
Background 18
5. Cortical neuronal activity:
Metal microelectrodes can be used to record action potentials of single neurons in
the cerebral cortices during movements.
Related with it, there are the intracortical BCIs, (ECoG signals), which are an
invasive method that use action potential firing rates or local field potential ac-
tivity recorded from individual or small populations of neurons within the brain.
Signals recorded within cortex may encode more information and might support
BCI systems that require less training. [32]
Figure 2.18: Cortical neuronal activity [10]
Background 19
2.2.3 BCI operation
Any BCI consists of five essential elements: Signal acquisition, feature extraction, feature
translation, device output and the operation protocol. [26]
1. Signal acquisition:
Is the measurement of the neurophysiologic state of the brain. For example the
EEG recorder from the scalp or the surface of the brain or neuronal activity
recorder within the brain. The recorder interface tracks neural information re-
flecting a person’s intent embedded in the ongoing brain activity.
2. Feature extraction:
Here is where the signal processing starts operating in the BCI. In this step, it
extracts signal features that encode the intent of user. They can be in time-
domain or in frequency-domain, or both. An algorithm filters the digitized data
and extracts the features that will be used to control the BCI.
3. Feature translation:
In this stage, the algorithm translates these signals features into device commands-
orders. These commands can produce output as a letter, a muscle movement, cur-
sor movement. . . This algorithm may use linear methods or non-linear methods.
4. The output device:
The output device could be a computer screen, a Functional Electrical Stimulation
(FES) device, or others assistive devices, and the output could be for example to
select targets or letters, or move any muscle or robotic arm.
5. The operating protocol:
The protocol defines how the system is turned on and off, the details of and
sequence of steps in operation of the BCI, and the timing of the operation.
[10, 26]
Background 20
2.2.4 BCI in stroke
The main interest in BCI technology is to help people who suffered from neurological
disorders such as amyotrophic lateral sclerosis, stroke or other traumatic brain disorders.
One way is to substitute for the loss of neuromuscular functions by using stroke survivors
brain signals to interact with the environment instead of using their impaired muscles,
another recent way is pay attention to a motor task that required the activation or
deactivation of specific brain signals, to restore an impaired motor function. This second
way use the thoughts of moving the impaired limb instead of physically moving the
impaired limb, and is called MI, which is the mental rehearsal of physical movement
tasks, and can be used to move an impaired muscle without any physical demands.
Several researches have proven that is possible to detect this MI and motor execution
(ME) from EEG signals during ERD and ERS in µ- and β-rhythms. [27]
Functional electrical stimulation (FES) and motor imaginary have been extensively ap-
plied in the rehabilitation training of stroke patients.
Studies examining neuromuscular electrical stimulation use by individuals following
stroke report improved force production, selective activation of muscles, passive range
of motion, and reduction of abnormally high muscle tone.
Recent studies have described participants who practiced electrical stimulation-assisted
tasks involving object manipulation and reported improved selective movement and func-
tion in the arm and hand following stroke, with good results [33, 34]
In the next chapter, there are described several studies about BCI, from the beginning
of this technology, until nowadays and what is expect in the future.
In the next chapter there will be described several studies about BCI in stroke people,
since the first studies until nowadays.
Background 21
2.2.5 Functional Electrical Stimulation (FES)
Neurons are electrically active cells. [35] In Neurons, electrical impulses transmit the
information, these impulses are called action potentials (see section 2.1.2). Nerve signals
are frequency modulated with a frequency typically between 4 and 12 Hz. An electri-
cal stimulation can artificially substitute this action potential by changing the electric
potential across a nerve cell membrane (this also includes the nerve axon), inducing
electrical charge in the immediate vicinity of the outer membrane of the cell.[36]
Functional electrical stimulation (FES) delivers a shock to the muscle, that activates
nerves and makes the muscle move. FES can be used to generate muscle contraction in
paralyzed limbs to produce muscle functions. [37]
FES devices take advantage of this property to electrically activate nerve cells, which
then may go on to activate muscles or other nerves.
Figure 2.19: Implantable FES system for upper limbs [13]
Nerves can be stimulated using either surface (transcutaneous) or subcutaneous (percu-
taneous or implanted) electrodes. The surface electrodes are placed on the skin surface
above the nerve or muscle that needs to be “activated”. They are noninvasive, easy to
apply, and generally inexpensive. [37]
Chapter 3
State of the art
In this chapter there is a review of some interesting studies related to BCI. First, about
works of BCI in stroke people, then works of BCI in imaginary finger movements and
finally there is a review of some interesting algorithms based on CSP for the feature
extraction and translation.
3.1 BCI on stroke patients
First known attempts to use BCI in motor rehabilitation, although not yet in stroke
patients, were performed in the early 2000s.
In 2003, Pfurtscheller et al. [38] presented a case on a subject that was able to learn via
MI how to properly deliver electrical stimulations to hand and arm muscles, with the
result of basic hand tasks performance. It was not a stroke patient but a tetra paresis
one as result of a spinal cord injury (SCI), and the outcome was achieved using SMR
modulation.
Using neuronal spike activation of cortical cells, Hochberg et al. [39] reported in 2006
the case of a patient moving a cursor on a screen and controlling a robotic limb.
It was also in 2006 that the first study used in stroke patients was reported. Mohapp et
al. [40] did perform a study of the cortical activity on both hemispheres in a sample of
10 stroke patients using ME and MI. They found out that a more pronounced cortical
activity was appreciated in the contralesional than in the affected hemisphere, regardless
of whether the hand movement was executed or imagined with the unaffected or affected
hand.
23
State of the art 24
In 2008, Bai et al. [41] performed investigations on β-rhythm-based BCI with visual feed-
back in subjects without previous BCI training. They concluded that the sensorimotor
β-rhythm of EEG associated with human motor behavior could be a useful BCI-signal
for clinical applications, and that it would not require any exhaustive training of subjects
or patients to provide reliable results.
It was also in 2008 when the first clinical study of BCI application on stroke patients
was published. Buch et al. [42] reported the first application of a MEG-based BCI
on a sample of eight chronic stroke patients, which were using their modulation of
the µ-rhythm ERD to control a screen cursor. Patients were also receiving visual and
kinesthetic feedback upon successful completion of any MI and/or ME tasks. Despite
the fact that no significant improvements in motor outcome were achieved, the study
revealed that six out of eight patients significantly improved performance in terms of
classification accuracy (CA) with this technique.
During 2009 and 2010, Ang et al. [43, 44] carried out several studies of stroke patients
rehabilitations, comparing results when performed via simple robotic rehabilitation ver-
sus when performed via MI BCI robotic rehabilitation. After a number of sessions in
one-hour trials during several weeks, no significant differences between groups were ob-
served, both groups of hemiparetic stroke patients reached a significant improvement in
rehabilitation.
Regarding FES techniques introduction, in 2008 Fei Meng et al. [45] reported the
design of a training platform for chronic stroke patients to train their upper limb motor
functions. It was based on BCI-FES combination, where electrical stimulation was
driven by the EEG signals resulting from their intention to move wrist and/or hand,
and also applying the common spatial pattern (CSP) algorithm. A pilot study with
two chronic stroke patients was conducted, with the purpose to determine both the
feasibility for stroke patients to carry out BCI FES training for rehabilitation, as well
as to asses possible functional improvements after such training. Pilot study resulted in
an improvement of the error rate of the BCI control, that was less than 20% after 10
training sessions.
In 2009, Daly et al. [46] experimented also with a FES device, in this case delivering
electrical stimulus to the index finger extension muscles of a stroke patient, also for motor
learning purposes. Outcome was reported to be recovery of the index finger extension
after 9 sessions carried out 3 times per week during 3 weeks.
Related to imaginary fingers movement, there are some other studies on this field that
we will review in the next point of the state of the art.
State of the art 25
Also in 2009, Takahashi et al. [47] accomplished another study on FES, but on healthy
subjects this time, examining how ERD is affected by the functional electrical stimulation
(FES) on both feet. They detected a direct relationship between the FES stimulus
increase and bigger ERD extraction, suggeting that the muscular and articular sensations
induce ERD on foot motor area (Cz).
In 2010, Tan et al. [48], worked on a study related with the ability of stroke patients
to learn how to modulate their own sensorimotor rhythms to activate FES on muscles.
They reported that four out of six patients managed to do so to activate FES of the
wrist muscles. And as important as that, that they did so less than three months after
lesion, paving the way to the implementation of early stage theraphies, with the possible
inclusion of a BCI.
Following year Tam et al. [49], also carried out another study to determine minimal
set of electrodes required by an individual stroke subject for motor imagery to control
an assistive device using FES. After 20 sessions with five chronic stroke patients, and
reaching accuracy higher than 90%, this study showed that one training day with 12
electrodes using the SVM-RFE method achieved the best balance between the number
of electrodes and accuracy in the 20-session data. They also concluded that generally,
8-36 channels were required to maintain accuracy higher than 90% in 20 BCI training
sessions.
In 2010, the study of Prasad G. et al. [50] with five chronic stroke patients confirmed
the usefulness of MI-based BCI and physical practice combination. In the study they
evaluated not only BCI performance but also other clinical indices such as patients upper
limb movement control, fatigue and mood.
Also in 2010, but again only in healthy subjects, Tavella et al. [51] proposed a senso-
rimotor rhythm based BCI-FES to manipulate objects and carry out daily living tasks,
obtaining a good timing performance and low error rate.
Back to studies of combined theraphies for motor recovery and brain reorganization,
in 2010 and 2011 Broetz et al. [52] and Caria et al. [53] did investigate the effect of
combining BCI training with physical therapy in chronic stroke patients and reported a
significant recovery of the hand motor function. The authors highlighted the significant
brain plasticity and recovery effect got by the BCI and functional neuroimaging.
State of the art 26
Overall BCI benefits are applicable to majority of stroke patients, as derived from the
large clinical study performed by Ang et al. [54] in 2011 over 54 patients. They used
EEG-based MI BCI which was demonstrated to able to be used by the majority of stroke
patients, and suggests the convenience to try to reinforce and spread the practice of BCI
screening for any stroke intervention. Other studies to analyze the effect of combining
BCI with transcranial direct current stimulation (tDCS) were carried out in 2012 by Ang
et al. [55] and Kasashima [56]. tDCS resulted in bigger ERD so that it would be advised
to be used in modulating MI in stroke patients, and also stressing the importance of
BCI feedback for stroke rehabilitation.
In the same direction aims Mihara et al. [57] in 2013 when they show greater motor
improvements in patients using BCI with visual feedback.
[27, 28]
In conclusion, there have been several studies about BCI on stroke patients, and these
studies are growing sharply in the last years. Motor imagery EEG has been widely
performed recently because of its discriminative property and inexpensive acquisitions.
Using ME and MI tasks through EEG signals have given good results, and the senso-
rimotor β-rhythm is a real useful BCI-signal for clinical applications. Furthermore, to
use a FES system, it has been proved in some studies like Daly et al. [46] and Tam et
al. study [49] with good results as a post-stroke recovery system.
Most of the articles remark that the BCI feedback is important in a BCI system for
stroke rehabilitation.
State of the art 27
3.2 BCI Imagery finger movements
We can start the review with one study of the year 2005: Liu et al. [58], where finger
movement was used as the basic and typical tasks to be identified in the BCI exper-
iments. BP and ERD ideas were introduced and discussed in this study. The CSSD
(common spatial subspace decomposition) algorithm was used for classifying single-trial
EEG during the preparation of left-right finger movements after the two kinds of phe-
nomena were expounded in detail. Experiment and simulating results were got with up
to the 75,6% of averaged classification accuracy .
Also in 2005, Bai et al. [59] studied ERD and their spatiotemporal patterns preced-
ing voluntary sequential finger movements. Movements analized were performed with
dominant right hand and no dominant left hand. Nine subjects performed self-paced
movements consisting of three key strokes with either hand. Subjects randomized the
laterality and timing of movements. Electroencephalogram (EEG) was recorded from
122 channels. Reference-free EEG power measurements in the β-band were calculated
off-line. The results showed that for right-handers, activation on the left hemisphere
during left hand movements is greater than that on the right hemisphere during right
hand movements.
In Lehtonen et al. [60] study in 2008 tried to determine BCI control accuracy by inex-
perienced subjects. After a 20 min training, ten subjects tried to move a circle from the
center to a target location at the left or right side of the computer screen by moving
their left or right index finger. Seven out of the ten subjects were able to control the BCI
well, choosing correct target in 84%-100% of the cases, 3.5-7.7 times a minute. Their
mean single trial classification rate was 80% and bit rate 10 bits/min. These results en-
courage the development of BCIs for paralyzed persons based on detection of single-trial
movement attempts.
Xian et al. [61] went in depth in 2007 in feature extraction during finger movement
tasks. They introduced discriminative spatial patterns (DSP) for better extraction of
the difference in the amplitudes of MRPs., integrated with CSP to extract the features
from the EEG signals. And they also designed a support vector machines (SVM) based
framework as the classifier for the features. The results showed that the combined spatial
filters can realize the single-trial EEG classification better than anyone of DSP and CSP
alone. Based on this outcome, they recommend the use of such an EEG-based BCI
system with the two feature sets, one based on CSP (ERD) and the other based on DSP
(MRPs), classified by SVM.
State of the art 28
Other neuroimaging methods were also studied for finger movements. In 2009, Kubanek
et al. [62] concluded that ECoG is a reliable method for studying cortical dynamics
associated to motor functions, and becomes very appropiate and accurate for powerfull
clinically practical BCI systems. They got highly specific finger flexion time courses
from the ECoG signals.
Mohamed et al. [63] investigated the EEG in order to look for ways and means to discrim-
inate between wrist and finger movements. They used Bhattacharyya distance (BD) for
feature reduction, artificial neural networks (ANN) together with Mahalanobis distance
(MD) as classifiers, and independent component analysis (ICA) and time-frequency tech-
niques to extract spectral features based on event-related (de)synchronisation (ERD/ERS).
With these combination of techniques they demonstrated the feasibility to discriminate
between wrist and finger movements with high accuracies.
A further step was achieved by Xiao R and Ding L [64, 65] in 2013 when evaluating
the feasibility of discriminating individual fingers from one hand using noninvasive EEG
in stead of the previous invasive ECoG methods. This study was finally successful
in decoding individual fingers and thus cleared a path for developing noninvasive BCI
applications with rich complexity and intuitive and flexible controls. In this study it
was demonstrated a much higher accuracy for finger movement detection of the features
from some identified different spatial and spectral patterns as compared with classic µ/δ
rhythms.
Last year, Lee et al. [66] evaluated possible adaptations of fNIRS methods for BCIs
systems. Final conclusion was that fNRIS methods allow to clearly differentiate finger
and thumb movements, opening the door for their usage within BCI systems.
In conclusion, as Bai et al. [59] said, for right-handers, activation on the left hemisphere
during left hand movement is greater than that on the right hemisphere during right
hand movement. For that reason, we are going to do the tasks with the left hand,
in order to find better results. Spatial filtering algorithms demonstrate to be a good
solution in finger imagery tasks. Also, it has been demonstrated in the Xiao R and Ding
L [65] study that is possible to discriminate between wrist and finger movements. Using
a PCA on EEG data in a finger movement task improved finger movement detection,
much better than classic µ/β rhythms. It also revealed discriminative information about
movement of different fingers.
State of the art 29
3.3 Algorithms based on CSP
Ramoser et al. [67] demonstrated that spatial filters for multichannel EEG effectively
extract discriminatory information from two populations of single-trial EEG, recorded
during left- and right-hand movement imagery, with 90.8%, 92.7%, and 99.7% of clas-
sification results. The spatial filters were estimated from a set of data by the method
of common spatial patterns and reflect the specific activation of cortical areas. They
concluded saying that the high recognition rates and computational simplicity made it
a promising method for an EEG-based BCI.
In Blanchard et al. [68], presented a method that estimates subject-specific spatial
filters which allow for a robust extraction of the rhythm modulations. The effectiveness
of the method was proved by achieving the minimum prediction error on data set IIa in
the BCI Competition 2003, which consisted of data from three subjects recorded in ten
sessions.
To address the problem of manually selecting the operational frequency band and chan-
nels group, several approaches have been proposed. Yang et al. [69] proposed a novel
channel selection method by measuring the inconsistencies from the outputs of the mul-
tiple classifiers, while Chin et al. [70] proposed DCA approach and DCR approach to
select subject-specific discriminative channels by iteratively adding or removing channels
based on the classification accuracies.
For optimization of the spectral filter, several novel approaches, namely, common spatio-
spectral pattern (CSSP) [71] were developed.
Common sparse spectral spatial pattern (CSSSP) method was proposed [72], this method
allows simultaneous optimization of an arbitrary FIR filter within CSP analysis. How-
ever, due to inherent nature of optimization problem, the solution of filter coefficients
will depend greatly on the initial points.
After, Iterative spatio-spectral patterns learning (ISSPL) [73], and filter bank common
spatial pattern (FBCSP) [74] were proposed.
Novi et al.[1] said that the change in the rhythmic patterns varies from one subject
to another, causing a time-consuming fine-tuning process in building a BCI for every
subject. To address this issue, they proposed a new method called Sub-band Common
Spatial Pattern (SBCSP), using a standard database from BCI Competition III to test it,
comparing the method with other ones used before. The results showed that it achieves
similar result as compared to the best one in the literature which was obtained by a
time-consuming fine-tuning process.
For using the CSP algorithm, is needed to set a relatively broad frequency range and
channels, or try to find subject-related frequency bands and channels. To solve this
State of the art 30
problem, Liu et al. [2] proposed an adaptive boosting algorithm to perform autonomous
selection of key channels and frequency band. Several comparisons were performed on
three datasets. Results showed that the algorithm yields relatively higher classifica-
tion accuracies compared with seven state-of-the-art approaches. Finally, these spatial
patterns (spatial weights) and spectral patterns (bandpass filters) can also be used for
further analysis of the data.
In conclusion, among various approaches developed for EEG signals, common spatial
patterns (CSP) has been proved to be one of the most effective algorithms.
Both algorithms that are going to be used in the project, developed several issues found
in other CSP algorithms: SBCSP method solve the problem of time-consuming fine-
tuning process and CSSBP method solves the problem of setting a relatively broad
frequency range and channels, or trying to find subject-related frequency bands and
channels.
Chapter 4
Signal processing
In this chapter, there are described the different methods and processes that take part
into the signal processing.
As it is explained in the background (see section 2.2.3) a BCI consists of five essential
elements: Signal acquisition, feature selection, feature translation, device output and
the operating protocol. Being the signal processing part the one that is divided into:
preprocessing, feature extraction, feature selection and classification.
The processing of EEG data is the central part of every BCI system, and the most
important one.
To do the preprocessing, it will be divided into: data loading and data handling.
From the feature extraction, selection and classification, two algorithms will be imple-