DEVELOPMENT OF FACIAL MUSCLES CONTROLLED ROBOTIC …
Post on 18-Dec-2021
3 Views
Preview:
Transcript
DEVELOPMENT OF FACIAL MUSCLES
CONTROLLED ROBOTIC SYSTEM
BY
AHMAD JORAIMEE BIN MOHAMAD
A dissertation submitted in partial fulfillment of the
requirements for the degree of Master of Science
(Mechatronic Engineering)
Kulliyyah of Engineering
International Islamic University
Malaysia
MAY 2013
i
ABSTRACT
The work in this dissertation focuses on the development of the facial muscles
controlled robotic system. The main idea of the work is to control the movement of
the small mobile robot by the recorded facial muscle signals. The characteristic of the
recorded facial muscles is discussed thoroughly in the dissertation. The facial muscle
signals used, which are known as Electromyography (EMG) are obtained after passing
the raw signals through EMG system that consist of stage-by-stage amplification and
filtering circuits. A set of three channels of EMG circuit consists of a set of
instrumentation amplifier INA128P and four precision amplifiers OP177 are used to
build the circuit. The raw captured signal is amplified by the gain of 8,415 and band
pass filtered from 50Hz to 500Hz. A microcontroller PIC16F877A starter kit is used
to convert the analogue signal from the circuit into digital form before the signal is
transferred to the receiver on the small mobile robot wirelessly by using XBee module
which can work within 100m radius. The alternative features of the system can be
used to create more interesting facial muscle rehabilitation therapy for the patients
who suffer from strokes.
البحثملخص
. الإنسان الآلييتركزالعمل في هذه الأطروحة على تطوير عضلات الوجه بواسطة السيطرة عن طريق عن طريق الإشارات المحمول الصغير الإنسان الآليالعمل هو السيطرة على حركة الفكرة الرئيسية لهذ
المسجله لعضلات الوجه في هذه الأطروحه تم مناقشة المزايا . عضلات الوجه المسجلة من الصادرةيتم الحصول عليها بعد (EMG)إشارات عضلات الوجه المسجلة والمعروفه بنظام ال . بشكل مستفيظ
والذي يتكون من دوائر التضخخيم والترشيح مرحلة تلو EMG المرور بالإشارات الأصليه خلال نظام التتكون من مجموعة الأجهزه المكبره (EMG)ن من ثلاث قنوات من الدوائر هناك مجموعة تتكو .أخرى
INA128P وآربعة مكبرات ضبطOP177 الإشارة الاصلية المحتجزه تكبر عن . تستخدم لبناء الدائرةوالذي PIC16F877A المتحكم الدقيقهرتز. 500هرتز الى 50موجة ترشيح من 8,415طريق اكتسابها
الإنسان الآليوتنقل إلى المستقبل في هو البادئ يستخدم لتغيير الإشارة الرقمية قبل أن تبدأ الإشارةمميزات هذ النظام م. 100والذي يعمل بنصف قطر XBee المحمول الصغير لاسلكيا بواسطة نموذج
ين يعانون من السكتات هتمام بتأهيل عضلات الوجه لعلاج المرضى الذللإهوامكانية خلق إثارة أكثر الدماغية.
iii
APPROVAL PAGE
I certify that I have supervised and read this study and that in my opinion it conforms
to acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Master of Science in Mechatronics
Engineering.
…………………………………..
Shahrul Naim Sidek
Supervisor
I certify that I have read this study and that in my opinion it conforms to acceptable
standards of scholarly presentation and is fully adequate, in scope and quality, as a
dissertation for the degree of Master of Science in Mechatronics Engineering
…………………………………..
Amir Akramin Shafie
Examiner
This dissertation was submitted to the Advanced Engineering & Innovation Center
and is accepted as a fulfillment of the degree of Master of Science in Mechatronics
Engineering.
…………………………………..
Iskandar Idris Yaacob
Head, Advanced Engineering &
Innovation Center
This dissertation was submitted to the Kulliyyah of Engineering and is accepted as a
fulfillment of the degree of Master of Science in Mechatronics Engineering.
…………………………………..
Md. Noor Bin Salleh
Dean, Kulliyyah of Engineering
iv
DECLARATION
I hereby declare that this dissertation is the result of my own investigations, except
where otherwise stated. I also declare that it has not been previously or concurrently
submitted as a whole for any other degrees at IIUM or other institutions.
Ahmad Joraimee bin Mohamad
Signature: ………………………………………. Date: ………………..
v
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
DECLARATION OF COPYRIGHT AND AFFIRMATION
OF FAIR USE OF UNPUBLISHED RESEARCH
Copyright © 2013 by International Islamic University Malaysia. All rights reserved.
DEVELOPMENT OF FACIAL MUSCLES CONTROLLED
ROBOTIC SYSTEM
No part of this unpublished research may be reproduced , stored in a retrieval
system, or transmitted, in any form or by any means, electronic, mechanical,
photocopying, recording or otherwise without the permission of the copyright holder
except as provided below.
1. Any material contained in or derived from this unpublished research may
only be used by others in their writing with due acknowledgement.
2. IIUM or its library will have the right to make and transmit copies (print
or electronic) for institutional and academic purpose.
3. The IIUM library will have the right to make, store in retrieval system and
supply copies of this unpublished research if requested by other
universities and research libraries.
Affirmed by Ahmad Joraimee bin Mohamad
……………………….. …….……………….
Signature Date
vii
ACKNOWLEDGEMENT
First and foremost, praise to Allah who has given me guidance and support in
completing this research work and receiving His blessing.
I would like to express my deepest appreciation to my supervisor, Dr. Shahrul
Naim Bin Sidek for all the supports he has given towards completing this research
work. His precious time and dedication has pushed me through the difficult gain.
Also a special thanks to the staff of Faculty of Electrical and Automation
Engineering Technology TATI University College especially En Mohd Tarmizi
Ibrahim and En Kharuddin Ali for giving me the technical support and advises, which
has significantly contributed to the success of this research work.
Last but not least, a very special thanks to my parent and wife, Dr Norulakma
bt Zainon for their boundless love and patience to encourage me towards realizing my
goals.
viii
TABLE OF CONTENTS
Abstract .......................................................................................................................... i Abstract in Arabic ........................................................................................................ ii
Approval Page ............................................................................................................. iii Declaration ................................................................................................................... iv Acknowledgement ...................................................................................................... vii List of Figures ............................................................................................................... x List of Tables ............................................................................................................. xiii
List of Symbols .......................................................................................................... xiv
List of Abbreviations ................................................................................................. xv
CHAPTER ONE: INTRODUCTION ........................................................................ 1 1.1 Background ................................................................................................. 1 1.2 Problem statement ....................................................................................... 2 1.3 Research objectives ..................................................................................... 3
1.4 Research Methodology................................................................................ 3 1.5 Research Scope ........................................................................................... 6 1.6 Thesis organization ..................................................................................... 7
CHAPTER TWO: LITERATURE REVIEW ........................................................... 8 2.1 Introduction ................................................................................................. 8
2.1.1 Electrocardiography (ECG) ............................................................. 8
2.1.2 Electroencephalography (EEG) ....................................................... 9 2.1.3 Electrooculography (EOG) ........................................................... 10
2.1.4 Electroretinography (ERG) ........................................................... 10 2.1.5 Electromyography (EMG)............................................................. 11
2.2 Method for Rehabilitation of Facial Muscle for Stroke Patient ................ 12
2.3 Review of previous researches and projects ............................................. 15 2.4 Summary ................................................................................................... 22
CHAPTER THREE: SYSTEM DEVELOPMENT AND DESCRIPTION ......... 23 3.1 Introduction ............................................................................................... 23 3.2 Block Diagram of the Research ................................................................ 23
3.3 Flow Chart of the Research ....................................................................... 24 3.4 Relation between Facial Muscles and Facial Expression ......................... 27
3.5 Method to Acquire EMG signal ................................................................ 29 3.6 Signal conditioning ................................................................................... 30
3.6.1 Electrode ........................................................................................ 31 3.6.2 Amplifier ....................................................................................... 33 3.6.3 Bandpass Filter .............................................................................. 34
3.6.4 Precision full-wave rectifier circuit ............................................... 38 3.7 PIC Microcontroller .................................................................................. 42
3.8 Radio Frequency (RF) communication ..................................................... 44 3.8.1 X-Bee transreceiver ....................................................................... 45
3.9 Small mobile robot .................................................................................... 46
3.10 Summary ................................................................................................... 48
ix
CHAPTER FOUR : RESULTS AND DISCUSSION ............................................. 50 4.1 Introduction ............................................................................................... 50 4.2 Analysis of Signal Conditioning Circuit ................................................... 50
4.3 EMG Classification Algorithm for Muscle Contraction Identification .... 57 4.4 Experiment Setup ...................................................................................... 58
4.4.1 Algorithm for data input and output. ............................................. 58 4.5 Summary ................................................................................................... 63
CHAPTER FIVE : CONCLUSION AND RECOMMENDATION ...................... 64 5.1 Conclusion ................................................................................................ 64 5.2 Recommendation....................................................................................... 65
BIBLIOGRAPHY ...................................................................................................... 67
APPENDIX I: PIC PROGRAMMIMG ................................................................... 69
x
LIST OF FIGURES
Figure No Page No
1.1 Research Methodology workflow 6
2.1 Typical ECG recording illustrating the beating of the heart 9
2.2 Electrodes used to record ERG (Creel, 2008) 11
2.3 (a) Electroretinography (ERG) test and (b) Basic waveform of ERG 11
2.4 EMG block diagram used in the experiment 16
2.5 Scenery testing of the research 16
2.6 Electrode placement in the experiment 17
2.7 A prototype cricket car 18
2.8 The neck of AIBO controlled by the movement of human wrist 18
2.9 Rehabilitation robot configurations 19
2.10 Transmitter and receiver block diagram of the research. 19
3.1 Block diagram of the Facial EMG system to control the small mobile robot 23
3.2 Flowchart of the EMG system implementation 26
3.3 Position of the facial muscles selection for electrodes placement 28
3.4 Muscle contraction: (a) Right cheek muscle contraction (b) Left cheek muscle
contraction (c) Both cheek muscle contraction (d) Forehead muscle
contraction 29
3.5 Electrodes (a) Surface Electrode (b) Needle Electrode 30
3.6 Signal conditioning process 31
3.7 AgCl electrodes 32
3.8 Electrodes position 32
3.9 Schematic for INA 128P connection 34
xi
3.10 Constructed Pre-Amplifier INA128P connection 34
3.11 50Hz High pass filter with unity gain 36
3.12 500Hz Low-pass with 165 gain 37
3.13 Constructed High Pass and Low Pass filter 37
3.14 Precision of full-wave rectifier circuit 39
3.15 Constructed Full wave rectifier 39
3.16 Complete schematic circuit of the Facial EMG interface 40
3.17 Complete EMG circuit simulate using MultiSim 41
3.18 PIC16F877A starter kit with PIC16F877A processor and LCD 42
3.19 PIC16F877A pin configuration for EMG system 43
3.20 PIC16F877A pin configuration located at small mobile robot 44
3.21 Wiring connection between PIC microcontroller starter kit and XBee 46
3.22 Schematic circuit for forward reverse DC motor 48
3.23 (a) Completed three channels EMG circuitry (b) Completed small mobile
robot system 49
3.24 EMG signal captured 49
4.1 Complete system of three channels EMG circuitry, ADC and Transmitter 50
4.2 The oscilloscope show the noise value in EMG system 51
4.3 Waveform observed from Right Zygomaticus major muscle signal has been
passed through band filter 52
4.4 Waveform observed from Left Zygomaticus major muscle signal has been
passed through band filter 52
4.5 Waveform observed from Frontalis muscle signal has been passed through band
filter 53
xii
4.6 Waveform observed from Right Zygomaticus major muscle signal has been
passed through precision full-wave rectifier 54
4.7 Waveform observed from Left Zygomaticus major muscle signal has been
passed through precision full-wave rectifier 54
4.8 Waveform observed from Frontalis muscle signal has been passed through
precision full-wave rectifier 55
4.9 Series of video frames of the small scale mobile robot trajectory by contraction
of the facial muscles. 63
xiii
LIST OF TABLES
Table No Page No
2.1 Common frequencies found in EEG recordings and their associated
conditions 10
2.2 Facial exercises for stroke patient 12
2.3 Summary of the literature of researches and projects done 20
3.1 Facial muscles and location 27
3.2 List of muscle of facial and its actions 28
4.1 Voltage output after band filter with gain 8,415 when the muscle flex 53
4.2 Voltage output after full wave rectifier when the muscles flex 55
4.3 Benchmarking data for algorithm development 56
4.4 Relationship between facial movements, muscle contraction and small
mobile robot movement 57
xv
LIST OF ABBREVIATIONS
ADC Analogue to Digital Converter
CMRR Common Mode Rejection Rate
DC Direct current
ECG Electrocardiography
EEG Electroencephalography
EMG Electromyography
EOG Electrooculography
ERG Electroretinography
LCD Liquid Crystal Display
1
CHAPTER ONE
INTRODUCTION
1.1 Background
Stroke occurs when a blood vessel that supplies blood to the brain either is blocked or
bursts. The damage occurred decreases the amount of blood flow and oxygen to the
brain and leads to brain cell death. The symptoms depend on what area of the brain was
affected. Some patients experience difficulty moving their facial muscles, making facial
expressions, speech and eating difficulty after a stroke. Specific exercises can help
regain control and strength in these muscles. In this research work, an alternative
method that is using a small mobile robot as an excitation agent is proposed in order to
have a lighter facial exercise. In addition, it can also be used as an enjoyable motivator.
The method identifies patterns in the electromyography (EMG) signals from the facial
muscles, and commands the small mobile robot to perform certain tasks.
The electromyography (EMG) study is the study of muscle function through
analysis of electrical potential that emanates from the muscle itself and it began as early
as 17th century (Norali, Som, & Kangar-arau, 2009). An argentum chloride surface
electrode (AgCl) is placed on the skin above a superficial muscle that receives electrical
signals emanating from several muscle fibers, which are associated with different motor
units. These electrical signals (EMG) provide an effective means of monitoring muscle
activity. EMG signal obtained by electrode is relatively small with amplitude ranges up
to 10 mV or ±5 mV peak-to-peak (Ibrahim et al., 2008)(Raez, Hussain, & Mohd-Yasin,
2006). Therefore, it is not easily observed. In order to solve the problem, the EMG
2
signals need to be amplified by specifically designed signal conditioning circuits, so that
the signal can be useful and observed.
The EMG signals are picked up by the electrodes and then amplified. Generally,
more than one amplification stages are needed in order to eliminate the low and high
frequency noises or other factors that can affect the signal quality. The frequency of the
signal is between 0 to 500Hz but the strongest amplitudes are to be found in the range of
50 to 150 Hz band (Mei, Ying, & Zheng, 2009). The EMG signals are dependent on
many factors since the signals are susceptible to noise interference such as hum, signal
acquisition such as clipping and baseline drift, skin artifacts, processing errors, and
interpretation problems (Siriprayoonsak, 2005). During the measurement, if there is no
contraction of the muscle, there should not be any electrical potential, but when the
muscle is contracted, the action potentials appears and it depends on the strength of
contraction that determines the level of voltage captured.
Its amplitude can also be affected by parameters such as the type of electrode,
electrode distance placement and electrode orientation. Furthermore, skin preparation,
skin and electrode gel temperature, subcutaneous fat between the electrode and muscle
may also contribute to the amplitude of the signal capture.
1.2 Problem statement
Most of the stroke patients demonstrate lack of enthusiasm in having to exercise for the
stroke recovery process, especially the one that affects the facial muscle. Facial muscle
therapy through regular exercise of the facial muscles had been proven to help in muscle
recovery (May, M., & Schaitkin, 2000) . Hence, an alternative method that is using a
mobile robot as an excitation agent is proposed in this work in order to have a lighter
3
facial exercise. Apart from that, it can also be used as an enjoyable motivator. The
method identifies the patterns in the electromyogram (EMG) signals from the facial
muscles, thus command the mobile robot to perform certain tasks.
1.3 Research objectives
The main objective of this research is to develop a method to control the small mobile
robot by using the facial muscles as the input of the system. To achieve the main
objective, it is necessary:
i. To develop analog signal conditioning circuit that is able to capture and
process EMG signal from the facial muscles to become a useful signal.
ii. To analyze the EMG signal generated from difference facial expressions in
order to develop the algorithm to control robotic system.
iii. To develop a small mobile robotic platform for rehabilitation of facial
muscles.
1.4 Research Methodology
The research is about using facial expression that is governed by a set of face muscles to
control the movement of a small mobile robot. In order to achieve the objectives of the
research, the following steps are taken and the research workflow is shown in Fig 1.1.
i. Review of relevant technical and scientific literatures for available
techniques and methods to control the small mobile robot.
ii. Measurement of EMG signals. It can be done by putting an argentum
chloride surface electrode (AgCl) to the exact selected facial muscles by
using accurate method.
4
iii. Development of analogue circuitry hardware for the signal amplifier and
processing.
iv. Programming of the transceiver between the processing circuitry signal and
small mobile robot.
v. Programming the algorithm to control the small mobile robot movements.
vi. Software and hardware integration.
vii. Implementation and evaluation of the system.
5
Bio-potential signal
capture methods
Method for facial
muscle Rehabilitation
Research and projects
done
Data analysis
EMG
signal
capture
Constructed a set of
three channel analogue
EMG signal
ADC
Fed into PIC
Programming
Data
transfer/received
using XBee
Troubleshooting the
circuit
Report
Troubleshooting the
programming and
connection
Start
Literature Review
Yes
No
Yes
No
A
B
6
Fig 1.1: Research Methodology workflow
1.5 Research Scope
The scope of this research is to use facial expression that is governed by a set of face
muscles to control the movement of the mobile robot. In order to do that, a set of
electrodes are used to measure muscle activities during a particular facial expression.
The data recorded is then processed by an algorithm to control the mobile robot
movement. In this research, the mobile robot’s movements focus on the left, right,
forward and backward movements.
Constructed small
scale mobile robot
Programming the
algorithm
Small mobile robot
move accordingly
Motor Run
Troubleshooting the
programming and
connection
Finish
Report
Yes
No
A
B
7
1.6 Thesis organization
The thesis outline is:
i. Chapter 2: This chapter discusses literature review of EMG signal generated
by facial muscles, as well as hardware and software used in developing the
EMG circuitry and small mobile robot.
ii. Chapter 3: This chapter discusses the development of signal acquisition
circuit and small scale mobile robot.
iii. Chapter 4: This chapter discusses the implementation of EMG system and
the discussion of the results obtained.
iv. Chapter 5: This chapter summarizes all the results obtained in the previous
chapters. Consequently, new development and improvements are suggested
for further study.
8
CHAPTER TWO
LITERATURE REVIEW
2.1 Introduction
Today, the electrical potential of muscle can be measured by using specific tools. All of
these are begun by capturing the electric potential measured between the point in living
cells, tissue and organisms which is called bio-potential. There are many bio-potentials
commonly applied in medical and engineering such as electromyography(EMG),
electrocardiography (ECG), electroencephalography (EEG) electrooculography (EOG)
and electroretinography (ERG).
2.1.1 Electrocardiography (ECG)
Electrocardiography (ECG) is a graphic recording of electrical activity generated by the
heart (Tamil, Kamarudin, Salleh, & Tamil, 2008) shown in Fig 2.1. As the heart pumps
blood, different chambers of cardiac muscle are activated in a special order. The ECG
waveform represents the rhythmical depolarization of each chamber of the heart. The
ECG can be used to determine whether the heart is functioning properly or if there any
problem exists.
top related