HUMAN ACTIVITY RECOGNITION USING BODY AREA SENSOR NETWORKS By BO XU Bachelor of Science in Electronic Engineering University of Science and Technology Beijing Beijing, China 2005 Submitted to the Faculty of the Graduate College of the Oklahoma State University in partial fulfillment of the requirements for the Degree of MASTER OF SCIENCE May, 2009
70
Embed
HUMAN ACTIVITY RECOGNITION USING BODY …digital.library.okstate.edu/etd/Xu_okstate_0664M_10207.pdfHUMAN ACTIVITY RECOGNITION USING BODY AREA SENSOR NETWORKS By BO XU Bachelor of Science
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
HUMAN ACTIVITY RECOGNITION USING BODY
AREA SENSOR NETWORKS
By
BO XU
Bachelor of Science in Electronic Engineering
University of Science and Technology Beijing
Beijing, China
2005
Submitted to the Faculty of the Graduate College of the Oklahoma State University in partial fulfillment of the requirements for the Degree of MASTER OF SCIENCE
May, 2009
ii
HUMAN ACTIVITY RECOGNITION USING BODY
AREA SENSOR NETWORKS
Thesis Approved: Xiaolin Li
Thesis Adviser Weihua Sheng Venkatesh Sarangan
A. Gordon Emslie
Dean of the Graduate College
iii
ACKNOWLEDGMENTS
I would like to express my gratitude to my supervisor, Dr. Xiaolin Li, whose
expertise, understanding, and patience, added considerably to my graduate experience. I
appreciate his vast knowledge and skill in many areas, and his assistance in writing this
thesis. I would also like to thank other members of my committee, Dr. Weihua Sheng,
and Dr. Venkatesh Sarangan for the assistance they provided at all levels of the research
project. Finally, I would like to thank my parents to support me go to United States,
where I learned a lot both in my study and in my life.
iv
TABLE OF CONTENTS Chapter Page 1. INTRODUCTION .....................................................................................................1
1.1 Wireless Sensor Networks .................................................................................1 1.2 Wireless Body Area Networks...........................................................................2 1.3 Hidden Markov Model .......................................................................................3 2. REVIEW OF LITERATURE AND SYSTEM ARCHITECTURE ..........................5 2.1 The Application of Hidden Markov Model .......................................................5 2.2 System Architecture for HMM Based Human Activities Identification ...........6 3. METHODOLOGY OF HUMAN ACTIVITIES RECOGNITION ...........................7 3.1 Data Collection ..................................................................................................7 3.1.1 Hardware Selection ...................................................................................7 3.2 TinyOS ...............................................................................................................9 3.3 Data Collection ..................................................................................................9
3.3.1 Sensor Nodes Data Collection ..................................................................9 3.3.2 Wireless Pulse Oximeter Data Collection ...............................................12 3.4 Data Parsing .....................................................................................................12 3.4.1 Observation Sequence Classification ......................................................13 3.4.2 Hidden States Definition .........................................................................16 3.5 Hidden Markov Model Training using Baum-Welch Algorithm ....................17
3.5.1 Baum-Welch Algorithm ..........................................................................18 3.6 Hidden States Decoding ...................................................................................20 4. EXTENDED KALMAN FILTER IN INDOOR LOCALIZATION .......................22
4.1 Kalman Filter ...................................................................................................22 4.2 Extended Kalman Filter ...................................................................................25 4.3 Application of Extended Kalman Filter in Indoor Localization ......................26
4.3.1 RSSI Based Distance measurement ........................................................26 4.3.2 System Model .........................................................................................28
v
Chapter Page
5. CONCLUSION AND FUTURE WORK ................................................................32 5.1 Human Activities Recognition Result… .........................................................32 5.2 Indoor Localization Result………………………………………………….. 40
5.2.1 Experiment Set-up ..................................................................................40 5.2.2 Experiment Result ...................................................................................42
5.3 Conclusion and Future Work ...........................................................................51 REFERENCES ............................................................................................................53
vi
LIST OF TABLES Table Page Table 2.1 PC/Laptop Side Architecture ........................................................................ 6 Table 2.2 Motes Side Architecture ................................................................................6
vii
LIST OF FIGURES Figure Page Figure 3.1 Sensor nodes distributed in one subject ....................................................10 Figure 3.2 Crossbow Micaz sensor node and MTS400 sensor board ........................11 Figure 3.3 Data Collection Mechanism .....................................................................12 Figure 3.4 Four Parts of Wireless Pulse Oximeter ....................................................12 Figure 3.5 Component Structure in Wireless Pulse Oximeter ...................................13 Figure 3.6 Observation Sequence Classification .......................................................15 Figure 3.7 States Transition Form for Hidden States ................................................17 Figure 4.1 Relation between distance and RSSI ........................................................28 Figure 5.1 Reluctant Acceleration Data for Proposition 1 .........................................32 Figure 5.2 Hidden States Decoding for Proposition 1 ...............................................33 Figure 5.3 Reluctant Acceleration Data for Proposition 2 .........................................34 Figure 5.4 Hidden States Decoding for Proposition 2 ...............................................34 Figure 5.5 Reluctant Acceleration Data for Proposition 3 .........................................36 Figure 5.6 Hidden States Decoding for Proposition 3 ...............................................36 Figure 5.7 Accuracy of human activities recognition with HMM and without
HMM........................................................................................................37 Figure 5.8 Subject using his/her own model and Subject using other trainee’s
model........................................................................................................38 Figure 5.9 Comparison of accuracy for human activities with one sensor, two
sensors and three sensors mounted on each subject ................................39 Figure 5.10 Comparison of accuracy among sensors mounted on different parts of
human body ............................................................................................40 Figure 5.11 Two target nodes deployed vertically to record three axis
acceleration data .....................................................................................42 Figure 5.12 Real route coordinate, EKF implemented coordinate, in a
1.8m*2.1m*1.5m area (walk diagonally) ..............................................42 Figure 5.13 X-axis data for EKF coordinate, real route coordinate ...........................43 Figure 5.14 Y-axis data for EKF coordinate, real route coordinate ...........................43 Figure 5.15 Z-axis data for EKF coordinate, real route coordinate ...........................44 Figure 5.16 Average error for EKF in three axis in the 1.8m*2.1m*1.5m area ........44 Figure 5.17 Real route coordinate, EKF implemented coordinate, calculated
coordinate in a 1.8m*2.1m*1.5m area (walk straightly) .......................45 Figure 5.18 X-axis coordinate of EKF and real route ................................................46 Figure 5.19 Y-axis coordinate of EKF and real route ................................................46 Figure 5.20 Z-axis coordinate of EKF and real route ................................................46 Figure 5.21 Average error for EKF in three axis in the 1.8m*2.1m*1.5m
viii
Figure Page
area .........................................................................................................47 Figure 5.22 Real route coordinate, EKF implemented coordinate, in a
1.8m*6m*1.5m area (walk diagonally) .................................................48 Figure 5.23 X-axis coordinate of EKF and real route ................................................49 Figure 5.24 Y-axis coordinate of EKF and real route ................................................49 Figure 5.25 Z-axis coordinate of EKF and real route ................................................49 Figure 5.26 Average error for EKF in three axis in the 1.8m*6m*1.5m
area (walk diagonally) .............................................................................50
1
CHAPTER 1
INTRODUCTION
1.1 Wireless Sensor Networks
As an emerging technology that bridges the physical world and the digital
information world, wireless sensor networks provide a tiny, low-power and low cost but
powerful platforms for data collection and transmission [1]. A wireless sensor network
consists of distributed smart devices equipped with processor, radio, memory, and
sensors, monitoring physical or environmental conditions cooperatively in multiple
modalities, for example, temperature, humidity, sound, motion, vibration and so on.
Wireless sensor networks are now used in many areas, such as traffic control, precision
agriculture, environment and habitat monitoring, healthcare applications, home
automation, and so on.
Wireless sensor networks consist of many sensor nodes which can be considered as
miniature computers. Each sensor node is equipped with a microcontroller, a radio
transceiver or other communications device, and an energy supply, usually a battery.
This thesis will focus on human activity recognition using wireless sensor networks.
Each human being will be equipped with several sensor nodes in his/her body to build up
a body area sensor network. Sensor data will be saved and later processed for training and
analysis purpose. The other topic investigated in this thesis is indoor localization. Our
method uses RSSI based distance measurement and Extended Kalman Filter. The
2
distance measured is used as input data in the Extended Kalman Filter to get the location
of a target node. In the next section, we will briefly introduce body area sensor networks.
1.2 Wireless Body Area Sensor Networks
Wireless body area sensor networks (WBASNs) are an extension of wireless sensor
network,which focus on the application of wireless sensor networks in human body.
The health care system will benefit from introducing the continuous and low-cost
monitoring tools with real time updates of medical sensor reading. A wearable wireless
body area network can integrate several smart sensors being used for computer assisted
rehabilitation and early detection of medical conditions.
T.G Zimmerman firstly presented the concept of wireless body sensor networks in the
article [2]. There is a lot of space for research in the WBASNs. WBASNs consist of
sensor nodes which can be either wearable or inside human body, monitoring vital data of
the subject and then send the data to the gateway. This thesis will only cover the
application of wearable sensors mounted on human body.
In WBASNs, two types of device are necessary: The first type of device is the wearable
sensor device which can transmit data to host. The second type of device is the host, such
as a base node connected with the laptop. A number of wearable sensor nodes mounted
on different parts of human body such as arms, legs, toes, and waist will transmit the data
to host where raw data will be processed.
With the growing technology in wireless sensor network and wearable computing,
human activities monitoring and physiological monitoring become practical. We briefly
review the previous work on human activities monitoring in wireless sensor networks.
3
Live Net is a long-term health monitoring project conducted in the media lab of MIT,
which provides human activities monitoring and danger warning mechanism based on the
real time sensor data.[9] In [7], measured acceleration and angular velocity data gathered
through wearable sensors is introduced to determine a user’s location between pre-
selected locations and classify walking ,sitting, and standing behaviors. In [8], a gait
assessment system is developed for elderly monitoring. And many others [10,11,12,13,14]
also discuss physical activity recognition using acceleration sensors.
In this thesis, we will use Crossbow Micaz sensor node attached with Crossbow
MTS400 accelerometer sensor board to get acceleration from different parts of human
body. Three sensor nodes are placed at different parts of the subject, which are left toe,
right outer leg joint, and right side of waist. We will use another sensor node attached
with wireless pulse oximeter to get the heart rate data. Heart rate data plays an assistant
role in this thesis, which will be used for fall detection.
Series of human activities can be regarded as Markov process. Individual human
activity can be regarded as a state in Markov model. The activity series can not be
observed directly from acceleration data. Therefore, Hidden Markov Model (HMM) is
introduced to find the most probable hidden states of human activities. This thesis uses
observation sequence (acceleration data) which generated by sensor nodes to estimate the
hidden states (human activity series) under the HMM.
1.3 Hidden Markov Model
Hidden Markov Model is a statistical model. In a Hidden Markov Model, the system
being processed is considered as a Markov process, which has unknown parameters. The
4
hidden parameters are determined from the observable sequence. The state transition
probabilities are the only parameters in a Markov Model as the state is observable
according to the observation sequence. In Hidden Markov Model, the state is not directly
observable according to the observation sequence. The application of Hidden Markov
Model is to discover the sequence of states using observable sequence. There are three
common algorithms to solve problems regarding to Hidden Markov Model:
Forward-Backward algorithm: If the parameters of HMM have been given, calculates
the probability of an observed sequence and figure out the hidden state associated with
the observed sequence.
Viterbi Algorithm: If the parameters of HMM and observation sequences have been
given, find out the hidden states which lead to the highest probabilities that generate the
observed sequence.
Baum-Welch Algorithm: If the parameters of HMM are not given, only the
observation sequence is given, find out the value of state transition probabilities and
observation transition probabilities. The parameters of HMM will be fully discovered
after running the Baum-Welch Algorithm.
In this thesis, firstly, we define the parameters for the Hidden Markov Model
according to the system being modeled. Secondly, we have the system model trained by
using Baum-Welch algorithm. Finally, the Viterbi algorithm is introduced to identify the
activity series based on the new observation sequences. Meanwhile, forward-backward
algorithm is conducted in order to compute the probability of a particular output sequence.
The indoor localization will be discussed in the later chapter.
5
CHAPTER 2
REVIEW OF LITERATURE AND SYSTEM ARCHITECHTURE
2.1 The application of Hidden Markov Model
Hidden Markov Model (HMM) was introduced and developed in the late 1960s [15].
L.E Baum, T. Petrie, N. Weiss, and G. Soules published the famous Baum-Welch
Algorithm that solved the model training problem [16] in 1968. The Baum-Welch
Algorithm can be regarded as an Expectation-Maximization (EM) method for maximum
likelihood calculation. In [20], properties of the EM algorithm are proved by C.F.J. Wu.
In [21], S.E. Levinson, L.R. Rabiner, and M.M. Sondhi published another EM method for
HMM training which laid a solid foundation for the application of HMM. Thereafter,
HMM was widely used in speech recognition [25]. L.R. Rabiner gave a classic tutorial on
how to apply the Hidden Markov Model to speech recognition in [24]. Nowadays, HMM
is also used for handwriting recognition [23] and other areas.
Introducing Hidden Markov Model for human activities recognition in wireless sensor
networks is a new research area. A project named SATIRE carried out by UIUC provides
a smart jacket which monitors the daily activities of human being and performs the
outdoor localization with the help of GPS component mounted on the sensor node. The
smart jacket serves as a wearable platform for personal health monitoring. [28]. A study
integrated with optimization approach in 3-D model based framework is introduced in
[29] for posture estimation and a motion-tracking scheme with GA based particle filter is
6
described as a solution for motion capture problem.
2.2 System Architecture for HMM Based Human Activities Identification
The following tables show the system architecture for human activities recognition in
this thesis.
Application Layer (Simple user interface in Cygwin) Interpretation Layer (Hidden states decoding) Parsing Layer (Raw sensor data parsing) Physical Layer (USB connection between PC/Laptop and Gateway)
Table 2.1 PC/Laptop side architecture
Application Layer (motes programmed for realizing certain function) Sensor Layer (Synchronization protocol, Memory logging protocol) Physical Layer (UART and RF radio communication)
Table 2.2 Motes side architecture
In chapter three, we will show the choice of hardware, mechanism of data collection and
data parsing. After that, we will train the system model with Baum-Welch algorithm
using observed sequences that come from the acceleration data generated by sensor nodes.
In the hidden states decoding step, Viterbi algorithm is applied to find the Viterbi path,
which is the most probable hidden states that lead to the observation sequences. In
chapter five, we will show all the result graphs and discuss several issues regarding to
Hidden Markov Model and indoor localization with Extended Kalman Filter.
7
CHAPTER 3
METHODOLOGY OF HUMAN ACTIVITIES RECOGNITION
3.1 Data Collection
3.1.1 Hardware Selection
In this thesis, we use Crossbow Micaz sensor node with Crossbow MTS400 sensor
board to get sensor data. Crossbow Micaz sensor is equipped with a 7.38 MHz Atmel
processor, 128 KB program memory, 4 KB RAM, and 512 KB non-volatile storage. The
radio component is a Chipcon SmartRF CC2420, with 2.4GHz frequency, Manchester
encoding, and linear RSSI (received signal strength indicator). Output power is digitally
programmable by setting the PA POW register.
MTS400CA sensor board utilizes the latest generation of IC-based surface mount
sensors. It combines several independent sensors into one sensor board. Specifications
are Dual-axis Accelerometer: Analog Devices ADXL202JE with Acceleration range ±2g
[40] V. Ramadurai and M. Sichitiu, “Localization in wireless sensor networks: A
probabilistic approach,” 2003.
[41] A. Smith, H. Balakrishnan, M. Goraczko, and N. Priyantha, “Tracking moving
devices with the cricket location system,” 2004.
58
[42] Greg Welch and Gary Bishop, “An Introduction to the Kalman Filter”, Technical
Report TR 95-041
[43] Marko Helen, Juha Latvala, Hannu Ikonen, Jarkko Nittylahti, “Using Calibration in
RSSI-Based Location Tracking System”, Proceedings of the 5th World Multiconference
on Circuits, Systems, Communications & Computers (CSCC20001)
[44] Cesare Alippi, Giovanni Vanini, “A RSSI-based and calibrated centralized
localization technique for Wireless Sensor Networks,” Proceedings of the Fourth Annual
IEEE International Conference on Pervasive Computing and Communications
Workshops (PERCOMW’06).
[45] K. Whitehouse, C. Karlof, A. Woo, F. Jiang, and D. Culler. The effects of ranging
noise on multihop localization: an empirical study. In The Fourth International
Conference on Information Processing in Sensor Networks (IPSN ’05). Los Angeles,
California, Apr. 2005.
[46] J. Latvala, J. Syrjärinne, S. Niemi, J. Niittylahti , “Patient Tracking in a Hospital
Environment Using Wireless Stations and Extended Kalman Filtering”, Proceedings of
the 1999 Middle East Conference on Networking.
[47] Flavio Cabrera-Mora, and Jizhong Xiao, “Preprocessing Technique to Signal
Strength Data of Wireless Sensor Network for Real-Time Distance Estimation,” 2008
IEEE international Conference on Robotics and Automation Pasadena, CA, USA, May
19-23, 2008
[48] A. P. Jardosh, E. M. Belding-Royer, K. C. Almeroth and S. Suri, “Real-world
environment models for mobile network evaluation,” IEEE Journal on Selected Areas in
Communications, vol. 23, No. 3, pp. 622-632, March 2005.
59
[49] J. Blumenthal, F. Reichenbach and D. Timmermann, “Minimal transmission power
vs. signal strength as distance estimation for localization in wireless sensor networks,” In
Proceedings of the 3rd IEEE International Workshop on Wireless Ad-hoc and Sensor
Networks (IWWAN), New York, NY, USA. June 2006.
[50] K. Woyach, D. Puccinelli and M. Haenggi, “Sensorless sensing in wireless networks:
implementation and measurements,” In Proceedings of the Second International
Workshop on Wireless Network Measurement (WiNMee), Boston, MA, USA. April 2006.
[51] K. Lorincz and M. Welsh, "MoteTrack: A Robust, Decentralized Approach to RF-
Based Location Tracking," Proceedings of International Workshop on Location and
Context-Awareness (LoCA 2005), May 2005.
[52] L. Hu and D. Evans, “Localization for mobile sensor networks,” in roceedings of the
10th annual international conference on Mobile computing and networking, 2004, pp.
45–57.
[53] A. Galstyan, B. Krishnamachari, K. Lerman and S. Pattem, "Distributed Online
Localization in Sensor Networks Using a Moving Target," INTERNATIONAL
SYMPOSIUM ON INFORMATION PROCESSING IN SENSOR NETWORKS, vol. 3, pp.
61-70, 2004.
[54] R. R. Brooks, P. Ramanathan, and A. Sayeed, “Distributed target tracking and
classification in sensor network,” Proceedings of the IEEE, Vol. 91, No. 8, pp. 1163-
1171, August 2003.
[55] Pritpal Dang, Prasanna Ballal, Frank Lewis, Dan Popa, Real Time Relative and
Absolute Dynamic Localization of Air-Ground Wireless Sensor Networks.
60
[56] Ali Shareef, Yifeng Zhu, and Mohamad Musavi, “Localization Using Neural
Networks in Wireless Sensor Networks,” Mobilware’08, February 12-15, 2008,
Innsbruck, Austria.
VITA Bo Xu
Candidate for the Degree of
Master of Science or Arts Thesis: HUMAN ACTIVITY RECOGNITION IN WIRELESS SENSOR NETWORK Major Field: Computer Science Biographical:
Personal Data: Born in Dongying, Shandong, China on August, 27th, 1982. Education: Completed the requirements for the Master of Science or Arts in computer science at Oklahoma State University, Stillwater, Oklahoma in May, 2009. Received B.S Degree of Electronic Engineering from University of Science and Technology Beijing, Beijing, China, 2005. Experience: Worked for China Petroleum Engineering Co. Ltd as system administrator from October, 2005 to May, 2006.
ADVISER’S APPROVAL: Xiaolin Li
Name: Bo Xu Date of Degree: May, 2009 Institution: Oklahoma State University Location: Stillwater, Oklahoma Title of Study: HUMAN ACTIVITY RECOGNITION USING BODY AREA SENSOR
NETWORKS Pages in Study: 60 Candidate for the Degree of Master of Science.
Major Field: Computer Science Scope and Method of Study: In this thesis, we model the series of human activity as Markov process. Hidden Markov Model (HMM) is used for human activities recognition. Acceleration data which are continuously collected from wearable sensor mounted on human body are imported as observation sequence for HMM. HMM is established and applied to recover the hidden states in human activity recognition. In the second part of this thesis, we apply Extended Kalman Filter (EKF) for indoor target localization. We use Received Signal Strength Indicator (RSSI) to measure the direct distance between target node and anchor nodes. The measured distance is used as the measurement function in EKF. Besides, acceleration data of target node is recorded for system input in EKF as well. In this way, we demonstrate that EKF gives reasonably accurate estimation for tracking the target node in indoor environments. Findings and Conclusions: The experimental results show that Hidden Markov Model (HMM) outperforms using acceleration data directly in human activities recognition. We also demonstrate the accuracy of mounting one, two and three sensors on human body and the accuracy of mounting sensors in different parts of human body. Extended Kalman Filter (EKF) obtains reasonable accuracy in tracking target node within a relatively small indoor environment. HMM and EKF methods can be applied to many areas such as patient monitoring, firefighter monitoring and so on.