PIECEWISE LINEAR DYNAMICAL MODEL FOR HUMAN ACTIONS CLUSTERING FROM INERTIAL BODY SENSORS WITH CONSIDERATIONS OF HUMAN FACTORS Jiaqi Gong 1 , Philip Asare 1,2 , John Lach 1 , Yanjun Qi 2 1 Charles L. Brown Department of Electrical and Computer Engineering 2 Department of Computer Science University of Virginia {jgong, pka6qz, jlach, yq2h}@virginia.edu ICST BodyNets, September 29 th , 2014
13
Embed
PIECEWISE LINEAR DYNAMICAL MODEL FOR … INERTIAL BODY SENSORS WITH CONSIDERATIONS OF HUMAN FACTORS ... September 29th, 2014 . ... Initialize a random sparse input 12 satisfying the
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
PIECEWISE LINEAR DYNAMICAL MODEL FOR HUMAN ACTIONS CLUSTERING
FROM INERTIAL BODY SENSORS WITH CONSIDERATIONS OF HUMAN FACTORS
Jiaqi Gong1, Philip Asare1,2, John Lach1, Yanjun Qi2 1Charles L. Brown Department of Electrical and Computer Engineering
2Department of Computer Science University of Virginia
{jgong, pka6qz, jlach, yq2h}@virginia.edu
ICST BodyNets, September 29th, 2014
Motivation 2
¨ Dividing motion data into discrete segments is useful ¤ Training
¤ Diagnosis
¨ Segmentation can be at different granularities ¤ Walking vs. not-walking
Credit: A. Bulling, U. Blanke, and B. Schiele, “A Tutorial on Human Activity Recognition Using Body-Worn Inertial Sensors”, ACM Computing Surveys (CSUR), vol. 46, no. 3, pp. 33, 2014.
Problems 4
¨ ‘Human Factors’
100 200 300 400 500 600 700 800 900 1000-150
-100
-50
0
50
100
o *
100 200 300 400 500 600 700 800 900 1000-300
-200
-100
0
100
200
300
400
o *
100 200 300 400 500 600 700 800 900 1000-1000
-500
0
500
1000
100 200 300 400 500 600 700 800 900 1000-500
0
500
1000
Gyroscope data of inertial BSNs mounted insecurely
Gyroscope data of inertial BSNs mounted incorrectly
Gyroscope data of inertial BSNs mounted correctly
Heel Strike Event
Chien et al. (2013)*: Model-based estimation
*C. C. Chien, J.Y. Xu, H-I Chang, X. Wu and G. J. Pottie, “Model Construction for Human Motion Classification using Inertial Sensors”, IEEE Workshop on Information Theory and Applications, San Diego, Feb 2013
Physical placement affects signal
Addressing Problems 5
¨ Our Approach
S1
S2
Sk
… … … … …
Raw Data𝐘 𝒕 = (𝒚𝟏, 𝒚𝟐 …𝒚𝒌)
…
𝒚𝟏(𝒕) ∈ 𝑹𝒑
…
𝒚𝟐(𝒕) ∈ 𝑹𝒑
…
𝒚𝒌(𝒕) ∈ 𝑹𝒑
t Samples
Preprocessing𝒀′(𝒕) = (𝒚′𝟏, 𝒚′𝟐 …𝒚′𝒌)
t Samples
…
𝒚′𝟏(𝒕) ∈ 𝑹𝒑
…
𝒚′𝟐(𝒕) ∈ 𝑹𝒑
…
𝒚′𝒌(𝒕) ∈ 𝑹𝒑
SegmentationW= {𝒘𝟏,𝒘𝟐 …𝒘𝒎}
…
…
…𝒘𝟏 𝒘𝒎…
m Segments
Feature Extraction𝑿𝒊∈[𝟏,𝒎] = {𝒙𝒊𝟏, 𝒙𝒊𝟐, … 𝒙𝒊
𝒇}
𝒙𝟏𝟏
𝒙𝟏𝒇
𝒙𝟐𝟏
𝒙𝟐𝒇
𝒙𝒎𝟏
𝒙𝒎𝒇
Classification 𝑷𝒊(𝒔|𝑿𝒊, 𝜽)
𝒑𝟏𝟏(𝒔𝟏|𝑿𝟏)
𝒑𝟏𝟏(𝒔𝟐|𝑿𝟏)
𝒑𝟏𝟏(𝒔𝒄|𝑿𝟏)
𝒔𝟏
𝒔𝒎
Class
m Segments C Classes
Body Motion M
odeling
X(t) X(t+1)X(t-‐1)
S(t) S(t+1)S(t-‐1)
Y(t) Y(t+1)Y(t-‐1)
Linear Dynamical Transition
Nonlinear Observation
HumanBody
InertialSensorsMotion
Stimulus𝑿𝒊
Sensor Data𝐘 𝒕
Our Approach 6
¨ Basic Approach: Piecewise Linear Dynamical Modeling
S1
S2
Sk
… … … … …
Raw Data𝐘 𝒕 = (𝒚𝟏, 𝒚𝟐 …𝒚𝒌)
…
𝒚𝟏(𝒕) ∈ 𝑹𝒑
…
𝒚𝟐(𝒕) ∈ 𝑹𝒑
…
𝒚𝒌(𝒕) ∈ 𝑹𝒑
t Samples
Preprocessing𝒀′(𝒕) = (𝒚′𝟏, 𝒚′𝟐 …𝒚′𝒌)
t Samples
…
𝒚′𝟏(𝒕) ∈ 𝑹𝒑
…
𝒚′𝟐(𝒕) ∈ 𝑹𝒑
…
𝒚′𝒌(𝒕) ∈ 𝑹𝒑
SegmentationW= {𝒘𝟏,𝒘𝟐 …𝒘𝒎}
…
…
…𝒘𝟏 𝒘𝒎…
m Segments
Feature Extraction𝑿𝒊∈[𝟏,𝒎] = {𝒙𝒊𝟏, 𝒙𝒊𝟐, … 𝒙𝒊
𝒇}
𝒙𝟏𝟏
𝒙𝟏𝒇
𝒙𝟐𝟏
𝒙𝟐𝒇
𝒙𝒎𝟏
𝒙𝒎𝒇
Classification 𝑷𝒊(𝒔|𝑿𝒊, 𝜽)
𝒑𝟏𝟏(𝒔𝟏|𝑿𝟏)
𝒑𝟏𝟏(𝒔𝟐|𝑿𝟏)
𝒑𝟏𝟏(𝒔𝒄|𝑿𝟏)
𝒔𝟏
𝒔𝒎
Class
m Segments C Classes
S(t+1)=AS(t)+Bx(t) y(t)=CS(t)+P
sensor data (observations)
human action (motion stimulus/input)
noise
dynamics/ properties
System generally non-linear, but approximately piecewise linear (over short time segments)
1) Initialize a random sparse input 𝜲 satisfying theconstraints: 𝐱 𝒕 ≤ 𝟏 ∀𝐭
2) Repeat:(a) Given 𝜲:
Identify a LDS: 𝐀, 𝑩, 𝐂, 𝑺𝒐;(b) Given 𝐀, 𝑩, 𝐂, 𝑺𝒐:
𝒎𝒊𝒏𝒊𝒎𝒊𝒛𝒆𝑿 (𝐂𝐨𝐬𝐭 𝐅𝐮𝐧𝐜𝐭𝐢𝐨𝐧)𝐱 𝒕 ≤ 𝟏 ∀𝐭
3) Calculate histogram of X into 11 bins after recursion.
20 40 60 80 100-500
0
500
20 40 60 80 100-500
0
500
20 40 60 80 100-2000
0
2000
20 40 60 80 100-2000
0
2000
Data Piece𝒀(𝒕)𝒊 𝟎 < 𝐭 < 𝒕𝓵
PLDM Features𝑿𝒊∈[𝟏,𝒎] = {𝒙𝒊𝟏, 𝒙𝒊𝟐, … 𝒙𝒊
𝒇}
ClusteringPreprocessing
Hierarchical Temporal Selection of Data Pieces based on detected moments of motion stimulus
Motion Stimulus Detection(Peaks of Calculated Curvature)
What does this buy us? 10
¨ Coarse-grain segmentation
Start 1 minute 2 minute 3 minute 4 minute
Start 1 minute 2 minute 3 minute 4 minute
Start 1 minute 2 minute 3 minute 4 minute
Start 1 minute 2 minute 3 minute 4 minute
Start 1 minute 2 minute 3 minute 4 minute
Start 1 minute 2 minute 3 minute 4 minute
Start 1 minute 2 minute 3 minute 4 minute
Ground Truth
CorrectMounting
ErrorMounting
InsecureMounting
CorrectMounting
ErrorMounting
InsecureMounting
PLDM
ACAWalk
StraightRun Jump Side Walk Punching Body Rotation
What does this buy us? 11
¨ Fine-grained segmentation ¤ Heel-strike and toe off detection
0
50
100
150
200
250
300
350
400
450
500
1 2
Method in [5] PLDM
No.
of
pairs
of
even
ts
[5] S. Chen, C. L. Cunningham, J. Lach, and B. C. Bennett, “Extracting Spatio-Temporal Information from Inertial Body Sensor Networks for Gait Speed Estimation”, IEEE International Conference of Body Sensor Networks (BSN), pp. 71-76, 2011.
Expected range
Humans walk at about two steps per second (one per leg) so we expect about 360 event pairs per leg with some variance
Recap 12
¨ Segmentation is important for BSNs ¨ Human factors can be a problem ¨ Linear dynamical systems modeling can help for
¤ Fine grained ¤ Coarse grained
Future Work 13
¨ Reduce Computation Complexity ¨ Optimize clustering process ¨ Other Applications
¤ Surgery education data analysis ¤ Head impact identification (in sports) ¤ …