Swimming Tracker - Motion Recognition

©2015 GlobalLogic Inc.

Swimming Tracker:

Motion Recognition

Orest Hera

©2015 GlobalLogic Inc.

Introduction

Raw data processing

Swimming data analysis

01

02

03

Introduction:● Objectives● MEMS sensors● Classification problems● Sensor-based activity recognition

01

Introduction

Objectives

● Swimming and resting detection● Lap counting● Swim style recognition● Minimal usage of RAM of microcontroller

Introduction

MEMS (MicroElectroMechanical System) sensors● 3D Accelerometer● 3D Gyroscope● 3D Magnetometer

Introduction

Applications of MEMS sensors

● Mobile devices (Android, iOS)○ Display/map orientation

○ Step counter, Compass applications

○ Augmented reality

● Small custom devices○ Small vehicle navigation and stabilization (quadcopter)

○ Industrial automation

○ Innovative smart systems

Introduction

Pattern recognition tools

● Image recognition open source software:○ OpenCV library

○ Many specialized tools (face, poses, hands tracking)

● Speech recognition open source software:○ CMU Sphinx (HMM)

○ Julius (HMM 3-gram)

○ Kaldi (Deep neural network)

● Sensor-based activity recognition:○ Custom classifiers

Introduction

Sensor-based activity recognition

● Accelerometer“Activity recognition from accelerometer data” / N.Ravi, N.Dandekar, P.Mysore, M.L.Littman, IAAI’05 Proceedings, Vol. 3, 1541-1546, 2005

● Gyroscope and AccelerometerA Public Domain Dataset for Human Activity Recognition Using Smartphones / D.Anguita, A.Ghio, L.Oneto, X.Parra and J.L.Reyes-Ortiz, ESANN 2013. Bruges, Belgium 24-26 April 2013 (UCI Machine Learning Repository)

○ 30 subjects performing activities of daily living

○ 561-feature vector with time and frequency domain variables for fixed-width window

○ 10299 labeled instances

● Techniques of Natural Language Processing

Introduction

Activity recognition from accelerometer data

Classifiers:● Decision Trees● K-nearest neighbors● SVM● Naive Bayes

Accuracy:● Multiple subjects cross-validated: 92 - 99%● One subject training, same subject another data for testing: 70 - 90%● One subject training, another subject for testing: 46 - 73%

Raw data processing:● Gravity force detection by accelerometer● Rotation speed by gyroscope● Complementary filter● Compass

02

Raw data processing

Raw sensor data processing

Angle by vector of gravity force

Raw data processing

Advantages:● Direct measurement without

error accumulation

Disadvantages:● System own acceleration should

be filtered● Relaxation time due to Low-pass

filter

Raw data processing

Angle by rotation speed integration

Advantages:● Can be used during accelerated motion

Disadvantages:● Integration error accumulation

Raw data processing

Complementary filter

Advantages:● High frequency by gyroscope● Low frequency by gravity force vector

Disadvantages:● Cannot compensate error accumulation drift of rotations around vector of

gravity force

Expected

Magnetometer raw data

Raw data processing

Reality

Raw data processing

Magnetic distortions

Distortion types:● Hard iron (permanent magnet)● Soft iron (easily magnetized and demagnetized)

Types of compensating methods:● Offline (least squares methods)● Real-time adaptive (Kalman filter, neural networks,...)

Raw data processing

Magnetometer calibrationEllipsoid equation:

The least-squares problem Pseudo-inverse matrix

Swimming data analysis:● Device orientation● Chains of motion subactions● Probabilistic classification

04

Swimming data analysis

X-,Z-axes horizontal direction (yaw)

Swimming data analysis

X-,Y-,Z-axes vertical direction (pitch)

Raw data processing

Chains of motion subactions

Dictionary of motion subactions: [A, B, C, D,...]● Typical for swimming: A, B,...● Typical for resting: C, D,...

Temporal chains of moves:● Swimming: AABAAAABABAAABABBAA● Swimming: ACACCACAACCACACCACC● Resting: ADADDADAADDADADDADD● Resting: ACACCCCACCCACCCCACC

Probabilistic classification

Swimming data analysis

● Input data points

Probabilistic classification

Swimming data analysis

● Input data points● Expectation-maximization

Probabilistic classification

Swimming data analysis

● Input data points● Expectation-maximization

Probabilistic classification

Swimming data analysis

● Input data points● Expectation-maximization● Probability distribution

Swimming data analysis

Results

©2015 GlobalLogic Inc.

Thank You