-Baljeet Aulakh -Arnold Csok -Jared Shepherd -Amandeep Singh EEC 490 Spring 2012 Kinect Fitness Trainer 1.

1

-Baljeet Aulakh -Arnold Csok

-Jared Shepherd-Amandeep Singh

EEC 490 Spring 2012

Kinect Fitness Trainer

2

Project OverviewRecognition of Body Movements and

ExercisesEverything is Voice ControlledAbility to Calculate Velocity And

Acceleration Write Statistics of Movements to an

Output File User Interface Implementation

3

Requirements

Hardware: Kinect + Laptop(2.66Hz above)

Software: Kinect SDK + Microsoft Speech Platform + C# 2010

Environment: Quite Room(voice) + Only one thing moving at a time

4

User Interface

5

Step by Step

Voice Recognition Detection Algorithms Velocity and Acceleration Ability to Write Statistics to an

Output file Documentation and Website

6

Voice Recognition

Allow User To Control Kinect By Voice

Ease of Access to the Program

As Simple as- Start && Stop

Say Which Exercise to Count

7

How Voice Recognition Work

Initialize the Audio Source from the Sensors

Initialize Speech Recognition by Speech Recognizer

Create a Speech Recognition Engine with Exercise Names

Listen to User SpeechRespond to User Speech

8

Problem With Voice RecognitionThe First Release of Language Pack

Doesn't Have a Reliable Confidence Model

Kinect Tries To Match Every Audio Source It Picks Up

Problem with Matching the Right Exercise Because of This

9

Possible Solutions For Voice Recognition

Test Confidence Interval for Best Accuracy

Use Fitness Trainer in a Quiet Environment

Introduce Noise Cancelation Use of Headset/BluetoothWait for a More Reliable Language Pack

10

Detection Algorithms

Resources Recognition algorithms List of Recognized Body Movements An Example Velocity and Acceleration

11

20 Joint Skeleton SystemProvided By Microsoft Research Kinect

12

Resources 20 Joint Skeleton System Each joint gives x, y and z valuesVector Math to find the angleTimer functionalityVoice recognition functionalityFlag Variables in programming.Counters to store the repetition.

13

Recognition algorithms

Simple cases:• Displacement of the Joints• Displacement in the X, Y, Z Direction Moderate cases:• Calculating Angles Between Joints• Setting Threshold For Some Angles Complex cases:• Set a Step by Step Routine using all of the

above to detect a motion

14

Recognized Exercises

The Following Exercises Are Recognized:

1. Squats

2. Upper Left Punch

3. Upper Right Punch

4. Right Punch

5. Left Punch

6. Right Arm Curl

7. Left Arm Curl

15

Recognized Exercises: Continued

8. Left Kick

9. Right Kick

10. Bowling

11. Hip Abduction

12. Lateral Weight Shift

13. Hamstring Stretch

Counter For All Of These Exercises

16

An example: Bowling motion

17

Bowling final position

JOINT STATISTICS

Each exercise has the average acceleration and velocity of all joints calculated.

The statistics are time stamped and saved into a text file.

Two points

Distance between them

Time to travel between them

AVERAGE VELOCITY

AVERAGE VELOCITY

Initial and final position of the joint

The distance formula

A Stopwatch

INITIAL AND FINAL POSITION

Save reference position(point) of skeleton for use in all calculations in defaultPosition[20].

getDisplayPosition(data.Joints[JointID.HipCenter]);

getDisplayPosition(data.Joints[JointID.Spine]);

The order is very important

DISTANCE FORMULAJoints lie on a Cartesian plane

However there is a caveat; The position is measured in pixels

Pixels Are Converted Into Centimeters with the conversion factor of 72 DPIx2 = (x2 * 2.54) / (72);

TEXT FILE

For every joint returned from the array generated from GetValues

foreach (JointID joint in Enum.GetValues(typeof(JointID)))

JointID Joint = (JointID)i;When int is typecasted to JointID it returns the

name of the joint at that position, NOT A STRING

24

Display / Interface

The display on the bottom of the screen shows what exercise is being done by the user

The state: Start / StopExercise counterExercise to be detected Exercise that was detected

25

Known Limitations

1. Blocked by an object

2. Overlapping joints

3. Distance from the Kinect Throws Off X, Y, Z coordinates

4. Two People In Front Of Kinect

5. Joints Move During Angle Calculation

These Limitations Make The Accuracy of the Exercise Recognition Difficult

E.g. Crossed arms...

26

Future ApproachesFix Voice Recognition problemHave a new accurate Skeleton SystemBe able to record and replay exercisesMore complex body motionsIntroduce more exercisesFind solutions to the Limitations for

example a solution to the overlapping joints and blocking objects.

27

Conclusion

Learned about Kinect programming Use with windows Use of Kinect in different areas other

than gaming Learned and getting used to C#

28

Website And Facebook Page

Link To Our Webpage:

“http://www.baljeetaulakh.com” Link To Our Facebook Page:

“https://www.facebook.com/pages/Kinect-Fitness-Trainer/236918233072748”

29

Questions?????

OR

Suggestions?

30

End of Slides

31

Fun Time!!!!!

Demonstration!!!!!