Stay Kinected: A Home Monitoring System Combining Safety and Comfort Abstract The purpose of this project is to use the Microsoft Kinect sensor to implement a home monitoring system. The target audience for this system are the elderly or those with physical disabilities who may be candidates for assisted living. The Stay Kinected system combines safety and the comfort of remaining at home. Purpose The purpose of this project was to develop a home monitoring system that used the Microsoft Kinect Sensor to track a user’s body and recognize different motions/positions that would suggest they are hurt or in danger. A part of developing the system was creating a phone application that would alert a caretaker or the local authorities when an accident occurred so that the user could receive immediate attention. The phone application would also allow a caretaker to view a live video feed from the Kinect senor so that they could check in on the user to ensure the user was safe during the day. The system was created using the traditional coding steps which include design, setting up the development environment, coding, debugging, and testing. After the functions of the program were established and the development environment and other necessary tools were acquired, the coding process was initiated. For testing purposes, many different situations were simulated to ensure the program could recognize all of the possible situations that could occur. By the end of testing, the Kinect Sensor was implemented successfully in a functional home monitoring system that promotes the monitoring of a safe living environment. Objectives This project was created with future marketing and real-world application in mind. The Graphical User Interface (GUI) was made in such a way that would allow for an easy in-home setup. Part of the GUI includes installation steps for mounting the Kinect Sensor. The goals of this project were to: Implement the Kinect Sensor in a home monitoring system Program the Kinect to recognize certain events (e.g., detecting if the user has fallen, if the user leaves the house, and if there is an intruder) Have the program notify a predefined caregiver or contact when one of the events occurs Create a simple Graphical User Interface for home installation in which the user decides which of the above features they would like the system to detect Kinect Sensor Installed Event Occurs Sensor Detects Event Program Sends Text Alert Assistance Arrives Quickly When the program is started, the user is greeted with set up instructions for the in-home system. Set up for the system is quick and easy and only requires that the Kinect is mounted and the angle of elevation is adjusted for a clear view of the room. • First the software program is started and set up • The program continuously runs and monitors the user’s motions • If the program detects a certain event (e.g., the user has fallen, left the house, or there is another person in the house) a text message is sent to a caretaker’s phone • A message can also be sent to a local hospital so that an ambulance can be dispatched for assistance Development Design Setting Up the Visual Studio Development Environment Coding/Debugging Testing C# The process used for creating Stay Kinected followed the normal steps of software development. These steps (diagramed above) were followed in order. The bulk of time used to create the software program was spent in the Coding/Debugging phase. The essential tools used to create the program were the Windows Kinect and Kinect programming SDK, Visual Studio Premium 2012 Development Environment, and the Twilio SMS Text Messaging Service. Design • Included in the design was a general function that I wanted the program to perform, which was customized to specific behaviors • The general goal was to use the Windows Kinect Sensor in a home monitoring system • After more thought and contemplation, I decided on more specific functions for the program. These specific functions included: • Implementing the Kinect skeletal tracking in fall detection • Monitoring if more than one person was in the home (real- world application being if there was an intruder in the home) • Detecting if a user walks out the door (for those who have Alzheimers) Setting Up Development Environment • I used the C# language in Microsoft’s Visual Studio Premium 2012 Development Environment • The language and development environment were chosen because their capabilities best fit what was needed for creating the software • One setback of using Visual Studio 2012 was that I had no prior experience using this development environment. Because of this, a large portion of time was spent teaching myself how to use Visual Studio 2012 Coding/Debugging • Most of the time was used for programming and debugging • Some of the features of the Kinect that were pre-programmed • My program implements some of these capabilities, such as Skeletal Tracking, which were accessible though reference to other coding projects Testing • Trials of 50 scenarios were ran in order to determine the success rate of each function of the program (50 trials for each function = 150 total) Program Methods The method used to track if the user has fallen down. To do this, the program compares the position of the Shoulder Center joint and the Hip Center joint. If these joints have the same Y value, the program recognizes the position and returns true. This Boolean method is used to determine if a text message should be sent. The method used to detect if the user has left the house. In this method, the program compares the pre-defined location of the door (measured during set up) and the user’s position to recognize if they have left the house. The method is used to detect if there is another person in the house. In this method, the program takes a count of the number of tracked people in the home and if the number of people is larger than some predefined number (2 is used here), the program will indicate that an alert should be sent out.