PARKINSON’S DISEASE PROGRESSION ASSESSMENT USING ...€¦ · through clinical assessments during regular visits of patients to health institutions. These assessments are not an

PARKINSON’S DISEASEPROGRESSION ASSESSMENT

USING BEHAVIOURALINFERENCES AND SMARTPHONES

(PANDAS)

A REPORT SUBMITTED TO THE UNIVERSITY OF MANCHESTER

FOR THE CONTINUATION TO THE SECOND YEAR OF THE DOCTOR OF

PHILOSOPHY DEGREE PROGRAM

IN THE FACULTY OF ENGINEERING AND PHYSICAL SCIENCES

2015

ByJulio Vega

School of Computer Science

Contents

Abstract 6

1 Introduction 71.1 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.2 Aims and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.3 Original Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.4 Report Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2 Background 152.1 PD features monitored using wearable devices . . . . . . . . . . . . . 16

2.2 Correlation between PD features and clinical scores . . . . . . . . . . 17

2.3 PD monitoring, choosing between single-purpose devices or smartphones 20

2.4 Smartphones for PD monitoring . . . . . . . . . . . . . . . . . . . . 22

2.5 Identified weaknesses . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3 Pilot study 353.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.2 Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.3 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.2 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

3.2.1 Risks and mitigation plans . . . . . . . . . . . . . . . . . . . 46

4 Future work 494.1 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Bibliography 54

2

A Appendices 66A.1 List of Parkinson’s Disease symptoms according to [38] . . . . . . . . 66A.2 Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70A.3 Project plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71A.4 Collectable data from a smartphone . . . . . . . . . . . . . . . . . . 72

Glossary 74

3

List of Tables

2.1 Comparison of PD monitoring projects that investigate the correlationbetween their output metrics and PD clinical scores . . . . . . . . . . 19

2.2 Comparison of projects that use a smartphone to assess PD features . 242.3 Comparison of related works classified by PD feature and monitoring

device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.1 Comparison of mobile applications that collect sensor and interactiondata within a smartphone . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2 Comparison of smartphones for PD monitoring . . . . . . . . . . . . 39

4

List of Figures

3.1 Data workflow of the proposed methodology . . . . . . . . . . . . . . 43

A.1 Project Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5

Abstract

Parkinson’s Disease (PD) is a neurodegenerative disorder that affects motor and non-motor functionality of patients. Traditionally, the progression of PD is measuredthrough clinical assessments during regular visits of patients to health institutions.These assessments are not an accurate representation of the current state of the diseasebecause the evaluation sessions are short, subjective, and its symptoms vary through-out the day. To try to solve this problem, works using electronic devices have beendeveloped to assess PD. Nevertheless, most of them require patients to perform isol-ated, disruptive and intrusive evaluation routines. In contrast, our approach aims to belongitudinal, non-disruptive, non-intrusive, multi-source, naturalistic and macro-scaleand allows for a continuous monitoring of PD progression. Our solution employs thesensors contained within smartphones to collect personal, social, environmental andinteraction data about patients. Combining this data with other sources like Geograph-ical Information Systems, we can make behavioural inferences about people’s physicaland social behaviour. In this way, we want to detect proxies between human behaviourand the severity of one or more symptoms of the disease.

6

Chapter 1

Introduction

Idiopathic Parkinson’s Disease (PD) is a clinical neurodegenerative disorder charac-terised by cell loss in the substantia nigra in the brain [55]. Around 127,000 peoplein the UK1 and 5.2 million around the world 2 live with this disease, which decreasespeople’s quality of life [14] and productivity [80], especially in the elderly population.It has a mean onset age of 55.3 years old [32] and a higher prevalence as age increases[17]. Thus, the number of diagnosed people is going to rise as the population agesduring the coming years [44].

There is not a cure for the disease [47, 96]. In consequence, PD symptoms can only becontrolled using drug therapy, psychological support, physiotherapy or surgery [3, 75].However, the problem is that patients do not always respond to these treatments [38]or they do it irregularly [31].

PD has motor and non-motor symptoms [38]. Even though the consequences of non-motor symptoms are not as physically evident as the motor ones, the former are oftenjust as disabling as the latter [38, 98]. According to Jankovic [38], there are four car-dinal motor features: rest tremor, bradykinesia, rigidity in limb movement and posturalinstability. Besides them, secondary motor symptoms include voice, speech and gait al-terations, neuro-ophthalmological abnormalities, slow execution of Activities of DailyLiving (ADL), among others. Non-motor symptoms include cognitive impairment,pain, loss of the sense of smell, sphincter dysfunction, excessive sweating, weight

1http://www.parkinsons.org.uk/content/about-parkinsons2http://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_

part3.pdf

7

http://www.parkinsons.org.uk/content/about-parkinsons

http://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_part3.pdf

http://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_part3.pdf

8 CHAPTER 1. INTRODUCTION

loss, sleep disorders and behavioural and psychiatric problems such as depression andapathy. For a complete list of PD symptoms, please refer to Appendix A.1.

The progression of PD is non-linear [39] and slow (between 0% and 5.2% per year[85]). Traditionally, it is measured through clinical assessments during regular visits ofpatients to health institutions. In these evaluation sessions, trained clinicians use scalesto quantify the severity of the disease [74]. Among the most widely used scales are theHoehn and Yahr scale [32], the Unified Parkinson’s Disease Rating Scale (UPDRS)and the Movement Disorder Society Unified Parkinson’s Disease Rating Scale (MDS-UPDRS).

Hoehn and Yahr This scale is commonly used to compare groups and do gross as-sessments of the disease [38]. The scale is divided into five stages numbered Ito V, that quantify the disease severity from minimal to ‘bed confinement’. Thefirst three sections classify a patient as minimally disabled while the later twoas severely disabled. This scale has a low resolution because three years are theminimum necessary time to detect a change.

UPDRS (v3.0) The Unified Parkinson’s Disease Rating Scale is the most establishedand used scale [38, 74, 20]. It has four parts each with a numerical score. Whensummed, these subscores produce a total score that quantifies the general state ofthe disease. Part I focuses on mentation, behaviour and mood; Part II measuresthe impact of PD symptoms in Activities of Daily Living; Part III assesses motorfeatures of the disease and Part IV measures complications of medication.

This scale is “accepted by the US and [the] European Union as a reliance on newdrug approvals, studies on placebo response and trials of surgical interventions”[22]. Hence, its extended use in research projects outside clinical environments[59].

Although the UPDRS is reliable, consistent and valid [74], it was modified tocorrect the issues found in a revision in 2003 [59]. As a result, the MDS-UPDRSwas created.

MDS-UPDRS After the revision of the UPDRS v3.0, the Movement Disorder Societycreated a new version of such scale where they corrected its weaknesses andinconsistencies [26]. This new version is called Movement Disorder SocietyUnified Parkinson’s Disease Rating Scale and has four sections (I-IV):

9

• Part I. Nonmotor experiences of Daily Living.

• Part II. Motor experiences of Daily Living.

• Part III. Motor examination.

• Part IV. Motor complications [of medication-induced conditions].

Each section has a set of instructions and questions to assess physiological, sociolo-gical and psychological items based on a five-step scale (normal, slight, mild, moder-ate, severe). The elements in Part I and Part II are self-administrated while the last twoneed to be conducted by a health professional.

There are several improvements of the MDS-UPDRS over the UPDRS. First, the res-olution of ambiguities in instructions’ redaction. Secondly, a reorganisation and con-solidation of non-motor elements. In third place, a unification of the grading scalewith items using only a five-step scale. Fourthly, an augment from 42 to 50 questions.Next, a more accurate assessment of small changes in early disease, and finally, thewriting of clearer instructions for patients and clinicians to evaluate items. A clinimet-ric evaluation of the MDS-UPDRS is done in [28] applying both scales to 877 Englishspeaking patients. As a result, the MDS-UPDRS is found to be a valid scale with highinternal consistency and correlation with the UPDRS.

However, traditional PD assessment using clinical scales is not suitable for long-term,recurrent monitoring. This issue is caused by several reasons. Firstly, self-assesseditems in clinical scales are subject to recall bias because patients tend to under- oroverestimate the severity of the disease [30, 67, 88]. Other items are affected by cog-nitive bias [12] because they ask patients to perform difficult physical tasks or to an-swer embarrassing questions [28]. What is more, all items assessed by a clinician aresubjective because they depend on his/her expertise [71, 96]. Likewise, short sessions(E.g., the UPDRS takes approximately 30 minutes) do not provide an accurate pictureof PD as its symptoms vary throughout the day due to its natural development [69]or its medication [31]. However short, it is infeasible for patients to go to a hospitalevery day to be evaluated. Thus, the assessment of the progression of PD is divided insporadic chunks (E.g., every three months). In the end, it is hard to tailor treatmentsand medications to the real condition of each patient. [92, 97].

To tackle the problem above, researchers developed different solutions to follow PDprogression. For example, patients keep paper [65] and electronic diaries [70] to write


changes in their symptoms and any other related incidences. Nevertheless, these an-notations are subjective and prone to the same bias as self-reported items in clinicalscales. Furthermore, patients can have difficulties sticking to this time-consumingroutine [31]. Alternatively, with the popularisation of consumer electronics, technology-supported projects can assess PD objectively and automatically.

Technology-supported PD monitoring approaches use electronic devices to assess PDfeatures. They can be classified into three categories: ambient, video and wearable.

Ambient-based techniques monitor patients’ movements and routines using pressureor light sensors [60] installed in patients’ houses or laboratory facilities. Even thoughthe monitoring process does not interfere with patients’ daily lives, the devices can-not measure patients’ relative movements but only their interactions with their sur-rounding objects. More importantly, these approaches are unable to follow patientsoutside the monitored area [33]. Therefore, the disease cannot be assessed in thosemoments.

Video-based approaches [90] use image processing techniques to extract motor fea-tures related to PD symptoms. Although these developments are unobtrusive for pa-tients and do not rely on batteries, they have drawbacks. They can be expensive,privacy-invasive (every movement is recorded) [100] and computationally demand-ing. Additionally, they may have problems processing several people at the sametime and environmental characteristics such as lighting conditions and physical lay-out [35, 62].

Wearable-based approaches measure PD symptoms using inertial, environmental andother types of sensors embedded in single- or multi-purpose devices. Systems employ-ing these artefacts and machine learning algorithms can monitor PD for a longer timethan traditional techniques. Furthermore, in comparison with video- or ambient-basedapproaches, wearable-based ones can measure fine features of movements and can doit in places outside patients’ home or lab settings (naturalistic monitoring). Hence, theyare suitable to assess the daily fluctuation of PD symptoms [31, 67].

As it can be seen, wearable-based techniques offer advantages over traditional andother technology-based approaches. Nevertheless, they have other limitations accord-ing to the device they use. The rest of this section summarises these weaknesses so wecan later analyse how to address them in our proposed solution. An extensive literaturereview on this topic is in Chapter 2.

1.1. RESEARCH QUESTIONS 11

Wearable-based works use inertial and personal sensors, custom made devices or smart-phones to monitor PD. The type of device, its location and the chosen monitoringmethodology yield different results each with pros and cons. There are approachesthat lock devices in uncomfortable (intrusive) body areas, following scripted (disrupt-ive) assessment routines that make infeasible a longitudinal and naturalistic monitoringof the disease. Likewise, most works only monitor motor symptoms of PD, focusingon fine features of patients’ movements collected from one, two or three data sources.However, we believe this can be different. It is hypothesised that the effects of motorand non-motor symptoms on patients’ daily behaviour can be measured using multipledata origins. These disturbances can be analysed on a macro (larger) time scale ratherthan on a micro scale like fine movement data. What is more, we believe smartphonescan be used to monitor the global impact of PD following a methodology that tacklethe former weaknesses.

Considering the previous information and wanting to overcome the unsuitability oftraditional approaches to assessing PD longitudinally, this research project investigatesa PD monitoring methodology that uses smartphones. Such methodology aims to solveproblems identified in previous technology-supported monitoring works.

1.1 Research Questions

These are the initial questions that are going to be answered in this project:

Can human behaviour be linked to the severity of PD? So far, most of wearable-basedmonitoring projects evaluate the severity of the disease scoring the physical manifest-ations of motor symptoms. This approach is chosen as it is easy to extract features thatare correlated with PD.

However, evidence suggests that features related not only to physical movement butpatients behaviour are as well linked to the severity of the disease. This is why weare interested in exploring what aspects of human behaviour can be inferred from datacollected using an electronic device and whether or not such elements are related toPD. Human behaviour implies social interactions, mobility, daily activities, etc. Wepose that it is possible to identify these behavioural proxies and that we can measurethem using a methodology that has advantages over other works.

Can PD patients’ behaviour be inferred from data collected with a smartphone? There


is evidence that aspects of human behaviour such as mobility patterns in a city, trans-portation usage patterns and daily activities like walking, running or sitting, can beidentified and quantified using smartphones. However, only mobility has been ex-plored in the context of PD. We hypothesise that more aspects of human behaviourcan be inferred from data collected using a smartphone. Likewise, we believe that thecomplexity of these inferences can be reduced using a longitudinal approach and mul-tiple smartphone sensor data sources as well as external origins such as environmental,geographic or social interaction data.

Is it possible to monitor PD through such behavioural proxies? We believe it is pos-sible to monitor the progression of PD measuring behavioural proxies trends. So far,other works have found a correlation between motor symptoms metrics and the pro-gression of the disease. Thus, behavioural proxies could have a similar relation assuggested in [46]. To avoid biases and make this approach feasible, the monitoringmethodology should comply with certain attributes discussed in the next section.

1.2 Aims and Objectives

This project aims to monitor PD progression using proxies derived from heterogen-eous data and human behaviour inferences. It defines a proxy as a pointer from humanbehaviour to one or more PD symptoms. To quantify patients’ behaviour, each personwill have a ‘Profile of Living’ (PL) which will be inferred from the changes over timein patients’ physical and social routines. In turn, these inferences will be based on datacollected using smartphones and other sources. After this, the proxies will be evaluatedhaving as ground truth symptoms’ clinical scores from periodic medical assessments.The PL is a proposed metric that represents a time sequence a trait of human beha-viour. It is composed of a baseline and a set of branches that represent deviations fromit.

Due to the size and complexity of this solution, we will focus on a ‘thin slice’ thusidentifying at least one proxy and searching for more if there is enough time.

Our solution employs the sensors contained within a smartphone to collect personal,social, environmental and interaction data about patients. Combining this data withother sources like Geographical Information Systems, we will then make intelligent

1.2. AIMS AND OBJECTIVES 13

inferences about people’s physical and social behaviour. Our proposed approach al-lows for a continuous monitoring of PD progression and intends to be longitudinal,non-disruptive, non-intrusive, multi-source, naturalistic and macro-scale. Next, wedefine each of these attributes in the context of this work.

Longitudinal PD will be monitored for at least one month and ideally for a year. Thisis necessary to identify trends in patients’ behaviour [10, 52, 76] having researchthat supports the idea that changes in PD severity are slow and gradual [39, 86].

Non-disruptive The monitoring process will not impose any evaluation routines topatients. Even more, it will rely on passive sensing, eliminating the physical andcognitive burden related to such tasks and trading it off for a higher complexityof the yet-to-be-developed behaviour inference software.

Non-intrusive The monitoring technology will be comfortable for patients. For thisreason, this project will take advantage of the ubiquitous nature of smartphonesand use them to collect interaction data.

Multi-source This approach will incorporate a variety of data sources. It is hypo-thesised that by using different data origins (E.g., inertial, environmental, geo-graphical or social data), more robust and complex behaviour inferences can bemade.

Naturalistic Patients will be assessed while performing their regular routines in thereal world because there is evidence that most of the laboratory-based experi-ments have a lower generalisation performance when later verified in the wild.

Macro-scale PD progression will be monitored focusing on behaviour changes andnot in fine fluctuations of motor features. This approach would derive proxiesnot only from motor but also from non-motor symptoms such as depression,apathy, sleep disorders or sensory disorders, capturing a broader image of diseaseseverity.

The objectives of the project are four. First, to analyse the available smartphone cap-abilities to collect sensor and interaction data. Secondly, to deploy a pilot study toanalyse what is the best monitoring methodology according to the project aims. Next,to explore inference techniques of patients’ behaviour based on the collected data. Fi-nally, explore different behavioural proxies that can be inferred and their relation withPD severity.


1.3 Original Contribution

During the development of this work five main technical contributions are envisioned:

1. The proof of concept that PD progression can be assessed non-intrusively andnon-disruptively based on data collected using a smartphone.

2. The identification of one proxy (at least) between a particular facet of humanbehaviour and one or more PD symptoms. This aspect can be directly relatedto the disease (the manifestation of a comorbidity like depression) or indirectlyrelated (patient’s mobility and social interaction).

3. The development or modification of algorithms for behaviour inferencing.

4. The development or modification of algorithms for multi-source time series dataanalysis (e.g. dimensionality reduction, feature ranking).

5. The definition of a metric called ‘Profile of Living’ that models human behaviourand its deviations over time.

1.4 Report Structure

The rest of this report is organised as follows. Chapter 2 contains a literature review ofwearable-based PD monitoring and its identified weaknesses. Chapter 3 presents thepilot study used to analyse features and problems of the proposed PD monitoring meth-odology, and Chapter 4 discusses potential future work and the project planning.

Chapter 2

Background

In this section, several PD monitoring technology-supported wearable approaches arereviewed. A set of papers published after the year 2000 were extracted betweenSeptember 2014 and June 2015 from PubMed and Google Scholar using the keywords:parkinson’s disease AND [monitoring OR diagnosis OR wearable OR smartphone].From the papers belonging to the same project, we analyse only those that summarisetheir findings or represent a relevant update to their methodology or results.

To compare these works a double-entry table (Table 2.3) was created based on threesources. The first is the PD symptom list made by Jankovic [38], the second is the listof assessed features in the MDS-UPDRS [28] and the third contains any PD relatedcharacteristic assessed in the reviewed projects. Jankovic symptoms are marked witha JA superscript and MDS-UPDRS symptoms with M1, M2, M3, M4 superscriptsaccordingly to the subsection they belong to (I-IV). Any other feature that is measuredby a related project but is not on the previous two lists is added without a superscript.Columns in turn are extracted from the devices used to assess the features in the table’srows.

From Table 2.3, several questions are risen for this analysis:

• Which are the most measured PD features using wearable devices?

• Which PD features assessed with wearable devices show a correlation with PDclinical scale scores?

• What type of device is best suitable for this work (trade-offs and implications)?

15

16 CHAPTER 2. BACKGROUND

• Which projects monitor PD features using a smartphone?

2.1 PD features monitored using wearable devices

The devices employed in these works can be classified into two groups. First, single-purpose devices including inertial, environmental and health sensors. These sensorscan be encapsulated either individually or grouped within devices created specificallyfor one task. Secondly, there are multi-purpose devices. Smartphones are consideredmulti-purpose devices because, even though they contain sensors mentioned in theprevious category, they are off-the-shelf, mass-produced artefacts with extra capabilit-ies.

The most assessed features include tremor (nine projects), bradykinesia (ten projects),Activities of Daily Living (ADL) focusing on walking and standing (ten projects),and gait disturbances and characteristics (16 projects). Accelerometry is the mostused technique to assess these items being complemented in some works with gyro-scope data, magnetometer data, GPS data, or audio or video recordings (E.g., [41, 42,81]).

Tremor and bradykinesia are cardinal symptoms of PD. Both of them along with gaitare probably chosen due to their direct effect on patients’ motor activity. This circum-stance makes easier to assess them using inertial sensors. However, although there areprojects that measure the same symptoms, they do it using different devices attachedto various parts of the human body, and therefore, recording data regarding differenttypes of movement.

When it comes to ADL, walking and standing have most of the attention. Neverthelesssitting, shuffling, speech, lying down, hobbies, activity level and lifespace are assessedas well. Devices are attached to the limbs and waist [92]; shanks, thighs and trunk[93]; waist [18, 81]; shanks and trunk [82]; wrists and ankles [87]; wrist [25, 63]; arm[11] or in a free-to-choose location [46].

In the case of tremor, we can list four different types: rest, postural, intention andkinetic. However, most works focus on rest tremor of hands as it is the most commonand easily recognisable symptom of PD [38]. Researchers measure it using singlepurpose accelerometers in the wrists [7, 27, 67, 97] or smartphones attached to thehands [15, 45, 61].

2.2. CORRELATION BETWEEN PD FEATURES AND CLINICAL SCORES 17

The severity of bradykinesia is assessed from finger, hand, arm, wrist, toe, heel andleg movement. Specifically, in [66, 67] eight wearable devices in arms and legs re-cord the movement of hands and heels. In [72], finger tapping is assessed using thescreen of a smartphone, hand movement is evaluated using its frontal camera, andwrist movement is determined using the accelerometer included in the same device.Similarly, the authors of [92, 97] measure the severity of body bradykinesia using fiveaccelerometers attached to the limbs while in [40] the same is assessed analysing handmovement.

Finally, gait abnormalities are measured using devices in wrists and trunk [97]; feet,ankles, thighs, arms and back [51]; hip [69]; shank [56, 57]; waist [81]; trunk [99];forearms and calves [64]; sport shoes [8] and ankles [40].

Other sporadic efforts include sleep duration [11, 29], the impact on daily life ofON/OFF fluctuations [79], and patient’s reaction movement [27].

Almost all projects evaluate motor symptoms. However, there are two exceptions:sleep duration monitoring [11, 29] and lifespace monitoring [46]. Sleep duration isconsidered a non-motor symptom and is assessed using inertial, body temperature, skinconductivity and body heat flux data. Meanwhile, lifespace is seen as a behaviouralfeature and is estimated using GPS data. From Table 2.3, it is possible to see that mostof the non-motor symptoms have not been evaluated using wearable devices. Thesesymptoms include the rest of sleep abnormalities, autonomic dysfunction conditions,cognitive and neurobehavioral abnormalities and sensory problems. This situation isan opportunity for improvement for novel approaches.

Metrics extracted from wearable data that characterise PD features (motor or non-motor) need to be validated to ensure that they are related to PD severity. To know whatmeasured features have shown a correlation, all the collected papers that investigatethis aspect are analysed in the next section.

2.2 Correlation between PD features and clinical scores

In order to know whether or not a monitored feature is a reflection of PD severity, acorrelation can be computed with a clinical score coming from a valid third party entity(E.g., clinicians or clinical software/hardware). From the 34 works listed in Table 2.3,


13 were selected because they analyse the correlation between their output metrics anda clinical score. These works can be consulted in Table 2.1 in page 19.

From Table 2.1, it can be seen that UPDRS is the chosen scale in almost all cases,complemented by the Hoehn & Yahr scale in [61] and substituted by a custom scalein [15]. Several UPDRS scores and sub-scores were analysed: the scale’s total score[21, 25, 63, 67, 94], some or all of its sub-scores from sections I-IV and single itemscores that represent a measured feature (E.g., items 20.c and 20.d in [41] or items 20and 29 in [61]).

The authors analyse the correlation of the following PD features: dysphonia, hand andpostural tremor, bradykinesia, dyskinesia, gait (impairment), ADL (activity level) andlifespace.

Dysphonia shows a significant correlation in [94] and later these results are replic-ated and improved in [21]. Both use a dataset extracted under controlled conditionsfollowing scripted evaluation routines.

Four tremor-monitoring works have significant and strong correlations except for [67]that shows only a reliable estimation. This last result is due to the small number ofparticipants in that study (12) and therefore, its methodology cannot be compared tothe others. These papers, despite extracting different features related to rest and pos-tural tremor (frequency- or time-based), use devices in the wrist area (three employingsmartphones and one a wrist band).

Gait shows a significant [61] and strong correlation [99] when monitored in two differ-ent locations, ankle and lower back, using a smartphone and an accelerometer respect-ively. Both are compared against the UPDRS, the first one using a single item (29) andthe second one a subset of gait related features.

Activity level has two contradictory results, a strong correlation in [25] and only as-sociation in [63]. Both monitor participants use a wrist device, yet, the first projectassesses routines under controlled conditions while the second assesses patients 24x7every four months for three years. Both use proprietary algorithms to precess their data,but it is probable that the more controlled nature approach of the first work boosts itsstatistical significance just as in [27].

Bradykinesia has mixed results ranging from a significant correlation [72], a strongcorrelation [53] and a reliable estimation association [67]. Tapping performance is

2.2. CORRELATION BETWEEN PD FEATURES AND CLINICAL SCORES 19

evaluated in [53] with a touch screen. Hand and wrist movement and finger tappingare assessed using a smartphone in [72]. Finally, in [67], hand and arm movement,and heel tapping are evaluated with eight accelerometers. The best results are obtainedusing a smartphone/touch screen during controlled-condition tasks.

Dyskinesia shows mild results in [67] because, as mentioned before, the number ofparticipants is small and so the correlation is not analysed. Hand and arm movements,and finger tapping are measured using eight accelerometers.

Lifespace is a metric that represents the most frequented geographic areas of a person.The authors of [46] do not find a significant correlation between lifespace and scoresof sections I and II of UPDRS. Nevertheless, they encounter a suggested predictiverelation.

Features like tremor, bradykinesia, dysphonia and gait have a strong correlation withclinical scores. In most previous works, fine motor body movements are assessed usinginertial sensors within single-purpose devices or smartphones under laboratory condi-tions. Although their results are positive, this may not be the case if their proceduresare tested in the real world [29]. However, thanks to some works [25, 46, 63], it isknown that in-the-wild, macro-scale deployments show a mild association with PDseverity. This evidence supports the idea that it is possible for new research efforts toassess the disease’s symptoms following a longitudinal and naturalistic methodologyusing behavioural macro-scale features.

Table 2.1: Comparison of PD monitoring projects that investigate thecorrelation between their output metrics and PD clinical scores

Paper Measured PDfeature

Using asmartphone?

Clinical score Results

[94] Dysphonia No UPDRS (total andmotor)

Significantcorrelation

[21] Dysphonia No UPDRS (total andmotor)


[27] Rest tremor No UPDRS (motor) Significantcorrelation

[41] Postural tremor Yes UPDRS motor (itemsIII.20.b and III.20.c)


Continued on next page


Table 2.1* – continued from previous page

[15] Rest tremor Yes laboratory accelero-meter and custom 5-point clinical scale

Strongcorrelation

[61] Rest tremorand gait

Yes Hoehn & Yahrscale score, UPDRStremor (III item 20)and UPDRS gait (IIIitem 29)

Strongcorrelation

[99] Gait No UPDRS subset(items 13, 14, 15, 29and 30)


[25] Activity level(ADL)

No UPDRS scores Significantcorrelation

[63] Activity level(ADL)

No UPDRS scores Associationfound

[67] Bradykinesia,rest tremor and

dyskinesia

No UPDRS scores basedon videotapes

Reliableestimation

[53] Bradykinesia No UPDRS items23,24,25

Strongcorrelation

[72] Bradykinesia No UPDRS motor andbradykinesia (item23-25) scores

Significantcorrelation to

UPDRSmotor

[46] Lifespace Yes Self-reported UP-DRS (subsection Iand II)

Suggestedrelation

2.3 PD monitoring, choosing between single-purpose devicesor smartphones

As it was stated before, two classes of wearable devices emerge from the literatureanalysis: single-purpose sensors and smartphones.

Single purpose-sensors (accelerometers, magnetometer, gyroscopes, microphones, etc.)have a longer battery life depending on its configuration, and in some cases, they are

2.3. PD MONITORING, CHOOSING BETWEEN SINGLE-PURPOSE DEVICES OR SMARTPHONES21

smaller than a smartphone. It is possible to attach them to people’s limbs and trunk toget a fine measurement of body movement. Although, they can depend on external pro-cessing units or cables to transmit data. Nevertheless, most of the time this approachis not suitable for a longitudinal, non-intrusive monitoring. The reason is that devicesattached to the body in places different to the wrist [13] or waist are alien to dailyuse. A different configuration could represent a physical and physiological burden toparticipants [1, 56, 91].

Smartphones have technical advantages over single-purpose devices. They have morecomputational power, more storage space and built-in communication modules (WiFi,cellular network and Bluetooth). The presence of a touch graphical, audio and vibra-tion user interface makes possible to gather extra data from different sources [72] andto provide the user with feedback. Nowadays, people are used to carrying a smart-phone during their daily life in a bag, backpack, belt, or trousers or jacket pocket.However, these positions affect the accuracy of the necessary algorithms to extractsymptoms metrics from the collected data [5, 37, 49]. Nevertheless, these effects canbe neglected depending on the feature(s) monitored or minimised with further dataprocessing [37]. What is more, the ubiquity of smartphones makes them more suitablefor long-term, naturalistic monitoring. Thus, this allows a non-intrusive monitoringof PD, which if complemented with passive sensing methodologies, can also be non-disruptive –two desired characteristics of PD assessment when compared to traditionaland other technology-based approaches.

As of August 2015, only the techniques/artefacts listed in the first seven columns ofTable 2.3 can be found both as single-purpose devices and as a part of a smartphone.These artefacts are voice recording, video recording, accelerometer, gyroscope, mag-netometer, GPS and key/button stroke. At first sight, this could mean that all the pro-jects using these devices could substitute them with smartphones. However, this is notfeasible for two reasons. First, smartphones should be attached to participants’ bod-ies in locations that are intrusive (shanks, thighs, calves, shoes, back, etc.). Secondly,some projects should buy configurations of five or eight devices per patient, increasingthe price of the system.

Nevertheless, the limitation of smartphones to capture fine body movement (at a limbscale) in a longitudinal and naturalistic context presents a challenge. In the analysedworks, even in cases where single-purpose devices could be exchanged by smart-phones, they follow a disruptive monitoring methodology as a way to compensate for


the issue above. Participants need to perform routine activities under controlled orsemi-controlled laboratory conditions to capture specific and precise data of PD fea-tures. For example, in [72] participants did four tasks using a smartphone, three infront of its front camera (non-intrusive), and a fourth with the device attached to thehand (intrusive). Another similar case is the dataset used in [27] extracted from peri-odic recordings of people making sustained vowel phonations (”ah”). To overcomethe limitations of these disruptive methodologies, it is hypothesised that we can com-pute metrics related to PD severity applying a macro scale, multi-source, longitudinalmonitoring approach to measure motor or non-motor features.

2.4 Smartphones for PD monitoring

From the literature review, six desired attributes were identified an include in our PDmonitoring approach using smartphones: longitudinal, non-disruptive, non-intrusive,multi-source, naturalistic and macro scale.

To analyse what has been done and what could be achieved in this work, all the pro-jects using smartphones are compared against each other based on those six attributes(Table 2.2). Nine works are identified monitoring the following PD features: speech,ADL (standing, lifespace), tremor, bradykinesia, postural stability, freeze of gait, sway,gait, arising from a chair and facial expression.

The authors of [46] compute the lifespace metric complying with five out of six at-tributes. That work collects GPS data (no multi-source) over a period of eight weeks(longitudinal). It asks participants to carry the phone with them while doing their reg-ular activities for at least four to six hours per day (non-disruptive, non-intrusive andnaturalistic). The collected geographical data was segmented in 24 hours chunks andthe lifespace was calculated based on metrics such as the furthest distance travelled orthe time spent at home (macro scale). This project is the only one that computed abehavioural proxy of non-motor data and analysed its correlation with the severity ofthe disease. Even though there was no significant correlation between the metric andPD severity, there was a suggested relationship. Lifespace was calculated using onlyone data source. Thus, the results of this work could be improved by incorporatingextra data origins. The authors also provide recommendations based on their project’slimitations for future work, and since their approach is similar to ours, it is important

2.4. SMARTPHONES FOR PD MONITORING 23

to consider their suggestions to avoid similar issues. The recommendations regard thedata collection process: collect multiple data sources, ask participants to label inferreddata, and use a consistent collection methodology. They also refer to the data analysisprocess: have enough participants to be able to analyse the correlation between theirmetrics and clinical scores. Likewise, they tackle the evaluation process and highlightthe necessity of having trained staff to clinically assess patients. Finally, they advisehaving in mind the ethical and security aspects of handling location sensitive informa-tion.

There are two projects that comply with two attributes. In [81] the authors assess pos-tural stability, gait, sway and arising from a chair with a smartphone located in thewaist (non-intrusive) collecting data from the accelerometer, magnetometer and gyro-scope (multi-source). However, they do it using a scripted test routine that lasts a fewminutes (non-longitudinal, disruptive and non-naturalistic) extracting features of finemotor movements (micro scale). Similarly, in [72] the authors use a touchscreen, frontcamera and accelerometer (multi-source) to monitor three features non-intrusively.Nevertheless, the patients are required to follow laboratory routines focused on assess-ing fine motor movements of fingers, hands and arms (non-longitudinal, disruptive,non-naturalistic and micro-scale).

The rest of the works only present one or none of these characteristics. Some monitorPD following short, controlled assessment tasks focused on accelerometer data [15,69], another assess PD in short, controlled routines attaching the smartphones to thehand [42] and some others collect only one data source from a device positioned in anintrusive location [40, 45, 61].

There is plenty of room for new research efforts. So far, most projects leave out non-motor symptoms only assessing fine motor ones. Applying a macro-scale approachto PD monitoring could allow us to find proxies between patient’s behaviour (social,physical, psychological) and the severity of the disease. This attribute along with alongitudinal assessment could get a more precise image of PD progression. Becausethese proxies are complex in nature, multiple data sources can be combined to quantifythem. However, since evidence suggests that controlled laboratory experiments havea bad generalisation in the real world [29], it is necessary to monitor participants inthe wild, while they carry on with their daily life routine. To be able to do this, it isbetter to disrupt patients routines as little as possible and to assess them using non-intrusive devices, both characteristics being by themselves substantial improvements


to traditional approaches.

Table 2.2: Comparison of projects that use a smartphone to assess PDfeatures

Paper Longitudinal Non-disruptive

Non-Intrusive Multi-source Naturalistic Macro-scale

[46] Yes Yes Yes No Yes Yes

[81] No No Yes * Yes No No

[72] No No Yes * Yes No No

[69] No No Yes No No No

[42] No No No Yes No No

[15] No No Yes No No No

[40] No No No No No No



* Only true for some PD features’ assessment routines.

Tabl

e2.

3:C

ompa

riso

nof

rela

ted

wor

kscl

assi

fied

byPD

feat

ure

and

mon

itori

ngde

vice

Com

pute

rm

easu

rem

ent

tech

niqu

e/ar

tefa

ctVo

ice

reco

rdin

gsA

ccel

erom

eter

Gyr

osco

peM

agne

tom

eter

GPS

Key

/But

ton

stro

keV

ideo

reco

rdin

gsB

ody

tem

pera

ture

Skin

cond

uctiv

ityH

eatfl

uxSp

ecia

lised

arte

fact

/test

Slee

pdu

ra-

tion

[11]

,[29

][1

1][1

1][1

1]

RE

Mbe

ha-

viou

rdi

sord

erJA V

ivid

drea

ms

JA

Res

tless

legs

synd

rom

eJA

Day

time

slee

pine

ssM

1,JA

Slee

pab

-no

rmal

ities

Slee

ppr

ob-

lem

sM

1,JA

Sitti

ng[1

8],[

93],

[82]

Wal

king

[92]

,[18

],[9

3],[

82]

Dre

ssin

gM

2

Hyg

iene

M2,

JA

Con

tinue

don

next

page

25

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Eat

ing

task

sM

2,JA

,q

Han

dwri

ting

M2,

JA,a

Shuf

fling

[18]

Spee

chM

2

Saliv

aan

ddr

oolin

gM

2,JA

Trem

orM

2

Get

ting

out

ofa

bed/

-ca

r/de

epch

air

M2

Wal

king

and

bala

nce

(wal

king

aid)

M2

Act

iviti

esof

Dai

lyL

ivin

g

Turn

ing

inbe

dM

2,JA

Free

zing

M2,

b

Che

win

gan

dsw

allo

win

gM

2,JA

Lyin

gdo

wn

[18]

,[93

],[8

2]

Stan

ding

[81]

,[18

],[9

3],[

82]

[81]

[81]

Con

tinue

don

next

page

26

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Doi

ngho

b-bi

esan

dot

her

activ

ities

M2

[18]

Act

ivity

leve

l[2

5],[

63],

[11]

[11]

[11]

Lif

espa

ce[4

6]

Res

ttre

mor

M3,

c

[27]

,[97

][1

5],[

45],

[7],

[67]

,[6

1]

Post

ural

trem

orM

3,d

[41]

,[42

],[9

7][1

5][4

1],[

42]

Inte

ntio

ntr

emor

[15]

Trem

orM

2,JA

Kin

etic

trem

orM

3,e

[15]

Bra

dyki

nesi

aM

3,JA

,f[9

7][9

2],

[67]

,[66

],[7

2],[

40]

[72]

[43]

,[53

],[7

2],[

81],

[27]

g

[72]

Con

tinue

don

next

page

27

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Tim

esp

ent

with

dysk

in-

esia

M4,

h

[97]

,[67

],[1

6],[

29],

[84]

[97]

OFF

Dys

toni

aM

4,JA

,i

Func

tiona

lim

pact

ofdy

skin

esia

M4,

b

Hyp

okin

esia

[92]

Aki

nesi

a[9

7][9

2]

Post

ural

stab

ility

M3,

JA[8

1],[

93]

[81]

[81]

Rig

idity

M3,

JA

Spee

chM

3,j

[27]

k

Dys

phon

ia[9

4],[

21],

[84]

Hyp

okin

etic

diso

rder

s

Spee

chab

-no

rmal

ities

Dys

arth

ria

JA

Free

zeof

Gai

tM

3

[97]

[51]

,[6

9],[

57]

[97]

[57]

Con

tinue

don

next

page

28

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Sway

[81]

[81]

[81]

Gai

tM3,

JA,l

[81]

,[99

],[5

6],[

64],

[8],

[40]

,[6

1],[

93]

[81]

,[56

],[8

][8

1]G

ait

rela

ted

abno

rmal

it-ie

sFa

lls[9

7]

Faci

alex

pres

-si

onM

3,JA

,m,n

Com

plex

mo-

torf

unct

ion

[27]

o

Func

tiona

lim

pact

ofO

N/O

FFflu

ctua

tions

M4,

b

[97]

[29]

,[7

9],[

89]

Com

plex

ityof

mot

or(O

N/O

FF)

fluct

uatio

nsM

4,p

Tim

esp

ent

inO

FFM

4,h

Fatig

ueM

1,JA

Lef

t-ri

ght

asym

met

ry[1

6]

Con

tinue

don

next

page

29

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Post

ure

M3

Rea

ctio

nan

dm

ovem

ent

time

[27]

Gla

bella

rre

-fle

xJA

Ble

phar

ospa

smJA St

riat

alde

-fo

rmity

JA

Scol

iosi

sJA

Cam

ptoc

orm

iaJA

Oth

erm

o-to

rfea

ture

s

Neu

ro-

opht

alm

olog

ical

abno

rmal

ities

Uri

nary

prob

-le

ms

M1,

JA

Con

sitp

atio

npr

oble

ms

M1,

JA

Abn

orm

alsw

eatin

gJA

Sebo

rrho

eaJA

Con

tinue

don

next

page

30

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Wei

ghtl

oss

JA

Sexu

aldy

s-fu

nctio

nJA

Aut

onom

icdy

sfun

ctio

n

Lig

hthe

ad-

ines

son

stan

ding

M1,

JA

OC

D/c

ompu

lsiv

ebe

havi

ours

Cog

nitiv

eIm

pair

men

tM1,

JA[4

3]

Hal

luci

natio

nsan

dps

ycho

sis

M1

Dep

ress

edm

ood

M1,

JA

Anx

ious

moo

dM

1

Bra

dyph

reni

aJA

Cog

nitiv

ean

dne

uro-

beha

viou

ral

abno

rmal

it-ie

s

Tip-

of-

the-

tong

ueph

enom

enon

JA

Anh

edon

iaJA

Con

tinue

don

next

page

31

Tabl

e2.

3*–

cont

inue

dfr

ompr

evio

uspa

ge

Apa

thy

JA

Feat

ures

ofD

DS

M1,

r

Pain

and

othe

rse

nsat

ions

M1,

JA

Ano

smia

JA

Age

usia

JA

Pare

sthe

sias

JA

Sens

ory

ab-

norm

aliti

es

Aka

this

iaJA

aT

here

late

dfe

atur

em

entio

ned

byth

eau

thor

ism

icro

grap

hia.

bT

his

item

isas

sess

edin

the

cont

exto

fdai

lylif

e.c

InM

DS-

UPD

RS

Am

plitu

dean

dC

onsi

sten

cyof

hand

trem

orar

em

easu

red

intw

odi

ffer

enti

tem

s.d

InM

DS-

UPD

RS

post

ural

trem

oris

mea

sure

din

hand

s.e

InM

DS-

UPD

RS

kine

tictr

emor

ism

easu

red

inha

nds.

fIn

MD

S-U

PDR

Sbr

adyk

ines

iais

eval

uate

dun

dert

he‘G

loba

lspo

ntan

eity

ofm

ovem

ent’

item

.g

Inth

isw

ork

hand

tapp

ing

isal

soev

alua

ted.

hT

his

item

isas

sess

edas

ape

rcen

tage

ofaw

ake

time.

iT

his

item

isas

sess

edas

ape

rcen

tage

ofO

FFtim

e.j

InM

DS-

UPD

RS

this

item

asse

ssta

ckyp

hem

ia,p

alila

lia,s

lurr

ing

and

spee

chs

volu

me,

pitc

h,cl

arity

,mod

ulat

ion.

kIn

this

wor

kin

tens

ityde

cay

isas

sess

ed.

lIn

MD

S-U

PDR

Sth

isite

mas

sess

stri

deam

plitu

de,s

trid

esp

eed,

heig

htof

foot

lift,

heel

stri

ke,t

urni

ngan

dar

msw

ing.

mIn

MD

S-U

PDR

Sth

isite

mas

sess

eye-

blin

kfr

eque

ncy,

mas

ked

faci

esor

loss

offa

cial

expr

essi

on,s

pont

aneo

ussm

iling

and

part

ing

oflip

s.n

The

rela

ted

feat

ure

men

tione

dby

the

auth

oris

mas

ked

faci

es.

oIn

this

wor

ka

pegb

oard

isus

edto

asse

ssth

isite

m.

pT

his

item

refe

rsto

the

pred

icta

bilit

yof

OFF

times

byth

epa

tient

.q

In[3

8]th

ese

are

refe

rred

asfe

edin

gan

dcu

tting

food

.r

DD

S=

Dop

amin

eD

ysre

gula

tion

Synd

rom

e.

32

2.5. IDENTIFIED WEAKNESSES 33

2.5 Identified weaknesses

In this section, we summarise the identified weaknesses from the reviewed literature.They were used to set the requirements that make our approach original and significant.Such approach is tested in a pilot study described in Chapter 3.

Most of wearable-based monitoring projects focus on motor symptoms of PD

Non-motor symptoms have a similar impact on PD severity than motor ones. Nev-ertheless, there is a majority of projects that monitor the latter analysing fine bodymovements. However, just as motor features, non-motor ones affect people’s dailybehaviour. Thus, taking as evidence the monitoring of lifespace in [46], it is hypo-thesised that non-motor features can be monitored through proxies (links) betweenhuman behaviour and the severity of the disease. Therefore, instead of assessing finemotor changes on a micro-scale, it is necessary to measure behavioural changes ona macro-scale. For example, co-morbidities like depression, apathy or anxiety canmake a person stay extended periods of time at home and modify their patterns ofsocial interaction (either physically or digitally), sleep, mobility and physical activity[58]. Similarly, it is hypothesised that behavioural proxies can also be found amongmotor symptoms and collateral manifestations of the different PD features.

However, identifying behaviour patterns from the same data that is collected to monitormotor activities represents a challenge because there are only a few metrics to quantifysuch abstract entities. To try to alleviate this problem, it is believed that these para-meters can be constructed from multiple types of data. For example, mobility patternscould be inferred using GPS data to get an approximation of a person’s location. Theycould also be refined with WiFi and geographical information data to abstract patients’location as an entity like ‘house’, ‘work’ or ‘sports venue’. Similarly, it could be usefulfor them to know if the device is being carried using inertial data. This means that amulti-source approach would ease the task of inferring human behaviour. The use ofa wider variety of origins could produce better inferences of motor symptoms sincemost works only analyse one to three sources.

Most of wearable-based monitoring projects are not suitable for longitudinal PD mon-

itoring

It was concluded that most methodologies of wearable-based projects make infeasibleto monitor patients for periods of time bigger than a few days or a couple of weeks.


The reason is they use disruptive evaluation routines or intrusive monitoring devices.Thus, a longitudinal, non-disruptive and non-intrusive monitoring approach wouldenable us to get a more accurate image of the evolution of the disease.

Most of wearable-based monitoring projects are tested in a controlled environment

It was found that most works only assess patients in laboratory controlled or semi-controlled conditions. This makes harder to apply the same methodology in the wild.A naturalistic approach would benefit a longitudinal monitoring of patients.

No wearable-based projects monitor PD in a longitudinal, non-disruptive, non-intrusive,

naturalistic, multi-source and macro scale way

An approach with the attributes mentioned above does not exist. This is mostly due tothe device and methodology associated with the project. About two-thirds of the re-viewed works, use single purpose devices that are uncomfortable, alien to the generalpopulation, and technically incapable of providing multiple data sources for measur-ing more than fine human movements. Furthermore, most of the methodologies usedcannot monitor a patient for long periods of time during their regular activities withoutinterfering with their routines.

Chapter 3

Pilot study

After the background review, we identified six attributes that can improve some of theweaknesses of current PD monitoring works. To evaluate a methodology with theseattributes, we decide to conduct an exploratory Pilot Study (PS). The objectives of thepilot are:

• Analyse the technical challenges of monitoring PD patients using a smartphone.

• Analyse the technological and methodological challenges of a longitudinal, non-disruptive, non-intrusive, naturalistic, macro-scale and multi-source PD monit-oring approach.

• Analyse the feasibility of behaviour inferencing using smartphone collected data(what data is necessary to extract meaningful information).

• Use the PS as a guide for the future primary monitoring study planning.

The PS started on 30th June 2015 and will finish on 30th September 2015. It is car-ried in conjunction with a partner project from the School of Psychological Sciences(SPS) of the University of Manchester which already has an approved ethics applica-tion.

35

36 CHAPTER 3. PILOT STUDY

3.1 Method

3.1.1 Participants

Six PD patients were contacted through their local clinics during their routine evalu-ations. Three gave their signed consent to take part in the study. Participants includedone white female (P1) and two white males (P2 and P3) of 58.5, 72.4 and 70.9 yearsold respectively. All of them are diagnosed with clinically idiopathic PD.

3.1.2 Materials

The monitoring hardware and software platform consists of a phone and a set of mo-bile applications that collect and store the data coming from different sensors and in-terfaces.

As a first step, we studied what operative system the phone should run on. The threeoptions 1 were: iOS, Android and Windows Phone (WP). This decision had two con-straints. First, six phones were needed because although only three participants tookpart in the PS, the devices are going to be used in a future study by the partner projectin the SPS. Secondly, we only had a budget of £900. Therefore, iOS was discardedsince the cheapest iPhone is £319. Before choosing between Android or WP, we re-viewed if there were existing applications that would be suitable to collect patients’data. Thus saving us development time. Five data collection works were available todownload from the Internet, all of them for Android (Table 2.2). ‘AndWellness’ [30]was discarded as only records location traces from a GPS. ‘Ohmage’ [34] was rejectedbecause the 16 raw features that collects are as well gathered by ‘Funf’ and ‘AWARE’.‘EmotionSense’ libraries [73] were not used because time wasn’t enough to developa monitoring application using this project. Besides, the features sensed by Emotion-Sense are also present in AWARE. Finally, AWARE [24] was found suitable for thisproject. It collects 26 raw elements (the maximum as delimited by the Android API 19)and has six plugins to obtain external (weather and background noise) and contextualdata (activity recognition, ambient light and device usage). AWARE can store data inSQLite databases or upload it to both their own or a third-party server. What is more,it is maintained and well documented, being easy to modify it. Although Funf [2] has

1http://www.idc.com/prodserv/smartphone-os-market-share.jsp

3.1. METHOD 37

not been updated since 2013, it collects data which AWARE does not, uploading it tothe cloud service Dropbox and saving it locally as well. Thus, it was decided to useAWARE and Funf to collect participants’ data. Both applications are open source, theyanonymise most of the data and offer standalone and library applications. Nonetheless,we select the standalone applications due to the above time constraint. The featurescollected by AWARE and Funf are listed in Appendix A.4. Since we consider theseapplications are sufficient to collect patients’ data, Android was set as the preferredplatform. This idea is further supported by the wider availability of Android phonescompared to WP devices; just from January 2014 to August 2014, 18,796 differentAndroid phones models were spotted 2. Besides this, there is not data about WP phonemodels but they can be expected to be many fewer since WP has 2.7% of market sharein contrast to 78% of Android.

Table 3.1: Comparison of mobile applications that collect sensor andinteraction data within a smartphone

Project Open Source Library orApplication

Activedevelopment

Collectableraw features

Collectablecontextfeatures

Extensibility

AndWellness Yes Application No 1 1 Yes

Funf Yes Both No 37 0 Yes

EmotionSense Yes Library Yes 23 0 Yes

AWARE Yes Both Yes 26 6 Yes

Ohmage Yes Both Yes 16 2 Yes

The next step was to choose the best Android smartphone model to install AWAREand Funf. To do this, a subset of all the smartphones available in the market until June2015 was compared according to the following characteristics: battery life, includedsensors, price, storage and Android version. To simplify the search, battery life wasthe first filter. The phones with a battery life of more than eight hours 3 were taken intoaccount; only one information source was considered to keep a consistent comparison.Seventeen devices were analysed and are listed in Table 3.2. Their price, as shown inon-line stores, was converted to pounds in case it was presented in a foreign currency.The rest of features, hardware (sensors, storage) and software, were extracted from

2http://opensignal.com/reports/2014/android-fragmentation/3http://www.phonearena.com/


elsewhere4. All the hardware sensors found within the devices were listed except forthe accelerometer, GPS and proximity as they were present in all phones. We werelooking for the smartphone with the longest battery life, lowest price and most quantityof sensors.

From Table 3.2, the Blu Studio Energy (BSE) phone has the longest battery life (14h53min, 3h 27min more than the closest device) and has the cheapest price (£110).However, it does not have a gyroscope, barometer or NFC sensors. Considering thatat this point it is not possible to discard the usefulness of any sensor, we kept lookingfor other phones. From a ‘sensors quantity’ point of view, the Samsung Galaxy Note4, Xiaomi MI 4 and Xperia Compact Z3 (XPE) were the best. Nevertheless, the Note4 was discard due to its high price (£509), as well as the Xiaomi phone since it is notofficially available outside China. Despite the XPE had five out five sensors, we wereconcerned about its battery life (30% less than BSE). Thus, 1 XPE and 5 BSE phoneswere bought, spending £715, with the purpose of comparing both devices’ distinctspecifications (battery life and lower price vs. quantity of sensors and higher price). Interms of storage both can hold one memory card of at least 64GB, a size we considerenough as data was going to be backed up outside the device.

4http://gadgets.ndtv.com

Tabl

e3.

2:C

ompa

riso

nof

smar

tpho

nes

forP

Dm

onito

ring

Phon

eB

atte

rylif

ePr

ice

(£)

Mag

neto

-m

eter

Am

bien

tlig

htG

yros

cope

Bar

omet

erN

FCA

ndro

idve

rsio

nB

uilt-

inst

orag

e(G

B)

SDca

rd(G

B)

Scre

en(i

nche

s)

Blu

eSt

udio

Ene

rgy

14h

53m

in11

0Y

esY

esN

oN

oN

o4.

48

645

Hua

wei

Asc

end

Mat

e2

4G

11h

26m

in30

4Y

esN

oY

esN

oN

o4.

316

646.

1

ZT

EZ

Max

10h

53m

in17

0N

oN

oY

esN

oN

o4.

416

325.

7

Mot

orol

aD

roid

Turb

o10

h42

min

500

Yes

Yes

Yes

No

No

4.4

64N

o5.

2

ZT

EG

rand

XM

AX

+10

h20

min

191

No

Yes

Yes

No

No

4.4

1632

6

Sony

Xpe

ria

T2

Ultr

a10

h16

min

200

Yes

No

Yes

No

Yes

4.4

832

6

Sony

Xpe

ria

Z3

Com

pact

10h

2m

in32

2Y

esY

esY

esN

oY

es4.

416

128

4.6

Sony

Xpe

ria

Z3

9h29

min

469

Yes

Yes

Yes

Yes

Yes

4.4

1612

85.

2

39

Sony

Xpe

ria

M2

9h19

min

141

Yes

Yes

No

No

Yes

4.4

832

4.8

Sony

Xpe

ria

C8h

44m

in16

1Y

esN

oN

oN

oN

o4.

24

325

Mei

zuM

X4

Pro

9h20

min

354

Yes

Yes

Yes

No

Yes

4.4

64N

o5.

5

Hua

wei

Asc

end

MA

TE

7

9h3

min

335

Yes

Yes

No

No

Yes

4.4

3212

86

Sam

sung

Gal

axy

Not

e4

8h43

min

509

Yes

Yes

Yes

Yes

Yes

532

128

5.7

LG

G2

Min

i8h

32m

in13

9Y

esN

oN

oN

oN

o4.

48

No

4.7

Xia

omiM

I48h

32m

in27

9Y

esY

esY

esY

esY

es4.

464

No

5A

cerL

iqui

dJa

deS

8h32

min

200

Yes

Yes

No

No

No

4.4

1632

5

Sony

Xpe

ria

T3

8h23

min

206

No

Yes

No

No

Yes

4.4

832

5.3

40

3.1. METHOD 41

Once the phones and applications were chosen, we proceed to prepare the software forthe monitoring platform.

Funf was downloaded and configured using the web tool provided by its authors 5. Itwas set to register five features every 24 hours, Accounts (on-line services accountsinstalled on the device), Audio Media, Images, Applications and Videos. As well asAudio features (frequency- and time-based features) every two minutes for 60 seconds.It encrypts all the data and saves it on Dropbox as well as on the device. No otherchanges were made to the application.

AWARE needed extra work. As it was mentioned before, AWARE can send the senseddata to their own server, a third party server or store it directly on the device. As per theethics approval to this project, participants’ data cannot be transmitted to an externalentity without being encrypted. Therefore, keeping the data in AWARE’s server wasruled out as it could be accessed by its administrators. The option to set up our serverto store the data was also considered. However, before being redirected to our data-base, the data is transmitted to the AWARE’s server where it is manipulated in plaintext. Thus, this option was also discarded. The only alternative left was to store thedata on each device. Nevertheless, AWARE does this in a SQLite database withoutencryption. Hence, the application needed to be modified. Since AWARE is opensource, we downloaded the code and added compatibility with the SQLCipher librar-ies6. These libraries store data in encrypted SQLite databases that can be manipulatedas their non-encrypted counterparts. As a consequence of this circumstance, extrasoftware was needed to periodically backup the collected data and free up space in thedevice’s local storage. To accomplish these two tasks, the Automate application wasdownloaded from the Google Play Store and configured to compress and zip all theAWARE databases every day at 2.00 a.m. Along with this, the Synchronize Ultimateapplication was also downloaded and set up to upload the zip file created by Automateto the cloud service Google Drive each day at 3.00 a.m.

AWARE’s inertial sensors can be configured to collect data at different rates. WithinAWARE, there are four options: 200ms, 60ms, 20ms and 0ms. These numbers havea direct impact on battery life as they represent the suggested delay after which thesystem will let AWARE know that there was a change in a sensor. To decide whichwas the best delay, AWARE was installed in one BSE device and the XPE with a 0ms

5http://inabox.funf.org/create/6https://www.zetetic.net/sqlcipher/


delay. Both phones were left functioning over a desk during 12hrs, without beingmoved or used. After 12hrs, the BSE consumed 39% of the battery and the XPE 96%.This result is not suitable for the PS as the XPE doesn’t have battery left to supportdaily usage and all backing up applications running on the phone. To have comparabledata, the 200ms was chosen in all devices because participants’ routines are goingto be analysed from a macro-scale point of view without focusing on fine movementchanges. It was assumed that since the PS will run for three months, battery life perday is more important than a fast sensing rate as this extra data may not be needed inthe future data analysis.

Both the XPE and BSE were configured as described above. Besides this, the Ap-pLock, AppStart and Keep Running applications were downloaded and installed toblock, start at phone’s boot, and keep executing Funf, AWARE, Automate and Syn-chronize Ultimate. All phones received a class 10 64 GB micro SD. The XPE phonehas an extra modification. We added a routine in Automate to delete the AWARE data-bases every week. The reason is this device doesn’t allow AWARE to store its data onan external SD card filling the available 16 GB internal storage in about ten days.

Besides the phones and mobile applications, we bought a silicon case to protect thephone, a wireless charging adapter and a wireless charger base to let patients chargethe phones more comfortably. Likewise, we acquired a holster to give them the choiceto wear the phone on the waist, along with a SIM card with unlimited mobile Internetaccess as a way to ensure communication for the data backup and as an incentive touse the phone.

3.1.3 Procedures

The methodology of the PS comprises four stages: data collection, data processing,data analysis, and evaluation. Figure 3.1 represents its data workflow.

Data collection

During this stage, data was collected from all the sensors and interfaces present withina smartphone. It was complemented with external sources such as environmental data(temperature, pressure, time of sunset and sunrise, meteorological conditions) and am-bient data (noise and light levels).

3.1. METHOD 43

Figure 3.1: Data workflow of the proposed methodology

Each participant received one smartphone, the extra material mentioned before andseveral recommendations. First of all, they were instructed to use the phone as theypleased but carrying it in the belt using the provided holster, in a trousers’ pocket, or inany other position as long as it was with them at all possible times. They were told tonot fully discharge the phone’s battery and to recharge it every night. Similarly, theywere advised to not get it wet or leave it close to any direct heat source and to keep itaway from children.

All the participants were given a quick tutorial on how to charge, block and unblock thephone, and how to receive and make calls and text messages. They were also explainedhow we were going to use their data and that it was encrypted and not available tounauthorised people. Furthermore, they received a telephone number they could callto if the had any problems with the smartphone during the PS.

All participants were asked if they wished to substitute their current mobiles with theirassigned new smartphone. P1 had a traditional mobile phone and did not want to usethe new device as the main one because she “didn’t feel comfortable typing on a touchscreen”. Therefore, we assigned her one BSE with our SIM. P2 had a smartphone andwas keen to substitute it with the new one so we assigned him the XPE with our SIMcard. He decided to stop using his old phone number for the duration of the PS. P3 didnot have a smartphone but at the moment we talked, he was planning to buy one andwas keen to use it as his primary device. Thus, we assigned him one BSE with both,our SIM card and his old SIM card. To compare the inferences that can be made froman active user vs. a passive user, it was decided that the XPE was given to a participantwho would adopt it as their main device.


Data processing

In this stage, raw sensor data is manipulated to extract metrics that quantify PD mo-tor and non-motor characteristics, as well as traits of human behaviour. Then, thesefeatures are analysed and ranked according to the degree of influence with which theyrepresent the studied characteristic. After this, a subset of the most representative met-rics is chosen according to the generated rank.

One of the challenges in this stage is to analyse the many metrics that can be extractedfrom a time data series. The situation worsens after taking into account the wide varietyof sensor data that can be collected from a smartphone and other sources. This situationis why, a first approach aims to look for metrics and algorithms used in previous worksfor feature ranking, selection, and reduction in PD monitoring, human activity recogni-tion, and human behaviour inferencing topics. For example, the Davies-Bouldin (DB)clustering evaluation index [68, 77], the Bayesian Least Absolute Shrinkage and Selec-tion Operator (LASSO) [48, 78, 83], or Support Vector Machines [48, 72] for featureranking, selection, and reduction respectively.

Data analysis

The data analysis stage has two tasks: ‘Profile of Living’ generation and Proxy Iden-tification. In the former, a recurrent (daily, weekly, monthly, etc.) profile of the par-ticipants behaviour is constructed according to the metrics extracted in the previousstage. The exact type of modelled behaviour is not known yet but we plan to studysimple patterns first and then move into more complicated ones. A list of potentialalternatives is given in Chapter 4.

Once there is a periodic profile with multiple temporal instances, a portion of them isused to create a personal baseline. This procedure is necessary since different activitiesand behaviour mean different things to each person. Then we will measure deviationsfrom the baseline for the rest of the captured instances. Once again, the technique toextract and model a longitudinal behaviour profile is not known yet. Nevertheless, alist of potential alternatives is given in Chapter 4.

Finally, in the Proxy Identification task, the found deviations in the previous stage arescored and mapped to clinical changes in PD severity. I expect the strongest contribu-tions of my Ph.D. to be made at this point.

3.2. REMARKS 45

Evaluation

Lastly, to evaluate this approach, severity scores of each patient need to be regularlycollected during the monitoring time span. Trained staff will carry these clinical evalu-ations using valid severity scales such as the MDS-UPDRS. Then, we will analyse thedegree of correlation between the deviations’ scores computed in the previous stageand an interpolation of the obtained clinical score of each patient (ground truth).

This methodology will be repeated to find at least one proxy between patients’ beha-viour and PD severity.

3.2 Remarks

As August 2015, the data collection stage is still running. From what we have done wecan mention several observations that will be useful for a future monitoring study.

P2 had problems with the battery life of the XPE. As a consequence of a bug in Ulti-mate Synchronize, his phone’s battery life was reduced by 30%. The solution was todeactivate the Internet synchronisation and backup the data manually for the rest of thePS.

P3 dropped out of the study after three weeks. The participant mentioned that thebattery life was too short for his needs. The battery last approximately eight hours andwe presumed this happened due to the double SIM configuration.

There has been data loss. Around 50% of accelerometer data and 75% of phone pro-cessor data of P1 along with 5% of accelerometer data of P2 have been lost duringthe first six weeks of the study. This circumstance was caused by a read-write conflictbetween AWARE and Automate backup routine. To fix this conflict, the AWARE codewas modified to stop all sensing processes for an hour between 1.45 a.m. and 2.45a.m. As a consequence of this decision, data corruption was not an issue anymore butan hour of data is lost every day. However, it is believed that this is not going to be aproblem for the data analysis since the participants will not be using the device at thattime.


3.2.1 Risks and mitigation plans

From our experience during this PS and the reviewed literature, we also identifiedseveral risks that can affect this project along with possible mitigation measures. Thisis important to we avoid the same problems in our future monitoring study.

Battery life directly affects the amount and type of data we collect. Battery life relayson factors like platform energy management capabilities (depending on the OS andinstalled software), battery capacity (mAh), energy consumption of sensors and usagepatterns of participants. To mitigate this, the collected data needs to be analysed to es-timate what sources contribute the most to the computed behavioural proxies. Besidesthis, we can calculate the daily monitoring duration per patient that is enough to com-pute behavioural proxies and to adjust the sensing length accordingly. Likewise, themonitoring software and hardware setup can be optimised, and collecting applicationscan be substituted.

We can ignore a data source that is important for behaviour inferencing. There aremany sensors that can be installed in a smartphone (E.g., GPS, accelerometer, baro-meter, gyroscope, etc.). All of them influence the device’s energy consumption, price,and extractable data. Besides this, different models of the same sensor have a differentaccuracy and resolution. This situation was observed in the BSE and XPE. To mitigatethis, the collected data needs to be analysed to estimate what sources contribute themost to the computed behavioural proxies. In this way, we could determine what arethe best specifications of each sensor comparing the BSE vs. the XPE.

If the data collection software installed on the phone has a high memory and CPUusage, it will slow down the graphical user interface. This could led users to drop outof the study. To avoid this, it is necessary to evaluate the performance of the installedsoftware based on the PS participants’ feedback to decide if it is good enough or ifmodifications are needed.

Due to phone misuse, battery life, data transmission or faulty hardware, data could becorrupted or lost. To mitigate this risk, core monitoring software needs to be protectedagainst accidental or intentional modifications from the user. Collecting software needsto be resilient to failures and capable of recovering from crashes in the phone, notify-ing to the researchers if there is a problem with the device or with the transmissionconnection. Databases containing patients data must have a backup. Finally, hardwareinvolved in the monitoring process needs to be tested for production defects.

3.2. REMARKS 47

Participants may not be familiar with smartphones and therefore not be very keento interact with them. To evaluate this risk, it is necessary to analyse the collecteddata and potential behavioural proxies. If according to the analysis, interaction datais not required to identify behavioural proxies, we can recruit patients allowing themto wear the phone without using it. Otherwise, we should include ‘unwillingness touse the phone’ as an exclusion criterion for the recruitment process of the future mainstudy.

In the literature, there is evidence that a phone’s location on the human body affects in-ertial data used to extract motor-related metrics [54]. With traditional PD monitoringapproaches, the use of scripted methodologies may prevent this problem. However,this work aims to be non-intrusive, longitudinal and macro-scale. Thus, it is hypothes-ised that outliers and errors in the data introduced by the free position of the phonewill smooth throughout the length of the study. If this is not the case, we could toemploy algorithms that can filter data signals using supporting inferences from otherdata sources.

In previous technology-based projects, when a patient is assessed, their movements arelabelled by a person or a piece of software. These labels serve as ground truth duringthe data analysis stage of the study. Nevertheless, since this will be a longitudinalapproach, doing so is infeasible. Therefore, there is the risk of not being able to validatethe extracted behavioural inferences. That is why, we envision two solutions. The firstone requires to confirm the behavioural assumptions with each participant. This meansto interview them to find out if the obtained daily routine corresponds to what theyusually do. However, participants’ answers could be subject to cognitive or recall bias.The second one involves using data analysis techniques that work with unlabelled datato extract the behavioural proxies. Nevertheless, this process has problems handlingthe semantic meaning of the inferences. For example, it could be possible to say that aperson is moving poorly in the morning but not why –breakfast, illness, cooking– [4].The extraction of semantic meaning could be easier using data from different sourcesto make complementary inferences.

To conduct research studies with people, it is necessary to obtain the approval of theethics committees of the University and the National Health Service. This process canlast several months and requires an analysis of risks and mitigation plans. To avoid adelay, we will prepare the ethics application for our primary monitoring study using anextended risk analysis based on this section of this document, the lessons learned from


the PS and the experience of the researchers of the partner project in the SPS.

Since it is not feasible to clinically assess participants every day, the scores from theperiodic scale evaluations need to be interpolated to the same scale as the behaviouralinferences (daily, weekly, biweekly). The interpolation can affect the results if the usedmethod does not adjust to the real progression trend of the disease. It is possible totake works like [6, 95] as a guide to a solution. Additionally, it is necessary to analysethe available interpolation methods and evidence of their use with PD progressionscores.

Because human behaviour is complex and we do a naturalistic monitoring of patients,there is the risk of not finding a proxy related to PD severity. To try to avoid this, weplan to use the PS as a way to identify promising proxies, going from simple inferencesto more complicated ones. Nevertheless, if we don’t find one at the end of this project,the analysis of proxies with no correlation is still a valuable contribution if it was doneusing valid scientific methods.

Chapter 4

Future work

Before describing the main activities for the second year, we want to mention twotasks that we will do around January 2016. A systematic review on a yet-to-be-definedtopic (possible human behaviour inferencing using smartphones) and the applicationto doctoral consortiums of conferences like CHI 1 and UIST 2.

Meanwhile, during the following months we will clean, consolidate and analyse thedata collected in the PS. The analysis implies the exploration of feasible behaviourinferences from the collected data using a software tool for time series data analysis.We will review the literature on these tools and create or extend one. After this, weare interested in evaluating the methodology that we followed in the PS. Thus, we willneed to interview the PS participants about the usability, user experience and generalthoughts of the monitoring phones they had. Similarly, we need to solve any technicalproblem that occurred like data loss and short battery life. Most importantly, we willidentify the most suitable specifications of phones and sensors for our objectives, aswell as the data sources we need to find behavioural proxies. Even though, that weneed data to support our decision of what behavioural inferences we can pursue, thereare some initial leads that we can check:

Sleep patternsPD patients have problems with the quantity and quality of their sleep. It ishypothesised that patients’ voice (mumbles, screams, speech) recorded from asmartphone sitting near their beds can be used to discern whether a person is

1http://chi2016.acm.org/wp/2http://uist.acm.org/

49

http://chi2016.acm.org/wp/

http://uist.acm.org/

50 CHAPTER 4. FUTURE WORK

asleep or awake.

MobilityIn a similar fashion to the lifespace metric [46], it is hypothesised that mobilitypatterns at a city scale are related to the severity of PD. This is due to the impactof physical disabling conditions as tremor or gait abnormalities, and behaviouralconditions such as depression or apathy. Mobility may be measurable usingGPS, accelerometer data, geographical information systems data, and environ-mental information (weather) and it could be complemented by extra inferenceslike frequently visited places. It may be possible to obtain metrics such as trav-elled distance, physical activity associated with frequented places, time spent athome, etc.

In-house movementIt is hypothesised that physical or behavioural conditions suffered by PD pa-tients have an impact on their movement within their houses. These conditionsmay modify specific activities like going up or down the stairs. This activitycould be measured using inertial sensors (accelerometer), environmental sensors(barometer), environmental information (ambient pressure and temperature) andwireless signals (WiFi or Bluetooth).

Physical activityIt is hypothesised that motor or behavioural conditions suffered by PD patientshave an impact on the intensity and frequency of their physical activity. Physicalactivity may be measurable through global metrics like in [25]. These para-meters may be inferred from a combination of inertial sensors (accelerometer,magnetometer, gyroscope), environmental information (temperature), GPS, andwireless signals (WiFi or Bluetooth).

Calls and text patternsThere is evidence that behavioural conditions suffered by PD patients such asdepression have an impact on their social interactions. We hypothesise that theseinteractions may be reflected in the frequency and number of calls and messagesthat PD patients make or receive in their mobile devices. These interactions canbe ranked and analysed from the logs collected from such devices.

Activities of daily lifeIt is thought that physical conditions suffered by PD patients have an impact on

51

the duration, frequency and type of activities they perform during the day. Wecould pick one activity such as walking or sitting to measure trending changesusing inertial data. To identify the time intervals in which the events occur, wecould use other sources like magnetometer, GPS or wireless signals (WiFi orBluetooth).

Typing patternsIt is hypothesised that tremor and bradykinesia may impact the speed, consist-ency and rhythm of PD patients when typing on a smartphone’s keyboard (sim-ilar to tests like finger tapping). These features can be measured using the datacollected within a mobile device every time a person interacts with the keyboard.

After we obtain remarks and conclusions from the PS, we will use this information toplan and execute a bigger PD monitoring study during the rest of the second year andthe beginning of the third. This includes an application to obtain our own ethics ap-proval and recruit at least 30 patients. As a part of this bigger study, we will collect twodatasets containing the data streams of the sensors and interfaces of a smartphone. Onedataset will support the creation of the yet-to-be-developed algorithms and the othertheir evaluation. We will develop these algorithms in iterations, starting in the data ana-lysis stage of the PS, continuing a few months after the beginning of the main studyand finishing until the end of the third year. These algorithms are a core contributionof my Ph.D. As we mentioned in a previous chapter, their goal is to extract behavi-oural trends from the obtained inferences to measure deviations along time. Thus, wehave lightly consulted the literature to line up some algorithms that we can take as aninspiration and as a starting point.

Frequent Pattern Mining This algorithm is based on binary trees recursively con-structed considering different time granularities. In [4], this algorithm was usedto profile activities of daily living. This work could model patterns of the sameor other time-based events inferred from the collected data.

Topic models Topic models are based on Text Mining algorithms, this approach rep-resents behaviour events as words, their sequences over time as documents andidentifiable routines as topics. In [10, 23, 36] different implementations wereapplied to discover and learn routines of daily activities from labelled and unla-belled sensor data.

Bayesian framework based on instantaneous entropy The authors of [52] develop

52 CHAPTER 4. FUTURE WORK

a new metric and a novel framework based on GPS data to model mobility be-haviour. They can predict deviations from this behaviour over time.

Human behavioural motifs In [76] a set of algorithms was created to extract beha-vioural motifs from unlabelled sensed data using a mobile device to monitorhealthy people for two months. The authors combined different sensors to over-come the uncertainty and complexity of human behaviour. Although the dataanalysed included only WiFi and location (from GPS and cellular network), itsfundamental idea could be ported to other time-based events.

Sparse-coding framework The authors of [9] propose a new framework to modelpatterns of human activities using a small percentage of annotated data. It isapplied to two datasets. The first is used to identify transportation modes (car,bus, bike, walk, etc.), while the second identifies daily activities (opening a door,moving a cup, cleaning a table) from arm movements based on data coming frominertial sensors.

Eigenbehaviours In [19], the idea of eigenvectors (characterising vectors) is appliedto the modelling of human behaviour based on mobile sensed data. It extractsmobility patterns of a group of students monitored for one year. These patternsare based on call logs, nearby Bluetooth devices, cell tower IDs and applicationand phone usage (off, charging, idle).

Fuzzy sets In [50] fuzzy sets and fuzzy rules were used to model social human beha-viour in a work environment. It may be possible to apply this technique to otheraspects of patients’ daily lives.

During the third year and in parallel to the development of the mentioned algorithms,we plan to evaluate the behavioural proxies that we find. To do this, we will follow aprocedure similar to the one used in the PS that was described in the previous chapter.Finally, I will write up my dissertation in the fourth year.

The thesis outline can be consulted in Appendix A.2 and the project plan in AppendixA.3.

4.1. CONCLUDING REMARKS 53

4.1 Concluding remarks

It seems that the assessment of PD progression based on intelligent behaviour infer-ences extracted from interaction data coming from smartphones is feasible. Due tothe exploratory nature of this work, there are several proxies and inferences that can beanalysed and may be correlated with PD severity. The pilot study conducted during theend of the first year will serve as a base for the primary monitoring study, extractingguidelines and best practices that can ease future data analysis. Several techniques needto be tested during this analysis because human behaviour is a complex phenomenonwhich can be segregated and interpreted in different ways. If there is a positive out-come at the end of this proof-of-concept PD monitoring methodology, future workunder the same line of research could have significant impact on the quality of life ofPD patients.

Bibliography

[1] S. Abbate, M. Avvenuti, F. Bonatesta, G. Cola, P. Corsini, and A. Vecchio.A smartphone-based fall detection system. Pervasive and Mobile Computing,8(6):883–899, 12 2012.

[2] N. Aharony, W. Pan, C. Ip, I. Khayal, and A. Pentland. Social fmri: Investig-ating and shaping social mechanisms in the real world. Pervasive and Mobile

Computing, 7(6):643–659, 12 2011.

[3] A. Alexiou, M. Psiha, and P. Vlamos. Towards an expert system for accuratediagnosis and progress monitoring of parkinson’s disease. Advances in experi-

mental medicine and biology, 822:151–64, 1 2015.

[4] R. Ali, B. Lo, and G.-Z. Yang. Unsupervised routine profiling in free-livingconditions can smartphone apps provide insights? In 2013 IEEE International

Conference on Body Sensor Networks, pages 1–7. IEEE, 5 2013.

[5] S. A. Antos, M. V. Albert, and K. P. Kording. Hand, belt, pocket or bag: Prac-tical activity tracking with mobile phones. Journal of neuroscience methods,231:22–30, 7 2014.

[6] S. Arora, V. Venkataraman, A. Zhan, S. Donohue, K. Biglan, E. Dorsey, andM. Little. Detecting and monitoring the symptoms of parkinson’s disease usingsmartphones: A pilot study. Parkinsonism & Related Disorders, 21(6):650–653,3 2015.

[7] M. C. Barroso Junior, G. P. Esteves, T. P. Nunes, L. M. G. Silva, A. C. D. Faria,and P. L. Melo. A telemedicine instrument for remote evaluation of tremor:design and initial applications in fatigue and patients with parkinson’s disease.Biomedical engineering online, 10(1):14, 1 2011.

54

BIBLIOGRAPHY 55

[8] J. Barth, J. Klucken, P. Kugler, T. Kammerer, R. Steidl, J. Winkler, J. Horneg-ger, and B. Eskofier. Biometric and mobile gait analysis for early diagnosis andtherapy monitoring in parkinson’s disease. Conference proceedings : ... Annual

International Conference of the IEEE Engineering in Medicine and Biology So-

ciety. IEEE Engineering in Medicine and Biology Society. Annual Conference,2011:868–71, 1 2011.

[9] S. Bhattacharya, P. Nurmi, N. Hammerla, and T. Plotz. Using unlabeled datain a sparse-coding framework for human activity recognition. Pervasive and

Mobile Computing, 15:242–262, 5 2014.

[10] F. Castanedo, D. L. De-Ipina, H. K. Aghajan, and R. Kleihorst. Learn-ing routines over long-term sensor data using topic models. Expert Systems,31(4):365–377, 9 2014.

[11] E. Cereda, G. Pezzoli, and M. Barichella. Role of an electronic armband in mo-tor function monitoring in patients with parkinson’s disease. Nutrition (Burb-

ank, Los Angeles County, Calif.), 26(2):240–2, 2 2010.

[12] K. R. Chaudhuri, P. Martinez-Martin, A. H. V. Schapira, F. Stocchi, K. Sethi,P. Odin, R. G. Brown, W. Koller, P. Barone, G. MacPhee, L. Kelly, M. Rabey,D. MacMahon, S. Thomas, W. Ondo, D. Rye, A. Forbes, S. Tluk, V. Dhawan,A. Bowron, A. J. Williams, and C. W. Olanow. International multicenter pilotstudy of the first comprehensive self-completed nonmotor symptoms question-naire for parkinson’s disease: the nmsquest study. Movement disorders : official

journal of the Movement Disorder Society, 21(7):916–23, 7 2006.

[13] S. Chernbumroong and A. S. Atkins. Activity classification using a single wrist-worn accelerometer. In 2011 5th International Conference on Software, Know-

ledge Information, Industrial Management and Applications (SKIMA) Proceed-

ings, pages 1–6. IEEE, 9 2011.

[14] E. A. Chrischilles, L. M. Rubenstein, M. D. Voelker, R. B. Wallace, and R. L.Rodnitzky. The health burdens of parkinson’s disease. Movement disorders :

official journal of the Movement Disorder Society, 13(3):406–13, 5 1998.

[15] J.-F. Daneault, B. Carignan, C. E. Codere, A. F. Sadikot, and C. Duval. Using asmart phone as a standalone platform for detection and monitoring of patholo-gical tremors. Frontiers in human neuroscience, 6:357, 1 2012.

56 BIBLIOGRAPHY

[16] S. Das, B. Amoedo, F. De la Torre, and J. Hodgins. Detecting parkinsons’symptoms in uncontrolled home environments: a multiple instance learning ap-proach. Conference proceedings : ... Annual International Conference of the

IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medi-

cine and Biology Society. Annual Conference, 2012:3688–91, 1 2012.

[17] L. M. L. de Lau and M. M. B. Breteler. Epidemiology of parkinson’s disease.The Lancet. Neurology, 5(6):525–35, 6 2006.

[18] B. Dijkstra, Y. P. Kamsma, and W. Zijlstra. Detection of gait and posturesusing a miniaturized triaxial accelerometer-based system: accuracy in patientswith mild to moderate parkinson’s disease. Archives of physical medicine and

rehabilitation, 91(8):1272–7, 8 2010.

[19] N. Eagle and A. S. Pentland. Eigenbehaviors: identifying structure in routine.Behavioral Ecology and Sociobiology, 63(7):1057–1066, 4 2009.

[20] G. Ebersbach, H. Baas, I. Csoti, M. Mungersdorf, and G. Deuschl. Scales inparkinson’s disease. Journal of neurology, 253 Suppl:32–5, 8 2006.

[21] O. Eskidere, F. Ertas, and C. Hanilci. A comparison of regression methodsfor remote tracking of parkinsons disease progression. Expert Systems with

Applications, 39(5):5523–5528, 4 2012.

[22] S. Fahn, R. Elton, and M. of the UPDRS Development Committee. Unifiedparkinson’s disease rating scale. Recent Developments in Parkinson’s Disease,2:153–163, 1987.

[23] K. Farrahi and D. Gatica-Perez. A probabilistic approach to mining mobilephone data sequences. Personal and Ubiquitous Computing, 18(1):223–238, 22013.

[24] D. Ferreira, V. Kostakos, and A. K. Dey. Aware: Mobile context instrumentationframework. Frontiers in ICT, 2, 4 2015.

[25] P. J. Garcia Ruiz and V. Sanchez Bernardos. Evaluation of actitrac (ambulatoryactivity monitor) in parkinson’s disease. Journal of the neurological sciences,270(1-2):67–9, 7 2008.

[26] C. G. Goetz, S. Fahn, P. Martinez-Martin, W. Poewe, C. Sampaio, G. T.Stebbins, M. B. Stern, B. C. Tilley, R. Dodel, B. Dubois, R. Holloway,

BIBLIOGRAPHY 57

J. Jankovic, J. Kulisevsky, A. E. Lang, A. Lees, S. Leurgans, P. a. LeWitt,D. Nyenhuis, C. W. Olanow, O. Rascol, A. Schrag, J. a. Teresi, J. J. Van Hilten,and N. LaPelle. Movement disorder society-sponsored revision of the unifiedparkinson’s disease rating scale (mds-updrs): Process, format, and clinimetrictesting plan. Movement disorders : official journal of the Movement Disorder

Society, 22(1):41–7, 1 2007.

[27] C. G. Goetz, G. T. Stebbins, D. Wolff, W. DeLeeuw, H. Bronte-Stewart, R. El-ble, M. Hallett, J. Nutt, L. Ramig, T. Sanger, A. D. Wu, P. H. Kraus, L. M.Blasucci, E. a. Shamim, K. D. Sethi, J. Spielman, K. Kubota, A. S. Grove,E. Dishman, and C. B. Taylor. Testing objective measures of motor impair-ment in early parkinson’s disease: Feasibility study of an at-home testingdevice. Movement disorders : official journal of the Movement Disorder So-

ciety, 24(4):551–6, 3 2009.

[28] C. G. Goetz, B. C. Tilley, S. R. Shaftman, G. T. Stebbins, S. Fahn, P. Martinez-Martin, W. Poewe, C. Sampaio, M. B. Stern, R. Dodel, B. Dubois, R. Hollo-way, J. Jankovic, J. Kulisevsky, A. E. Lang, A. Lees, S. Leurgans, P. a. LeWitt,D. Nyenhuis, C. W. Olanow, O. Rascol, A. Schrag, J. a. Teresi, J. J. van Hilten,and N. LaPelle. Movement disorder society-sponsored revision of the unifiedparkinson’s disease rating scale (mds-updrs): scale presentation and clinimetrictesting results. Movement disorders : official journal of the Movement Disorder

Society, 23(15):2129–70, 11 2008.

[29] N. Y. Hammerla, J. M. Fisher, P. Andras, L. Rochester, R. Walker, and T. Plotz.Pd disease state assessment in naturalistic environments using deep learning. InConference on Innovative Applications of Artificial Intelligence, 2015.

[30] J. Hicks, N. Ramanathan, D. Kim, M. Monibi, J. Selsky, M. Hansen, and D. Es-trin. Andwellness. In Wireless Health 2010 on - WH ’10, page 34. ACM Press,10 2010.

[31] C. Hintermuller, C. Guger, G. Edlinger, J. Cabestany, I. Rojas, and G. Joya.Monitoring motor fluctuations in parkinsons disease using a waist-worn inertialsensor. 6691:386–392, 2015.

[32] M. M. Hoehn and M. D. Yahr. Parkinsonism: onset, progression, and mortality.Neurology, 17(5):427–427, 5 1967.

58 BIBLIOGRAPHY

[33] X. Hong, C. Nugent, M. Mulvenna, S. McClean, B. Scotney, and S. Devlin.Evidential fusion of sensor data for activity recognition in smart homes. Per-

vasive and Mobile Computing, 5(3):236–252, 6 2009.

[34] B. L. S. N. J. J. C. K. J. S. F. A. Z. K. J. O. N. R. D. E. Hongsuda Tang-munarunkit, Cheng-Kang Hsieh. ohmage: A general and extensible end-to-endparticipatory sensing platform. ACM Transactions on Intelligent Systems and

Technology, 6(3), 2015.

[35] Z. Z. Htike, S. Egerton, and K. Y. Chow. Real-time human activity recognitionusing external and internal spatial features. In 2010 Sixth International Confer-

ence on Intelligent Environments, pages 52–57. IEEE, 7 2010.

[36] T. Huynh, M. Fritz, and B. Schiele. Discovery of activity patterns using topicmodels. In Proceedings of the 10th international conference on Ubiquitous

computing - UbiComp ’08, page 10. ACM Press, 9 2008.

[37] M. Hynes, H. Wang, E. McCarrick, and L. Kilmartin. Accurate monitoring ofhuman physical activity levels for medical diagnosis and monitoring using off-the-shelf cellular handsets. Personal and Ubiquitous Computing, 15(7):667–678, 12 2010.

[38] J. Jankovic. Parkinson’s disease: clinical features and diagnosis. Journal of

neurology, neurosurgery, and psychiatry, 79(4):368–76, 4 2008.

[39] J. Jankovic and A. S. Kapadia. Functional decline in parkinson disease. Archives

of neurology, 58(10):1611–5, 10 2001.

[40] B. Kostek, A. Kupryjanow, and A. Czyewski. Knowledge representation ofmotor activity of patients with parkinsons disease. Natural Computing, 12 2014.

[41] N. Kostikis, D. Hristu-Varsakelis, M. Arnaoutoglou, and C. Kotsavasiloglou.Smartphone-based evaluation of parkinsonian hand tremor: Quantitative meas-urements vs clinical assessment scores. Conference proceedings : ... Annual

International Conference of the IEEE Engineering in Medicine and Biology So-


[42] N. Kostikis, D. Hristu-Varsakelis, M. Arnaoutoglou, C. Kotsavasiloglou, andS. Baloyiannis. Towards remote evaluation of movement disorders via smart-phones. Conference proceedings : ... Annual International Conference of the

BIBLIOGRAPHY 59

IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medi-

cine and Biology Society. Annual Conference, 2011:5240–3, 1 2011.

[43] R. Lakshminarayana, D. Wang, D. Burn, K. R. Chaudhuri, G. Cummins, C. Gal-trey, B. Hellman, S. Pal, J. Stamford, M. Steiger, and A. Williams. Smartphone-and internet-assisted self-management and adherence tools to manage parkin-son’s disease (smart-pd): study protocol for a randomised controlled trial (v7;15 august 2014). Trials, 15(1):374, 1 2014.

[44] a. E. Lang and a. M. Lozano. Parkinson’s disease. first of two parts. The New

England journal of medicine, 339(15):1044–1053, 1998.

[45] R. Lemoyne, T. Mastroianni, M. Cozza, C. Coroian, and W. Grundfest. Imple-mentation of an iphone for characterizing parkinson’s disease tremor througha wireless accelerometer application. Conference proceedings : ... Annual In-

ternational Conference of the IEEE Engineering in Medicine and Biology So-


[46] J. Liddle, D. Ireland, S. J. McBride, S. G. Brauer, L. M. Hall, H. Ding, M. Kar-unanithi, P. W. Hodges, D. Theodoros, P. A. Silburn, and H. J. Chenery. Meas-uring the lifespace of people with parkinson’s disease using smartphones: proofof principle. JMIR mHealth and uHealth, 2(1):e13, 1 2013.

[47] M. a. Little, P. E. McSharry, E. J. Hunter, J. Spielman, and L. O. Ramig. Suit-ability of dysphonia measurements for telemonitoring of parkinson’s disease.IEEE transactions on bio-medical engineering, 56(4):1015, 4 2009.

[48] I. Mandal and N. Sairam. Accurate telemonitoring of parkinson’s disease dia-gnosis using robust inference system. International journal of medical inform-

atics, 82(5):359–77, 5 2013.

[49] H. Martın, A. M. Bernardos, J. Iglesias, and J. R. Casar. Activity logging usinglightweight classification techniques in mobile devices. Personal and Ubiquit-

ous Computing, 17(4):675–695, 4 2012.

[50] J. Martınez-Miranda, M. Alvarado, A. Aldea, and R. Banares-Alcantara. Mod-elling human behaviour at work using fuzzy logic: The challenge of work teamsconfiguration.

60 BIBLIOGRAPHY

[51] S. Mazilu and G. Troster. Wearable technologies: one step closer to gait re-habilitation in parkinson’s patients. XRDS: Crossroads, The ACM Magazine for

Students, 21(2):48–53, 12 2014.

[52] J. McInerney, S. Stein, A. Rogers, and N. R. Jennings. Breaking the habit:Measuring and predicting departures from routine in individual human mobility.Pervasive and Mobile Computing, 9(6):808–822, 12 2013.

[53] M. Memedi, T. Khan, P. Grenholm, D. Nyholm, and J. Westin. Automatic andobjective assessment of alternating tapping performance in parkinson’s disease.Sensors (Basel, Switzerland), 13(12):16965–16984, 2013.

[54] F. Miao, Y. He, J. Liu, Y. Li, and I. Ayoola. Identifying typical physical activityon smartphone with varying positions and orientations. BioMedical Engineering

OnLine, 14(1):1–15, 2015.

[55] D. B. Miller and J. P. OCallaghan. Biomarkers of parkinsons disease (pd):Present and future. Metabolism, 10 2014.

[56] S. T. Moore, H. G. MacDougall, J.-M. Gracies, H. S. Cohen, and W. G.Ondo. Long-term monitoring of gait in parkinson’s disease. Gait & posture,26(2):200–7, 7 2007.

[57] S. T. Moore, H. G. MacDougall, and W. G. Ondo. Ambulatory monitoringof freezing of gait in parkinson’s disease. Journal of neuroscience methods,167(2):340–8, 1 2008.

[58] T. H. E. Nice, G. On, T. H. E. Treatment, M. Of, D. In, and U. Edition. the Nice

Guideline on the Treatment and Depression the Treatment and Management of

Depression. 2006.

[59] M. D. S. T. F. on Rating Scales for Parkinsons Disease. State of the art review theunified parkinson s disease rating scale ( updrs ): Status and recommendations.18(7):738–750, 2003.

[60] F. J. Ordonez, P. de Toledo, and A. Sanchis. Sensor-based bayesian detection ofanomalous living patterns in a home setting. Personal and Ubiquitous Comput-

ing, 19(2):259–270, 9 2014.

BIBLIOGRAPHY 61

[61] D. Pan, R. Dhall, A. Lieberman, and D. B. Petitti. A mobile cloud-based par-kinsons disease assessment system for home-based monitoring. JMIR mHealth

and uHealth, 3(1):e29, 3 2015.

[62] J. Pan. Sensor-based abnormal human-activity detection. IEEE Transactions on

Knowledge and Data Engineering, 20(8):1082–1090, 8 2008.

[63] W. Pan, S. Kwak, F. Li, C. Wu, Y. Chen, Y. Yamamoto, and D. Cai. Actigraphymonitoring of symptoms in patients with parkinson’s disease. Physiology &

behavior, 119:156–60, 7 2013.

[64] M. Pansera, J. J. Estrada, L. Pastor, J. Cancela, R. Greenlaw, and M. T. Arre-dondo. Multi-parametric system for the continuous assessment and monitor-ing of motor status in parkinson’s disease: an entropy-based gait comparison.Conference proceedings : ... Annual International Conference of the IEEE En-

gineering in Medicine and Biology Society. IEEE Engineering in Medicine and

Biology Society. Annual Conference, 2009:1242–5, 1 2009.

[65] S. S. Papapetropoulos. Patient diaries as a clinical endpoint in parkinson’s dis-ease clinical trials. CNS neuroscience & therapeutics, 18(5):380–7, 5 2012.

[66] S. Patel, B.-R. Chen, C. Mancinelli, S. Paganoni, L. Shih, M. Welsh, J. Dy,and P. Bonato. Longitudinal monitoring of patients with parkinson’s diseasevia wearable sensor technology in the home setting. Conference proceedings :

... Annual International Conference of the IEEE Engineering in Medicine and

Biology Society. IEEE Engineering in Medicine and Biology Society. Annual

Conference, 2011:1552–5, 1 2011.

[67] S. Patel, K. Lorincz, R. Hughes, N. Huggins, J. Growdon, D. Standaert,M. Akay, J. Dy, M. Welsh, and P. Bonato. Monitoring motor fluctuations inpatients with parkinson’s disease using wearable sensors. IEEE transactions on

information technology in biomedicine : a publication of the IEEE Engineering

in Medicine and Biology Society, 13(6):864–73, 11 2009.

[68] S. Patel, K. Lorincz, R. Hughes, N. Huggins, J. H. Growdon, M. Welsh, andP. Bonato. Analysis of feature space for monitoring persons with parkinson’sdisease with application to a wireless wearable sensor system. Conference

proceedings : ... Annual International Conference of the IEEE Engineering in

62 BIBLIOGRAPHY

Medicine and Biology Society. IEEE Engineering in Medicine and Biology So-

ciety. Annual Conference, 2007:6291–4, 1 2007.

[69] L. Pepa, L. Ciabattoni, F. Verdini, M. Capecci, and M. Ceravolo. Smartphonebased fuzzy logic freezing of gait detection in parkinson’s disease. In 2014

IEEE/ASME 10th International Conference on Mechatronic and Embedded Sys-

tems and Applications (MESA), pages 1–6. IEEE, 9 2014.

[70] T. M. Piasecki, M. R. Hufford, M. Solhan, and T. J. Trull. Assessing clients intheir natural environments with electronic diaries: Rationale, benefits, limita-tions, and barriers.

[71] N. E. Piro, L. Baumann, M. Tengler, L. Piro, and R. Blechschmidt-Trapp. Tele-monitoring of patients with parkinson’s disease using inertia sensors. Applied

clinical informatics, 5(2):503–11, 1 2014.

[72] B. P. Printy, L. M. Renken, J. P. Herrmann, I. Lee, B. Johnson, E. Knight,G. Varga, and D. Whitmer. Smartphone application for classification of motorimpairment severity in parkinson’s disease. In 2014 36th Annual International

Conference of the IEEE Engineering in Medicine and Biology Society, volume2014, pages 2686–2689, 8 2014.

[73] K. K. Rachuri, M. Musolesi, C. Mascolo, P. J. Rentfrow, C. Longworth, andA. Aucinas. Emotionsense. In Proceedings of the 12th ACM international con-

ference on Ubiquitous computing - Ubicomp ’10, page 281. ACM Press, 9 2010.

[74] C. Ramaker, J. Marinus, A. M. Stiggelbout, and B. J. Van Hilten. System-atic evaluation of rating scales for impairment and disability in parkinson’s dis-ease. Movement disorders : official journal of the Movement Disorder Society,17(5):867–76, 9 2002.

[75] O. Rascol, C. Goetz, W. Koller, W. Poewe, and C. Sampaio. Treatment in-terventions for parkinson’s disease: an evidence based assessment. Lancet,359(9317):1589–98, 5 2002.

[76] R. Rawassizadeh, E. Momeni, and P. Shetty. Scalable mining of daily behavioralpatterns in context sensing life-log data. page 10, 11 2014.

[77] M. H. U. Rehman, C. S. Liew, T. Y. Wah, J. Shuja, and B. Daghighi. Mining per-sonal data using smartphones and wearable devices: A survey. Sensors (Basel,

Switzerland), 15(2):4430–4469, 1 2015.

BIBLIOGRAPHY 63

[78] J.-L. Reyes-Ortiz. Smartphone- Based Human Activity Recognition. SpringerInternational Publishing, 2015.

[79] A. Rodrıguez-Molinero, A. Sama, D. A. Perez-Martınez, C. Perez Lopez,J. Romagosa, A. Bayes, P. Sanz, M. Calopa, C. Galvez-Barron, E. de Mingo,D. Rodrıguez Martın, N. Gonzalo, F. Formiga, J. Cabestany, and A. Catala.Validation of a portable device for mapping motor and gait disturbances in par-kinsons disease. JMIR mHealth and uHealth, 3(1):e9, 2 2015.

[80] L. M. Rubenstein, E. a. Chrischilles, and M. D. Voelker. The impact of parkin-son’s disease on health status, health expenditures, and productivity. estimatesfrom the national medical expenditure survey. PharmacoEconomics, 12(4):486–498, 1997.

[81] A. Salarian, F. B. Horak, C. Zampieri, P. Carlson-Kuhta, J. G. Nutt, andK. Aminian. itug, a sensitive and reliable measure of mobility. IEEE trans-

actions on neural systems and rehabilitation engineering : a publication of the

IEEE Engineering in Medicine and Biology Society, 18(3):303–10, 6 2010.

[82] A. Salarian, H. Russmann, F. F. J. G. Vingerhoets, P. R. P. Burkhard, andK. Aminian. Ambulatory monitoring of physical activities in patients with par-kinson’s disease. IEEE Transactions on Biomedical Engineering, 54(12):2296–2299, 12 2007.

[83] Saloni, R. K. Sharma, and A. K. Gupta. Voice analysis for telediagnosis ofparkinson disease using artificial neural networks and support vector machines.International Journal of Intelligent Systems and Applications(IJISA), 7(6):41, 52015.

[84] A. Sama, C. Perez-Lopez, J. Romagosa, D. Rodrıguez-Martın, A. Catala,J. Cabestany, D. A. Perez-Martınez, and A. Rodrıguez-Molinero. Dyskinesiaand motor state detection in parkinson’s disease patients with a single move-ment sensor. Conference proceedings : ... Annual International Conference of

the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in

Medicine and Biology Society. Annual Conference, 2012:1194–7, 1 2012.

[85] A. Schrag, R. Dodel, A. Spottke, B. Bornschein, U. Siebert, and N. P. Quinn.

64 BIBLIOGRAPHY

Rate of clinical progression in parkinson’s disease. a prospective study. Move-

ment disorders : official journal of the Movement Disorder Society, 22(7):938–45, 5 2007.

[86] W. M. M. Schupbach, J.-C. Corvol, V. Czernecki, M. B. Djebara, J.-L. Gol-mard, Y. Agid, and A. Hartmann. Segmental progression of early untreatedparkinson’s disease: a novel approach to clinical rating. Journal of neurology,

neurosurgery, and psychiatry, 81(1):20–5, 1 2010.

[87] V. Sharma, K. Mankodiya, F. De La Torre, A. Zhang, N. Ryan, T. G. N. Ton,R. Gandhi, and S. Jain. Spark: Personalized parkinson disease interventionsthrough synergy between a smartphone and a smartwatch. Design, User Exper-

ience, and Usability. User Experience Design for Everyday Life Applications

and Services, pages 103–114, 2014.

[88] L. M. Shulman, I. Pretzer-Aboff, K. E. Anderson, R. Stevenson, C. G. Vaughan,A. L. Gruber-Baldini, S. G. Reich, and W. J. Weiner. Subjective report versusobjective measurement of activities of daily living in parkinson’s disease. Move-

ment disorders : official journal of the Movement Disorder Society, 21(6):794–9, 6 2006.

[89] H. Solar, E. Fernandez, G. Tartarisco, G. Pioggia, B. Cvetkovi, S. Kozina,M. Lustrek, and J. Lampe. A non invasive, wearable sensor platform for multi-parametric remote monitoring in chf patients. Health and Technology, 3(2):99–109, 2 2013.

[90] G. Sprint, D. Cook, and D. Weeks. Towards automating clinical assessments: Asurvey of the timed up and go (tug). Biomedical Engineering, IEEE Reviews in,PP(99):1, 2015.

[91] L. Sun, D. Zhang, B. Li, B. Guo, and S. Li. Activity recognition on an acceler-ometer embedded mobile phone with varying positions and orientations. pages548–562, 10 2010.

[92] M. Sung, C. Marci, and A. Pentland. Wearable feedback systems for rehabilita-tion. Journal of neuroengineering and rehabilitation, 2:17, 1 2005.

[93] N. Toosizadeh, J. Mohler, H. Lei, S. Parvaneh, S. Sherman, and B. Najafi. Motorperformance assessment in parkinson’s disease: Association between objective

BIBLIOGRAPHY 65

in-clinic, objective in-home, and subjective/semi-objective measures. PloS one,10(4):e0124763, 1 2015.

[94] A. Tsanas, M. A. Little, P. E. McSharry, and L. O. Ramig. Accurate telem-onitoring of parkinsons disease progression by noninvasive speech tests. IEEE

Transactions on Biomedical Engineering, 57(4):884–893, 4 2010.

[95] A. Tsanas, M. A. Little, P. E. McSharry, and L. O. Ramig. Enhanced classicaldysphonia measures and sparse regression for telemonitoring of parkinson’s dis-ease progression. In 2010 IEEE International Conference on Acoustics, Speech

and Signal Processing, pages 594–597. IEEE, 2010.

[96] A. Tsanas, M. A. Little, P. E. McSharry, and L. O. Ramig. Nonlinear speechanalysis algorithms mapped to a standard metric achieve clinically useful quan-tification of average parkinson’s disease symptom severity. Journal of the Royal

Society, Interface / the Royal Society, 8(59):842–55, 6 2011.

[97] A. T. Tzallas, M. G. Tsipouras, G. Rigas, D. G. Tsalikakis, E. C. Karvounis,M. Chondrogiorgi, F. Psomadellis, J. Cancela, M. Pastorino, M. T. A. Wald-meyer, S. Konitsiotis, and D. I. Fotiadis. Perform: A system for monitoring, as-sessment and management of patients with parkinson’s disease. Sensors (Basel,

Switzerland), 14(11):21329–57, 1 2014.

[98] D. Weintraub and D. J. Burn. Parkinson’s disease: the quintessential neuro-psychiatric disorder. Movement disorders : official journal of the Movement

Disorder Society, 26(6):1022–31, 5 2011.

[99] A. Weiss, S. Sharifi, M. Plotnik, J. P. P. van Vugt, N. Giladi, and J. M. Haus-dorff. Toward automated, at-home assessment of mobility among patients withparkinson disease, using a body-worn accelerometer. Neurorehabilitation and

neural repair, 25(9):810–8, 1 2011.

[100] C.-C. Yang and Y.-L. Hsu. A review of accelerometry-based wearable mo-tion detectors for physical activity monitoring. Sensors (Basel, Switzerland),10(8):7772–88, 1 2010.

Appendix A

Appendices

A.1 List of Parkinson’s Disease symptoms according to[38]

• Motor symptoms

– Cardinal

∗ Rest tremor [+]

∗ Bradykinesia

· Slow execution of activities of daily living (ADL) [+]

· Slow movement and reaction times

· Lost of spontaneous movement

· Decreased arm swing [+]

∗ Rigidity [+]

∗ Postural instability [+]

– Other

∗ Freezing (gait, arm, eyelid)

∗ Shuffling gait [+]

66

A.1. LIST OF PARKINSON’S DISEASE SYMPTOMS ACCORDING TO [?] 67

∗ Festination difficulty arising from chair [+]

∗ Turning in bed [+]

∗ Micrographia [+]

∗ Dystonia (twisting and repetitive movements or abnormal posturescaused by muscle contractions) [+]

∗ Striatal deformity [+]

∗ Scoliosis (deviation of the spine) [+]

∗ Camptocormia (forward flexion of the spine) [+]

∗ Flexed posture

∗ Glabellar reflex [+]

∗ Mirror Movements

∗ Bulbar dysfunction

· Dysarthria (difficulty in articulating words, derived from bradyk-inesia) [+]

· Hypophonia (soft speech)

· Dysphagia (difficulty in swallowing) [+]

· Sialorrhea (excessive salivation, derived from bradykinesia) [+]

∗ Hypomimia (reduced degree of facial expression, derived from bra-dykinesia) [+]

∗ Neuro-ophtalmological abnormalities

· Decreased blink rate from bradykinesia

· Ocular surface irritation

· Altered tear film

· Visual hallucinations

· Blepharospasm (abnormal contraction of the eyelid) [+]

· Decreased convergence

68 APPENDIX A. APPENDICES

· Apraxia of eyelid opening

· Limitation of upward gaze

∗ Respiratory disturbances

· Restrictive

· Obstructive

• Non-motor symptoms

– Autonomic dysfunction [+]

– Orthostatic hypotension (head rush) [+]

– Sweating dysfunction [+]

– Sphincter dysfunction [+]

– Erectile dysfunction [+]

– Seborrhoea [+]

– Weight loss

– Cognitive and neurobehavioural abnormalities (Hedonistic homeostatic dys-regulation)

∗ Depression [+]

∗ Apathy [+]

∗ Anxiety

∗ Fatigue [+]

∗ Anhedonia (inability to experience pleasure) [+]

∗ Hallucinations

∗ Bradyphrenia (slowness of thought) [+]

∗ Tip-of-the-tong phenomenon [+]

∗ Obsessive-compulsive and impulsive behaviour

· Craving

· Binge eating

A.1. LIST OF PARKINSON’S DISEASE SYMPTOMS ACCORDING TO [?] 69

· Compulsive foraging

· Hypersexuality

· Pathological gambling

· Compulsive shopping

· Punding (execution of repetitive and mechanical tasks)

– Sleep disturbances [+]

∗ Excessive sleepiness

∗ REM disorder

∗ Sleep fragmentation

∗ Vivid dreams

∗ Restless leg syndrome (urge to move one’s body to stop uncomfortableor odd sensations)

– Sensory abnormalities

∗ Olfactory dysfunction

∗ Anosmia (inability to perceive odor) [+]

∗ Ageusia (lost of taste) [+]

∗ Pain [+]

∗ Paresthesia (sensation of tingling, tickling, pricking, or burning of aperson’s skin) [+]

∗ Akathisia (compelling need to be in constant motion)

∗ Oral pain

∗ Genital pain

[+] Represent the main symptoms of PD


A.2 Table of Contents

• Abstract

• Chapter I Introduction

– Overview

– Parkinsons disease symptoms and assessment

– Objectives of this research project

• Chapter II Background

– Technology-supported PD monitoring

– Behaviour inferencing

• Chapter III Methodology

– Model of the proxies between behaviour inferences and PD clinical scores

– Data collection experiments

– Data analysis

– Behaviour inference system

– Proxies identification

• Chapter IV Results

– Evaluation of the behaviour inference system

– Evaluation of the found proxies

– Discussion

• Chapter V Conclusions and Further Work

– Conclusions

– Future work

– Publications

• Bibliography

A.3. PROJECT PLAN 71

A.3 Project plan

Figure A.1: Project Plan


A.4 Collectable data froma smartphone

Using Funf app for Android OS

• Device

– Android Info

– Accounts

– Process Statistics

– Activity Services

– Battery Info

– Hardware Info

– Mobile Network Info

• Device Interaction

– Audio Media

– Images

– Applications

– Running Applications

– Videos

– Screen On/Off

• Environment

– Audio Features

– Pressure Sensor

• Motion

– Orientation Sensor

• Social

– Call Logs

– Contacts

– SMS Logs

Using AWARE app for Android OS

• Device

– Processor workload.

– Battery.

– Bluetooth devices.

– WIFI devices.

– Network events.

– Network traffic.

– Telephony (mobile operator andspecification).

• Device Interaction

– Installed applications

– Running applications

– Keyboard strokes (anonymised).

– Application notifications.

– Application crashes.

– Screen usage.

• Motion

– Accelerometer

– Gravity

– Gyroscope

– Linear accelerometer.

– Magnetometer.

– Proximity.

A.4. COLLECTABLE DATA FROM A SMARTPHONE 73

– Rotation.

• Environment

– Barometer

– Weather (temperature, pressure,humidity and other data providedby OpenWeather API’s)

– Lux meter.

– Ambient noise (binary using athreshold in decibels).

– Timezone.

• Location

– GPS location.

– Network towers triangulation.

• Social

– User in call or not.

– Call events.

– Message events.

– Voice call status changes

Glossary

ADL Activities of Daily Living. 7, 8, 16,73

bradykinesia Slowness of movement. 7,73

BSE Blu Studio Energy. 38, 73

dyskinesia Condition characterised by in-voluntary movements. 18, 73

dysphonia A set of voice disorders. 18,73

MDS-UPDRS Movement Disorder Soci-ety Unified Parkinson’s Disease Rat-ing Scale. 8, 9, 73

mentation Mental activity. 8, 73

ON/OFF fluctuations Periods of low med-ication efficacy alleviating PD symp-toms. 17, 73

OS Operative System. 46, 73

PD Parkinson’s Disease. 7, 73

PL Profile of Living. 12, 73

PS Pilot Study. 35, 73

SPS School of Psychological Sciences. 35,73

substantia nigra The substantia nigra isa brain structure located in the mid-brain. 7, 73

UPDRS Unified Parkinson’s Disease Rat-ing Scale. 8, 9, 73

XPE Xperia Compact Z3. 38, 73

74

PARKINSON’S DISEASE PROGRESSION ASSESSMENT USING ...€¦ · through clinical assessments during regular visits of patients to health institutions. These assessments are not an

Documents