Personalised Ambient Monitoring: aiding those with Bipolar Disorder

The PAM project

Personalised Ambient Monitoring:

aiding those with Bipolar Disorder

Sally Brailsford

John Crowe

Christopher James

Evan Magill

The PAM project

2

The PAM project

Enabling health, independence and wellbeing for psychiatric patients through Personalised Ambient Monitoring

The PAM project

A sandpit project• Funded by the Engineering and Physical Sciences

Research Council

• Sandpit theme: “Bringing Care to the Patient”

3

http://www.epsrc.ac.uk/

The PAM project

4

The PAM team • Sally Brailsford, Southampton

• John Crowe, Nottingham

• Christopher James, Southampton (PI)

• Evan Magill, Stirling

• plus 4 PhD students

– Syed Mohiuddin, Pawel Prociow, James Amor and Jesse Blum

Sensors

OperationalResearch

AmbientMonitoring

BehaviouralAnalysis

PAM

The PAM project

5

PAM external steering group• Ms Sylvia Wyatt, Future Healthcare Network, NHS Scotland

• Dr Amy Drahota, Research Fellow, University of Portsmouth

• Mr Peter Jones, Community mental health nurse, Lancashire Care NHS Trust

• Dr Paul Courtney, Consultant Psychiatrist, Hampshire Partnership Trust

• Mr Richard Barritt, Chief Executive, Solent MIND

• Mr James Stubbs, Service User Representative

The PAM project

6

The aims of PAM• To build a system of unobtrusive sensors, linked

(through a standard mobile phone) to a remote computer system, which automatically monitors the activity patterns of people with mental health problems

• To determine whether it is possible to use such a system to obtain ‘activity signatures’ in a manner which is acceptable to the patient and can provide useful information about the trajectory of their health status

• And if this is so, to determine how this information can best be used to maintain health and aid independence

The PAM project

Bipolar Disorder• Severely disabling mental illness which affects

functionality, relationships, employment and quality of life; affects 2% of the UK population (MHF, 2006)

• Bipolar disorder is the 6th most common disabling illness worldwide (WHO, 2004)

• In 2002, the estimated annual cost to the UK NHS of managing bipolar disorder was £199M, of which £70M was spent on hospital admissions (Gupta and Guest, 2002)

• Many pharmacological treatments are available but these can have unpleasant side-effects and adherence is often poor, leading to hospital admission

7

The PAM project

How does BD alter lifestyle?

Manic Depressive

Euphoric behaviour

Increased (excessive) social activity

Psychomotor agitation

Sleep deprivation

Flight of ideas

Low mood

Lack of interest in social interaction

Psychomotor retardation

Insomnia

Concentration problems

The PAM project

9

Managing Bipolar Disorder• Most patients want to manage their own condition,

using medication only when necessary

• Motivated patients of above-average intelligence, interested in self care and independence

• Early warning signs or prodromes can be detected while patient is still “self-aware” and can take action (seek medical help, start medication etc) to avoid hospital admission

• Paper-based “mood diaries” shown to be effective in trials

The PAM project

10

Problems with paper-based systems• Do not provide a sense of control over daily life

• Patients complain about vigilance and energy required

• Problems with accuracy, completeness and honesty of patient-reported data

• Patients may forget to document important details

• Comorbidity and drug response go unmeasured

• No reduction in depressive relapses (Perry et al, 1999)

The PAM project

The aim of PAM• To use a system of electronic sensors to provide an

automated equivalent of a mood diary, which alerts the patient to a change in activity pattern which could signal the onset of a bipolar episode

• Patient would be sent an SMS alerting them to a possible change, which they could then act on (if they chose)

• PAM is mainly aimed at people who live alone

• Aim is to identify a baseline “activity signature” and then identify significant deviations from this

11

The PAM projectDevice Nodes

• Worn– Mobile Phone

• Questionnaire• Gateway Application

– GPS Transceiver– Wearable Accelerometer– Wearable Microphone– Wearable Light Sensor

• Environmental– Microphone– Light Sensor– Passive Infrared Sensors– Micro-switches– Bed Sensor– Camera– Infrared Receiver For Remote Control– PC

The PAM project

Wearable sensor set

GPS module

XYZaccelerometer

Internal accelerometer

GSM location

User input:• General health

questionnaires • Mood self-assessment

Wearable Node• Acceleration

• General light level• Artificial light level

• Ambient sound properties

BluetoothEncounters*

- Bluetooth - 3G / GPRS - User input - Internal

The PAM projectEnvironmental sensor set

Environmental processing unit

• Processing• Storage• Backup• Upload

PIRsensors

Wide-angleCamera

Environmental NodeMonitoring of:

• Remote control activity Main and cupboard doors.• General light level• Artificial light level• Ambient sound.

Bed occupancysensor

Home appliances monitoring• Microwave• Refrigerator

• Oven

- Bluetooth - WiFi - 433 MHz RF

The PAM project

Example data – wearable light levels

Art

ifici

al lig

ht

Genera

l lig

ht

Working Bus awaiting Bus awaiting Commuting Walking Home

The PAM project

Threads of research activity• The four centres collaborated across the project

but we gravitated towards independent themes (as required by the four PhD students)

• accelerometry & behaviour analysis

• outside the home

• BD modelling

• rule-based sensor network

The PAM project

accelerometry & behaviour analysis


The PAM project

19

Accelerometry & Behavioural analysis• Determine what a person is doing (sleeping, eating,

restlessly pacing around, etc) by feature extraction algorithms on sensor data (e.g. the “neuroscale” algorithm)

• Develop an “activity signature” for that individual, describing their normal activity pattern when well

• Develop a set of decision rules which determine whether an individual’s current activity is “normal” – for them – or may indicate the potential onset of a prodrome


The PAM project

20

Tri-axial accelerometry

0 0.5 1 1.5 2 2.5

x 104

0

2

4

6

8

10

12Walk

-800 -700 -600 -500 -400 -300 -200 -100 0 100-15

-10

-5

0

5

10

15

20

25Walk

walking

0 2000 4000 6000 8000 10000 12000 140000

2

4

6

8

10

12Lecture

Acc

eler

atio

n

Samples (1 Hz)-800 -700 -600 -500 -400 -300 -200 -100 0 100

-15

-10

-5

0

5

10

15

20

25Lecture

at a lecture

activity over time

clustered activity


The PAM project

outside the home

outside the home

The PAM projectOutside the homeExample: tracking movement & position

Off-the-shelf GPS module BT enabled accelerometer

13 Feb 2010 13:06:41; G; 5256.0723; -112.181; 0.0; 13 Feb 2010 13:06:42; A; 0.044; -0.888; 0.484; 13 Feb 2010 13:06:44; A; 0.036; -0.892; 0.492; 13 Feb 2010 13:06:45; A; 0.036; -0.892; 0.496;

logfile.txt

outside the home

The PAM project

Positional data and pre-processingoutside the home

The PAM project

Identifying meaningful locationsoutside the home

The PAM project

Activity data – Bluetooth

• Participants on average encountered more than 1000 unique Bluetooth devices of which:

– 80% were one-off encounters– 15% were “occasional” (1-10) encounters– 4% were “frequent” (10-40) encounters– 1% were “regular” (40 or more) encounters

• This data can be used to monitor social interactions and enhance location information

outside the home

The PAM project

BD Modelling

BD modelling

The PAM project

27

Operational Research modelling of PAM• Aim is to develop a “natural history” model for BD

and use it to test the sensitivity and specificity of the PAM algorithms for detecting change in a patient’s health status, in the context of:-

– A random (personalised) selection of sensors– Unknown reliability of the chosen sensors and

the computer network system – Occasional failure (or deliberate removal) of a

sensor– Variety in patient behaviour, in all states of

health

BD modelling

The PAM project

28

Challenges for modelling BD• No OR modelling approach of BD in the literature,

although some Markov models for depression (Patten et al, 2005)

• No universally accepted staging models for BD found in the medical literature

• Symptoms vary among patients ; and patients may exhibit mixed behaviour (manic and depressed)

• Lack of easily measurable criteria

• Took advice from clinical psychiatrist on our Steering Group

BD modelling

The PAM project

“Normal”

Manic

Depressed

Initial conceptual model of BD

BD modelling

The PAM project

30

Final state transition model

= 0 = 1

…

• The parameter represents mental health state: totally depressed ( = 0), “normal” or healthy ( ≈ 0.5) and totally manic ( = 1)

• Each day, with a certain probability, the person may either stay in the same state, or progress to an adjacent state, in steps of 0.01

BD modelling

The PAM project

31

An illustrative sample path for λ

0

0.2

0.4

0.6

0.8

1

1.2

0 100 200 300 400 500 600

Days

Lam

bd

a va

lues

BD modelling

The PAM project

32

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

0.00 0.20 0.40 0.60 0.80 1.00 1.20

Lamda

Hou

rs /

Cal

ls

Sleep Phone

Hours of sleep Phone callsNormal 6 4Depressed 10 1Manic 2 12

BD modelling

The PAM project

PAM-detected physical activity levels during various mood states

33

1

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

0 100 200 300 400 500 600

Days

Ph

ysic

al A

ctiv

ity

Lev

els

Physical Activity Levels (PAL)

PAM detected PAL

BD modelling

The PAM project

rule-based sensor networks

rule-based sensor network

The PAM project

Programming sensor networks (PROSEN)• distribute rules to rule engines embedded in smart

sensors

• flexible programming

• support for run-time updating of rules

• aids personalisation and changing mental states

• initial work in a wind farm setting ….


The PAM project

PROSEN & REED

• REED (Rule Execution and Event Distribution):

– supports the distribution of rules and trigger events

– employs a rule-based paradigm :

• allows sensor networks to be programmed at run time

• allows allow sensor network behaviour to be changed at run time

– allows subscribe-notify service to be constructed– potential for processing, filtering and collating

data


The PAM project

Communications paradigm

• Low-level decision and event driven

• Interact by sending/receiving decisions and events

• Low-level decision:

– <trigger event, condition, action>

Event received from:• components in

PN• Neighbour PN• Policy server

Test of a local state

Executed if the condition is true• manipulate/store

local data• generate events• may generate low-

level decisions


The PAM projectREED Middleware architecture

Decision

Event Event Decision

StorageProcessing Communications

Operation System Interface

Operation System

event

event condition

condition

action

action

…

Middleware Interface

Low-level AI (“novelty” filter)

Sensor diagnostics Sensor controller

…

Decision

Event Function callDecision Space

Initial default decisions

<“power up”, true, “sending HELLO

event”>

<“temp sensor reading update”, “temp < -20”,

“send ‘temp too low’ event to Policy server”>

Decision Space management


The PAM project

39

Mobile phone-centric sensor-based care system


The PAM project

Backend – Gateway Connectionrule-based sensor network

The PAM project

Network Interfacerule-based sensor network

The PAM project

Mobile Phone Based Body Area Network


The PAM project

PAM Sensor Reading (PSR)<Readings>

<Readingset Message_type="gps" Entity="egps" Entity_instance=“aaa_extgps" Frequency="1" Unitoftime="4s" Id="1251993327994" />

<Sr Ref="1251993327994">50.936348, -1.393458, 0.0, 4.0</Sr>

<Readingset Message_type="wl" Entity="w" Entity_instance=“aaa_wearable" Frequency="1" Unitoftime="s" Id="1251993354943" />

<Readingset Message_type="wa" Entity="w" Entity_instance=“aaa_wearable" Frequency="1" Unitoftime="s" Id="1251993354952" />

<Sr Ref="1251993354952">-0.5083, 1.7986, 0.0782</Sr>

<Sr Ref="1251993354952">-0.1173, 1.1339, 0.2346</Sr>

<Sr Ref="1251993354952">-0.0782, 0.8993, 0.1173</Sr>

<Sr Ref="1251993354943">2.0, 0.0</Sr>...


The PAM project

MOBILE RULE-BASED APPLICATIONS

• Custom Symbian S60 Java ME applications installed on the mobile phone interfaced with m-Prolog. (Also Sony-Ericsson)

• PAM-Gateway

– Control data capture from wearable units (such as GPS, accelerometer, ambient light and sound levels)

• PAM-Transfer

– Perform automatic mobile to PC data transmission

• PAM-Q

– Dynamically adjustable questionnaires


The PAM project

RELIABILITY AND ACCEPTABILITY ISSUES

• Mobile phone battery life• On-body gateway disconnection• On-body device form factor issues• Environmental sensor reliability issues• Rule coherence


The PAM project

POWER ISSUES

TIMESTAMP (s)45.25 60.75 76.25 91.75 107.25 122.75 138.25 153.75 169.250

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Profile 1Profile 2Profile 3Profile 4P

ow

er

(W)


No BT & no user applications: 9 hours @ 0.41 w

BT, but no storage: 7.5

hours @ 0.48 w

BT, and storage: 5.5 hours @ 0.67 w

Internal GPS: 5 hours @ 0.68 w

The PAM project

47

Rule Coherence• when rules are:

– changing over time– possibly unique for particular individuals– originating from different stakeholders

• how can we ensure the integrity of the rules

– in particular the lack of conflicts between rules


The PAM project

48

Example: “traditional” feature interaction

• Alice cannot call Charlie

– Originating Call Screening (OCS)

• If Alice calls Bob

– Bob’s Call Forwarding transfers call to Charlie

Alice

Charlie

XOCS

Bob

CFx


The PAM project

49

classes of feature interactions

1. MAI: Two (or more) features control the same device (Multiple Action Interaction)

2. STI: One event goes to different services which perform different conflicting actions (Shared Trigger Interaction)

F Doff

F

on

FDhot

F

hot

Power Saving heater

Envcntrl

temp air con

windcntrl


The PAM project

50

classes of feature interactions

3. SAI: A service performs an action on a device which triggers another feature. The chain might involve any number of links (Sequential Action Interaction, Loops)

4. MTI: The existence of one feature prevents the another one from operating. (Missed Trigger Interaction)

Fclose

D F!

Foff

D Fcold

Env Cntrl blindsmove alarm

Power Saving

temp heat cntrl


The PAM project

51

Conflict Analysis

• Offline and online analysis looking for conflicts between device rules

• Like FI for call control

• Searching for 5 types of conflict:

– STI, SAI, LI, MAI, MTI

• 12 case studies were developed to explore the conflicts

51

Missed Trigger Interaction occurs when the Context Triggering rules delay the activation

of a home gateway.


The PAM project

{jmb, ehm}@cs.stir.ac.uk 52

initial results• 867 tests for combining:

– shared trigger, – multiple action, and– sequential action.

• that is; from 17 features against each other and themselves across the three criteria.

• 410 conflicts detected

• currently being analysed for patterns


The PAM project

in conclusion

in conclusion

The PAM project

54

PAM in practice• A short technical trial of PAM was been carried out

on the four PhD students

• NHS ethics approval obtained for a small patient study (max 4 patients) of PAM: completed in Southampton but still under way in Scotland

• Many practical aspects highlighted in clinical study!

• Further technical developments under discussion

• Further collaborations planned

in conclusion

The PAM project

Moving forward – other PAM like projects

• PSYCHE (Personalised monitoring SYstems for Care in mental HEalth) project develops a personal, cost-effective, multi-parametric monitoring system based on textile platforms and portable sensing devices for the long term and short term acquisition of data from selected class of patients affected by mood disorders.

• http://www.psyche-project.org

55

in conclusion

The PAM projectMONARCA

• MONitoring, treAtment and pRediCtion of bipolAr disorder episodes (Monarca)

• EC funding of near €4m

• An example of recently funded EU projects in this field.

• Monarca is investigating aspects of bipolar disorder disease by adopting a holistic approach to its assessment, treatment and self-management. The project focuses on objective assessment and prediction of bipolar disorder episodes and aims to advance the discovery of new markers for this disease.

56

in conclusion

The PAM projectOPTIMI• Online Predictive Tools for Intervention in Mental

Illness

• EU funding

• The Neuroscience Institute at the University of Bristol.

• The aim is to develop tools to perform predictions based on early identification of the onset of an illness by monitoring poor coping behaviour.

• The system will study an individual's behaviour patterns over a sustained period and spot any baseline changes suggesting they are becoming unwell.

• Will use wearable sensors and sensors fitted to domestic appliances to measure activity levels. EEG readings, voice analysis and physical activity analysis will be used.

57

in conclusion

The PAM project

58

Thank you for listening

The PAM project

State-of-the-art Health Sensor Networks• Wearable Sensor Networks & Body Sensor Networks for

medical and psychiatric monitoring is active an research area:

– Alarm-Net

– CodeBlue

– Care in the Community

– UbiMon

– MobiCare

– LiveNet

The PAM project

“Normal”

Manic Depressed

Initial conceptual model of BD

BD modelling

The PAM project

61

MDN

2

1

213214 22

N = value of parameter X when normal (λ close to 0.5)D = value of parameter X when depressed (λ close to 0)M = value of parameter X when manic (λ close to 1)

where X = number of phone calls made daily, or number of hours of sleep per 24-hour period

Using λ to model behaviourBD modelling

The PAM project

Time t in days

Mental health state

λ(t)

Actual activity on day t

PAM-detectedactivity on day t

Trigger alert?

Individual’s activity whennormal, manic or depressed

Decision rules

Sensor accuracy

and reliability

BD modelling

The PAM project

Patient types (the P in PAM) • Different people will accept different levels of

monitoring

• Defined on the basis of prodromes rather than sensors

– Patient types 1 to 10 chose a selection of two different prodromes

– Patient types 11 to 19 chose a selection of three different prodromes

– Patient types 20 to 24 chose a selection of four different prodromes

– Patient type 25 chose all five

• Pragmatic choice given vast number of combinations of actual sensors

63

BD modelling

The PAM project

Model outputs• True positive alerts (TP) and false positive alerts

(FP)

• True negatives (TN) and false negatives (FN)

• Average number of days to detect the onset of a depressive episode (ODE)

• Average number of days to detect the onset of a manic episode (OME)

• The ideal would be a very low FP, a very high TP, and very low ODE and OME

64

BD modelling

The PAM project

Example Results for Dataset

65

Patient types Choices of prodromes

Minimum no. of sensors required

ODE (days)

OME (days)

TP (%) FP (%)

Type 25

Activity level

Sleep

Talkativeness

Social energy

Appetite

Accelerometer; GPS; TV usage sensor; Pressure mat; Light sensor; Microphone; Phone

sensor; Camera; Cupboard door sensors

5.90 2.64 87.48 2.12

Type 20

Activity level

Sleep

Talkativeness

Social energy


sensor

8.10 3.12 85.22 0.85

Type 22

Activity level

Sleep

Social energy

Appetite

Accelerometer; GPS; TV usage sensor; Pressure mat; Light

sensor; Microphone; Camera; Cupboard door sensors

8.17 3.54 84.16 1.30

Type 21

Activity level

Sleep

Talkativeness

Appetite


sensor; Camera; Cupboard door sensors

8.37 4.01 83.65 1.05

BD modelling

The PAM project

PAM Infrastructure Vision

The PAM project

Implications• PAM was found to be inadequate for almost all the

personalised choices of two prodromes only

• PAM was found to be efficient for most choices of three prodromes.

• PAM was found to be less effective for a few specific combinations of personalised prodromal choices, especially ‘sleep’, ‘talkativeness’ and ‘social energy’, or ‘talkativeness’, ‘social energy’ and ‘appetite’, because these prodromes were associated with relatively few observable behaviours

• To be able to effectively offer choices such as these, the PAM system would need to increase the number of their associated observable behaviours

67

BD modelling

The PAM project


example: Context Triggering System1 % respond to changes upon receiving contextual information

2 cds_cts(Trigger,T) :-

3 T2 is T+1,

4 assert(happens(listen_for_connection,T)),

5 assert(happens(make_connection,T)),

6 assert(happens(receive_data,T)),

7 assert(happens(checks_data,T)),

8 assert(happens(listen_for_connection,T2)),

9 ((

10 holdsAt(message(Trigger), T2),

11 assert(initiates(checks_data,prompt(Trigger),T)),

12 assert(terminates(checks_data,message(Trigger),T))

13 );

14 assert(terminates(checks_data,message(Trigger),T))).

The PAM project


Conflict Analysis

• Offline and online analysis looking for conflicts between device rules

• Like FI for call control

• Searching for 5 types of conflict:

– STI, SAI, LI, MAI, MTI• 12 case studies were

developed to explore the conflicts

69

Missed Trigger Interaction occurs when the Context Triggering rules delay the activation

of a home gateway.

The PAM project


Detection Approach

• Prolog-based framework

• Evaluates pairs of feature rules to determine whether they are concordant or conflict

70

Example diagram describing MTI conflict detection rule

The PAM project


Device Priority Approach to Resolution

• Allows precedence across devices without their knowledge of each other

• How it works

1. Resolver receives a list of conflicts, device priorities and device rules

• Priorities are declared as ordered preference lists of particular properties (such as power efficiency, bandwidth minimisation, data integrity, etc)

• Rules may be listed for each property2. Resolver determines rules that should be disabled

71

The PAM project


Analysis Results

72

SAI Case Study

Data Transfer Data Transfer SAI

Data Transfer Data Redirect SAI

Data Redirect Data Transfer SAI

Data Redirect Data Redirect Concordance

MTI Case Study

Notification suppression Notification suppression MTI

Notification suppression Response prompting MTI

Response prompting Notification suppression MTI

Response prompting Response prompting MTI

Personalised Ambient Monitoring: aiding those with Bipolar Disorder

Documents

pam project2the pam

pam project10problems

pam projecthow

mental health problemsto

health statusand

community mental health

uk nhs

impact cost of mental