CommCare Workshop_Javetski and Wacksmon_4.22.13

1

“CommCare Workshop: How to Plan a

CommCare Workshop”

CORE Group Spring Pre-Meeting Workshop April 22, 2013

2

Drawing from the last four years of implementing CommCare, this workhop will provide:

§  An overview of Dimagi’s mHealth products and the potential benefits for equipping CHWs with CommCare

§  Ways to plan for initial implementation and long-term support of the project (beyond equipping frontline workers with phones)

§  Tools to support your organization in planning its CommCare implementation

§  Short report-outs from CORE Group members who have implemented CommCare

§  A discussion about the benefits in implementing an mHealth solutions like CommCare and associated challenges for your organizations.

“How to Plan a CommCare Project”

3

In addition to these slides, you will receive following materials to help you design/implement CommCare

§  Our CommCare beginner’s tutorial: to show you how to design a CommCare application

§  CommCare Worksheet Manual: to guide you through the process of designing your application

§  Implementation tools: Our Total Cost of Ownership (TCO) model, case studies, and guidelines about creating a CommCare project

§  Support resources: including information about our CommCare help site, access to the CommCare Users Google Group, CommCare Exchange (our open-source “app store”), and various mHealth list-servs

§  Information about upcoming Proof of Concept opportunities

Materials/Tools You Will Receive

4

Which areas would you like us to emphasize?

1.  Dimagi’s CommCare Platform 2.  Dimagi’s SMS tool 3.  Dimagi’s logistics tracking system 4.  CommCare evidence 5.  Designing a CommCare application 6.  Technical demo of building a CommCare application 7.  Process for implementing a CommCare project 8.  Tools to implement your CommCare project 9.  CommCare Case studies 10. Additional time for breakout groups with other

workshop attendees

5

Dimagi Product Overview

6

§ We are a leader in open source mobile technology. Ø This means that our code is

free on the Internet Ø Anyone can download, use, or

modify our code.

§ We make software solutions that do not require software developers to deploy

§ We focus on creating solutions for resource-limited settings

About Dimagi

7

Cross Sector Support

Dimagi  Products  

Health  

Logis4cs  

Agriculture  

Educa4on  and  Training  

Water  and  Sanita4on   Financial  

Services  for  the  Poor  

Emergency  Response  

Gender  Equality    &  Women’s  

Empowerment  

Governance  and  

Accountability  

Select  Implementers  

In Discussions

8

Who & Where We Are

§ Team of 50+ scientists, public health experts, physicians, engineers, and field consultants

§ 10+ years of experience with over 100 projects across numerous sectors

§ We have offices in Cambridge, Massachusetts (HQ), India, South Africa, and Mozambique

§ We also have field teams in Guatemala, Senegal, Benin, and Thailand

9

Global Project Locations

Key Stats

§  31 Active countries §  7 Active Sectors

§  492 Projects to Date §  6,050 Users to Date §  350,000 Cases to Date §  1.6 Million Forms to Date

§  135 Active Projects §  2,027 Active Users §  61,213 Active Cases §  200,875 Forms last Month

10

DIMAGI SOFTWARE PRODUCTS

11

Logis&cs  Communica&on  Data  and  Workflow  

Dimagi Products – Powered by CommCare

•  Data collection and case management solution

•  Java feature phones or Android smartphones & tablets

•  Secure and scalable

•  2-way SMS-based applications

•  Any SMS-enabled mobile device.

•  SMS point of service logistics management systems

•  Manage inventory and logistics for remote sites.

Implementa)on  Services:    technology enablement, design, and support

CommTrack CommConnect CommCare

12

COMMCARE (DIMAGI’S MOBILE PLATFORM)

13

CommCareMobile •  Community Health Workers •  Research Assistants •  Surveyors, etc.

CommCareHQ •  CHW Supervisors •  Project Managers •  Researchers •  Dimagi Field Managers

CommCare

14

CommCare: DEMO

CommCare Overview: http://tinyurl.com/crscommcare

15 15

Register and find existing clients

Specialize your application

Record client information and share interactive counseling

messages

Select                                                          Exit  

SMS  Reminder    Reena  is  overdue  for  her  follow-­‐up  treatment  please  follow-­‐up  with  her.  

Follow-up with clients

CommCare Mobile

16

Case Management Track Clients Over Time

-  Routinely visit the same clients

-  Manage entire case lifecycle offline if connectivity is unavailable

-  All data stored locally on the workers’ mobile device

Monitor Cases From the Web

- View all registered cases online

- Track how data is changing over time

- Monitor worker productivity

17

CommCare Application Builder Create Your Own Application

-  Online tools to build your own CommCare application

-  Collect information including dates, multiple choice, images, video, bar code, and GPS.

-  Manage applications remotely

Download From the App Store

- Leverage previously built apps to get started quickly

- Share your own apps with the community

18

CommCare Reports Continuous Performance Improvement

-  Support for Project Coordinators in managing workforce

-  Analyze all data submitted to server to improve worker performance

-  Targeted, actionable follow-up activities for supervisors

-  Can utilize Active Data Management for enhanced reporting

19

CommCare Examples

Pathfinder International (Kenya) •  Currently deploying entire Kenyan Essential

Package for Health •  Focuses on HIV/AIDS, TB, MNCH, malaria PATH (Navi Mumbai, Maharashtra, India) •  Project from May-August, 2012 for link workers

(LWs) in two health posts. •  Surveyed all eligible women in urban areas,

tracked pregnancies, and educated women about pregnancy and neonatal health.

•  Family planning counseling was the most frequently used audio prompt and was included in multiple locations at the request of the LWs.

20

CommCare Examples

Real Medicine Foundation (Madhya Pradesh, India) •  Child malnutrition identification and

counseling, with referral pathways URC (Toffo, Benin) •  Application is contributing to the uptake

of family planning services, FP commodity stock monitoring, and pricing

Catholic Relief Services (Uttar Pradesh, India) •  Helping 285 ASHAs track pregnancies •  ASHA supervisors monitor ASHAs’

performance through weekly reports

21

COMMCARE EVIDENCE

22

Community Health Worker Challenges

Accountability  -­‐  Monitoring  reports  delayed    -­‐  Only  aggregate  data  delivered  

-­‐  Data  quality  low  

Access  -­‐  Many  eligible  beneficiaries  not  enrolled  

-­‐  Missed  visits  and  referrals    -­‐  Inefficiency  -­‐  High  CHW  aPri4on  

Quality  -­‐  Short  visits  -­‐  Key  steps  skipped  -­‐  Sensi4ve  issues  avoided    -­‐  Insufficient  training  

Experience  -­‐  Job  aids  leT  at  home  -­‐  Low  credibility  -­‐  Messages  not  engaging  

23

Technology      PlaUorm  

Services    &  Adop4on  

Innova4on  &    Research  

Smarter Community Services

Measure

24

CommCare Evidence Base

Evidence  Level   #  Published  CommCare  

#Unpublished  CommCare  

#  Published  Alterna)ve  

#  Published  Related  

Total  

Conceptual     4   1   5  Implementa4on  narra4ves    

5   1   6  

Qualita4ve  CHW  interviews  

2   2   1   5  

CHW  Process  improvements  

3   2       3   8  

Client  KAP   2   2  Total   14   6   2   4   26  

§ Review of 26 papers showing improvements in Access, Quality, Experience, and Accountability of Care.

§ Available at: http://tinyurl.com/CommCareEvidence

25

CommCare: Evidence Partner   Experiment     Result  

ACCE

SS   Univ.  of  Washington   Randomized  controlled  study  on  87  CHWs      • Increase  4meliness  of  visits  by  86%.  

Harvard  Univ.   Blinded  comparison  on  1198  visits   • Task  shiTing  to  counselors  to  triage  HIV  pa4ents  with  86.5%  sensi4vity  and  48.3%  specificity.  

QUAL

ITY   Univ.  of  Washington   Pre/post  study  on  24  pa4ent  visits   • Improves  adherence  to  protocols  by  ~20%  

Harvard  Univ.   Pre/post  study  on  1221  pa4ent  visits   • Improves  adherence  to  protocols  by  ~18%.  • 1-­‐4%  improvement  in  classifica4on  accuracy,  compared  to  expert  clinician  (not  sta4s4cally  significant).  

IntraHealth   Pre/post  study  on  30  CHWs   • CHW  knowledge  of  danger  signs  increased  b/w  48%  to  70%    

EXPE

RIEN

CE  

Dimagi   Qualita4ve  mul4-­‐country  inves4ga4on   • Improves  ability  to  effec4vely  engage  their  clients.  Berkeley   Controlled  study   • Improves  CHW  performance,  confidence,  and  client  

engagement  Univ.  of  Pennsylvania  

Qualita4ve    user  experience  research   • Provide  credibility  to  the  message  of  CHWs  • Allow  CHWs  to  work  around  cultural  and  social  barriers  

Nanyang  Tech.  Univ.   Cross-­‐sec4onal  study,  in-­‐depth  interviews,  small  group  interviews  

• Nurses  reported  greater  reten4on  of  messages  among  CHWs.    • Messages  delivered  via  the  cell-­‐phone  added  authen4city.    

MicrosoT  Research  India  

Formal  evalua4on  of  10  CHW  deployment     • 90%  CHWs  self-­‐report  improved  social  respect  in  community  from  using  CommCare.  

ACCO

UNTA

BILITY

MicrosoT  Research  India  

Formal  evalua4on  of  10  CHW  deployment   • Reduced  4me  to  get  data  to  program  coordinator  by  98%.    • Improved  data  completeness  from  67%  to  84%.    

Univ.  of  Washington   10-­‐fold  cross  valida4on  with  known  falsified  and  known  true  data    

• Outlier  detec4on  algorithms  shown  to  detect  false  data  set  produced  in  Tanzania  with  80%  sensi4vity  of  90%.  

26

CommCare Benefits Frontline Workers

Accountability  +  CommCareHQ  provides  real  4me  monitoring  of  daily  ac4vity  of  each  CHW  

+  Dimagi’s  Ac4ve  Data  Management  improves  workforce  performance  +  Increase  Efficiency    

Access  +  Increases  4meliness  of  care  due  to  appointment  reminders    

+  Increases  client    enrollment  

+  Increases  reten4on  through  case  management  

+  Increases  rate  of  referrals  

Quality  +  Checklists  improve  performance  

+  Decision  support  increases  adherence  to  protocols  

+  Video,  audio  conveys  sensi4ve  topics  in  authorita4ve  voice  

Experience  +  Audio,  images,  and  video  are  easy  to  carry  on  phone  

+  Phone  gives  CHW  greater  credibility    

+  Audio  and  video  engage  clients  more  than  paper  materials  

27

Improved Access

Ø  85% More timely Visits Ø  Randomized Controlled Trial

B. DeRenzi, L. Findlater, G. Borriello, J. Jackson, J. Payne, B. Birnbaum, T. Parikh, N. Lesh, “Improving Community Health Worker Performance Through Automated SMS”, ICTD 2012, to appear

28

Improved Quality

Ø  20% Improvement to Protocol Adherence in a Randomized Controlled Trial

Ø  Intrahealth Preliminary Result in India: ASHAs had increased their knowledge retention of at least 3-5 key danger signs from 48% at baseline to 70%

Ø  Improvement in IMCI protocol adherence

B. DeRenzi, N. Lesh, T. Parikh, C. Sims, W. Maokla, M. Chemba, Y. Hamisi, and others, “E-imci: improving pediatric health care in low-income countries,” CHI 2008, pp. 753–762.

Inves)ga)on Current  prac)ce  adherence e-­‐IMCI  adherence p-­‐value

Vomi4ng 66.7%  (n=24) 86%  (n=28) -­‐

Chest  indrawing 75%  (n=20) 94%  (n=18) -­‐

Blood  in  stool 71%  (n=7) 100%  (n=3) -­‐ Measles  in  the  last  

3  months 56%  (n=9) 95%  (n=21) <  0.05

Tender  ear 0%  (n=1) 100%  (n=5) -­‐

All 61%  (n=299) 85%  (n=359) <  0.01

29

Improved Experience

Qualitative Assessment Ø  Phones/CommCare add credibility to CHWs Ø Multimedia helps engage clients and their families Ø  Audio helps CHWs recall key counseling messages Ø  Audio facilitates discussion on sensitive topics

Treatman, D., Lesh, N, Strengthening Community Health Systems with Localized Multimedia, M4D’12 to appear. Chittamuru, D. and Bhavsar, M. (2012). CommCare: Evaluation of a Mobile Application for Maternal Health in Rural India. IAMCR Durban: Communication Policy and Technology: Critical Perspectives on Digital Inclusion and Policy. Durban, South Africa.

30

Improved Accountability

Detec)ng  Surprising  Answers  §  Analyze  distribu4ons  of  

answers  by  a  CHW.  §  Example  

Ques4on:  Did  you  give  any  referral  in  this  household?  Overall:  93%  No,  7%  Yes  CHW  #3:    62%  No,  38%  Yes  

§  Detects  systema4c  errors    §  Supports  quality  control  

Detec)ng  Fake  Forms  

(Work by Ben Birnbaum et. al. ‘2012)

31

COMMCONNECT (DIMAGI’S SMS TOOL)

32

What is CommConnect?

Use Cases §  Large scale surveys or behaviour change communication (BCC) §  Drug adherence programs and simple patient tracking (wider reach) §  Reminder integration with existing CommCare projects (ex. BCC

communication to cases, reminders of clinic visits)

•  Larger-scale communication system (SMS, SMS Reminders, IVR, etc.)

•  Stand-alone (ex. surveys) or tied

into CommCare cases (cases can drive reminders)

33

CommConnect Features § Robust survey and reminder builder

§ Outgoing surveys and reminders based on rules

Ø Rules can be date/time based or based on state of a particular beneficiary (e.g., text all women due this month)

§  SMSes can be sent to beneficiaries or to the person who registered the beneficiary (e.g. a community health worker)

§  Incoming data collection support through SMS

§ Fully integrated with CommCare and CommCareHQ

34

How It Works §  Interactive communication (SMS Survey or IVR) are made

on CommCareHQ Ø Can use the same form in CommCare and through SMS

§ Reminders/surveys are designed through CommCareHQ

35

Coverage

§ Outgoing Ø Close to worldwide, but some messages may come from an

international number

§  Incoming Ø India Ø United States Ø Tanzania Ø Malawi Ø Nigeria

§ Future: Setup an Android phone as a gateway in a country

36

Possible CommConnect Usages §  Targeted and Group Messaging

§  SMS Mailing Lists

§  Charts

§  Maps

§  Appointment reminders

§  Scheduled events

§  Referral follow-ups

§  Adherence measurement algorithms

§  Session-based interactive protocols

 36

37

COMMTRACK (LOGISTICS MANAGEMENT)

38

What is CommTrack?

For health workers in low-resource settings who store and manage commodities, CommTrack is a logistics system-strengthening tool that provides reliable, real-time, and actionable information to improve logistics management in low-resource settings.

39

CommTrack: SMS Logistics Platform §  SMS data collection of stock information from any point in the logisitcs

chain, from the service delivery point, facility, district, or block level

Ø  Stockout notification, reporting and routine data collection

Ø  Forecasting/planning

Ø  Order fulfillment

§  Web based reports aggregate and display data for more relevant and accurate decision making

§  Currently in use in Tanzania, Ghana, Malawi, and Uganda. The Ministry of Health in both Tanzania and Uganda are in the process of deploying this technology on a national scale.

12 April 2012

40

CommTrack v1 (ILSGateway in Tanzania)

41

With CommTrack, you can… § Submit stock reports via SMS or CommCare Ø You can also revert mistaken reports

§ Send SMS/email alerts and reminders § Track stock levels, stockouts, receipts, disbursements,

etc. § Access web reports with stock levels, stockouts,

reporting rates, and also auto-calculate monthly consumption

§ Use configuration tools to specify products, facilities, and locations and open APIS to expose data for other logistics systems

42

There are two faces to CommTrack Standalone Mobile Logistics Hub §  For national deployments

§  Example end users: Ø Pharmacists at facilities

Ø Warehouse managers

Add-On to CommCareHQ (under development)* §  Part of an integrated mobile job aid for health workers §  Example end users:

Ø CHWs who also manage commmodities

Ø Village-level entrepreneurs Ø Mobile salesforce

* Note that CommTrack is currently built on RapidSMS,

but will soon be integrated into CommCareHQ.

43

CommCare Pricing Packages

44

CommConnect Pricing §  Outgoing Messages: $250/month §  Both Incoming and Outgoing Messages: $500/month §  Plus $0.01 and cost of any SMS sent or received

Prices include access to CommCareHQ functionality including the mobile phones, application builder, API access, data exports and standardized reports.

45

CommTrack Pricing

§ CommTrack Plus (same as CommCare) § CommTrack Full (same as CommCare) § Plus ~1 cent per-SMS fee (same as CommConnect) § Plus 0.75 USD per-CommCare user fee (same as

CommCare) §  Software is not yet mature enough for a free/self-

service layer although we want to provide that soon § CommTrack technical functionality could be provided as

part of a CommCare PLUS or FULL contract today - but not FREE (yet)

46

CommCare: Implementation

47

Agenda

Application Development 2

Implementation 3

Getting Started 1

48

GETTING STARTED

49

Addressing Challenges

Accountability  -­‐  Monitoring  reports  delayed    -­‐  Only  aggregate  data  delivered  

-­‐  Data  quality  low  

Access  -­‐  Many  eligible  beneficiaries  not  enrolled  

-­‐  Missed  visits  and  referrals    -­‐  Inefficiency  -­‐  High  CHW  aPri4on  

Quality  -­‐  Short  visits  -­‐  Key  steps  skipped  -­‐  Sensi4ve  issues  avoided    -­‐  Insufficient  training  

Experience  -­‐  Jobaids  leT  at  home  -­‐  Low  credibility  -­‐  Messages  not  engaging  

50

The Players: Partner Organization

§ Mobile User: User who will be entering data in CommCare on the phone

§ Technical Officer (TO): Managing the phones, working out any technical issues.

§ Trainer- leads the training along with the Dimagi team § Project Coordinator (PC): Will manage and supervise

users on the CommCare HQ website § M&E Officer: Will use data from CommCareHQ § Project Manager (PM): Oversees project

51

Dimagi’s Field Presence

Field Manager (FM): § Working with partner to design the application § Building the application § Training mobile users to use application § Training project staff to use CCHQ tools § Training project staff for technical troubleshooting § Remote support for deployment

52

Design Under the Mango Tree

52

User-focused Design

-  Rapid content prototyping

-  Mature technology platform

53

The Process

§ Decide what we the program will do Ø  What is the goal? Ø  Who are the users?

§  Iteratively design the app § Develop/build the app § Pilot/test the app with users § Make changes § Build capacity of local team to sustain, expand § Train all users and staff

54

Standard Implementation Plans

Duration Phase 1 Phase 2 Phase 3

Design & Preparation # weeks

§  Identify key Program Team members 2

§  Specify requirements & features, and Build application prototype

4

§  Procure Equipment & design plan for logistics 3

Refinement & Iteration

§ Train project coordinator, supervisors, & pilot CHW groups

2

§  Gather, summarize, & iterate feedback 3

§  Software Development Sprint 4

§ Finalize application & training materials 2

Training & Deployment

§ Project deployment – training starts 1

§ 1st Training cohort of CHWs & distribute phones/SIM

3

§ 2nd Training cohort 3

§ 3rd Training cohort 3

§ 4th Training cohort 3

§ Data Collection & Analysis -

Monitor  through  CommCareHQ  and  ADM  

55

APPLICATION DEVELOPMENT

56

Mobile Application Development § 1- Identify the goals to be achieved by the mobile

application Ø Data collection Ø Workflow support Ø Content delivery

§ 2- Design the system around the skills, knowledge and challenges of CHWs Ø Education Ø Performance

§ 3- Build and revise the application based on observations and feedback from the field Ø Usability, Content, Multimedia, Impact

57

1- What do you want your application to do?

Data collection?

Counseling?

Training reinforcement? Workflow support?

58

1- Know Your Program: Data Usage How does the organization want to analyze data?

•   Think  about  the  variable  names/keywords  when  designing  the  applica4on  •   Make  sure  they  understand  the  difference  between  case  exports  and  form  exports  •   Set-­‐up  or  show  them  how  to  set  up  saved  reports    

Who will be monitoring the data? •   Can  you  do  things  that  are  easier  for  field  staff  downloading  the  data?  •   Do  we  need  to  make  APIs  available  to  partners?  •   Do  they  need  indicator  reports  (custom,  NOT  free)  

What indicators are being monitored?

•   Are  they  collec4ng  these  in  the  applica4on?  

59

1- Know Your Program: Groups What are the denominations important for the program?

•   Care  coordina4on  (ASHA,  AWW,  ANM  teams)  •   Supervisors  (cluster  groups)  •   Government  (PHCs,  blocks,  districts)  •   Program  Managers?  •   Trainers  (training  groups)  •   Researchers?      

Who is involved in monitoring and supervision? What does the partner want? What is easiest for the field team? How will the data be analyzed?

60

1- Worksheet

61

2- Know Your Mobile User: Case List

•  How does a CHW plan home visits?

•  How does a CHW prioritize cases?

•  What information is absolutely critical for identifying cases?

•  What kinds of acronyms can be used? Symbols?

•  How can we leverage the sort/filter feature? Is it too complicated?

62

2- Know Your Mobile User: Case Detail View

•  Which data points will help distinguish cases that look identical on case list?

•  Which critical data does CHW need access to offline?

•  Should we use the call feature from detail screen?

A Review Form can be an alternative for low literate users with multimedia prompt.

63

2- Know Your Mobile Users: Language

Input

ALKA

अलका

64

3- Build App: CommCareHQ.org

65

3- Exchange: An Open Source App Store for CommCare (www.commcarehq.org/exchange)

66

IMPLEMENTATION

67

M-Health Training for an Entire Organization!

Train field supervisors

Train technical staff

Train the trainers Train the pilot CHWs Train all CHWs

68

Adoption and Scale-Up

Mobile   Web/HQ  Applica)on   Troubleshoo)ng   Monitoring   Data    Analysis   Maintenance  

CHW   ü  

Supervisors   ü   ü   ü  

Trainers   ü   Op4onal  

Technical  Support    

ü   ü   ü/Op4onal  

Managers   ü   ü   ü  

Data  Staff   ü   ü  

Organizational training needs for CommCare

69

Experiences from Training Programs §  Roll out initial training program at a slower pace, with ample time

allocated for follow-up soon after training, especially for groups with lower literacy

Content  Training

Mobile  Training

n  Days Day  1 Day  2

A.

B.

C. Week  1 Week  2 Week  n

Day  1 Day  2 Day  n

In field follow-up Refresher training

70

How to get started/resources (will send out after workshop)

§ General CommCare Support Ø  CommCare Help Site (“The wiki”) Ø  CommCare Users Google Group

§ Getting Started Ø  CommCare Exchange Ø  Worksheet

§ Helpful listservs Ø ICT4CHW Ø M-Health working group

71

CommCare: Impact and Tools

72

TOTAL COST OF OWNERSHIP

73

CommCare: Example India TCO model §  Surveyed CommCare partners in India to identify complete costs for an

organization to deploy CommCare

§  Produced (total cost of ownership) TCO tool to help organizations estimate total costs, explore permutations, and plan budgets – Excel Model Available.

Category   Key  Elements   Annual  (USD)  

Avg.  Training  Costs   2  days  ini4al,  1  day  annual  recurring    ($9  per  day)   $11  

CHWs  Op  Costs   Data  Transfer,  SMS,  Charging/Electricity   $9  

Field  Staff  Op  Costs   1  per  500  CHWs  ($6,600  per  year)  +  Site  transport  costs  +  Data/Air4me  

$19  

Project  Managers  Op  Costs   1  per  500  CHWs,  $4,400  per  year  +  Site  transport  costs  +  Data/Air4me  

$13  

Office  Op  Costs   Office  Internet   $1  

Yearly  Equipment  Costs  (Avg.    over  3  year  life4me)  

Mobile  Phones  /  chargers  /  SIM  card  &  registra4on  for  CHWs,  GPRS  Modem  &  Netbook  for  Office  

$37  

Dimagi  User  Fees   $.75  per  CHW  per  month  (aTer  the  20th  user)     $9  

Total  cost  for  each  CHW  per  Year   $99  

Total Cost of Ownership (TCO) Model: Average yearly running costs by category for a 3-yr project

74

TCO Model

CHWs,    $9    

Project  Managers,    $13    

Field  Staff,    $19    

Avg.  Yearly  Training  Costs,    $11    

Office,    $1    

Avg.  Yearly  Capital  /  Equipment,    $37    

CommCare  per  CHW  User  Fee,    $9    

Annual  Total  CommCare  Cost  per  CHW    (Modeled)  

75

RESEARCH

76

Technology      PlaUorm  

Services    &  Adop4on  

Innova4on  &    Research  

Smarter Community Services

Measure

77

Research and Innovation at Dimagi

§ Team: PhDs, field RAs, data analysts, external research partners

§ Example projects/studies Ø SMS-based reminders for visits Ø Performance feedback graphs Ø Anomalous data detection Ø Cost effectiveness models Ø Evaluations of CommCare’s impact on

visit quality, who attends sessions, CHW knowledge tec.

§  Strong focus on publishing results

78

Active CC Investigations at a Glance Hypothesis   Experiment    

ACCE

SS  

•   CommCare  increased  number  of  family  members  who  par4cipate  during  a  home  visit/counseling  session    

•   Comparing  number  of  home  visits  and  self-­‐reported  number  of  family  members  present  during  counseling  sessions  for  ASHAs  using  CommCare  

QUAL

ITY  

•   CommCare  improves  mo4va4on  of  CHWs   •     Studying  mo4va4on  levels  of  CHWs  in  control  site  vs.  CommCare  interven4on  sites  using  job  sa4sfac4on  survey  

•     CommCare  improves  knowledge  of  CHWs  and  reinforces  training  concepts  

•   Studying  pre/post  knowledge  assessments  

•   CommCare  increases  client  reten4on  of  informa4on   •   Test  knowledge  improvement  amongst  beneficiaries  in  CommCare  interven4on  and  control  sites  

EXPE

RIEN

CE   •   CommCare  allowed  CHWs  to  discuss  sensi4ve  or  taboo  

subjects.    • CommCare  allowed  CHWs  to  work  around  cultural  and  social  barriers  (use  of  mul4media)  

•   Studying  paPerns  of  mul4media  usage  in  home  visits  where  CommCare  was  used  

ACCO

UNT-­‐

ABILITY  

•   Regular  feedback  to  CHWs  improves  performance  metrics  

•   Studying  2-­‐3  performance  indicators  over  3  months  for  60  CHWs  in  Madhya  Pradesh,  who  received  weekly  performance  feedback  via  Call  Center  

* Two Large Evaluations Planned for 2012:

+ Randomized controlled study in Bihar. Partnering with CARE and Mathematica Policy Research to assess the overall health impact of CommCare as a package intervention in the delivery of maternal and newborn health services in Bihar.

+ Factorial randomized controlled study. Also partnering with the government of Kaushambi in Uttar Pradesh, Catholic Relief Services (CRS), Harvard Business School, and University of Washington to more deeply probe how CommCare influences CHW behavior and client outcomes.

79

CHW Performance Feedback

§  If CHWs know their relative performance, does that encourage them to complete more of their visits?

§ Background: Ø There are a number of studies about relative vs absolute

performance – with variable outcomes Ø We want to understand how we can provide actionable

feedback to the CHW

80

CHW Performance Feedback: Low Performer

81

CHW Performance Feedback: High Performer

82

PERFORMANCE

83

CHW

CLINIC/ SUPERVISOR

CLIENT

CommCareHQ SUPERVISOR/DISTRICT/PROVINCE

PROGRAM MANAGERS

SMS - Alerts - Activity reports - Health stats - Coordination

GPRS - Form Submission - Work Management - Activity reports

Data Collection Supervisor Reports Feedback Quality Improvement

CommCare Data Flow

84

Active Data Management Continuous Performance Improvement

-  Support for Project Coordinators in managing workforce

-  Analyze all data submitted to server to improve worker performance

-  Targeted, actionable follow-up activities for supervisors

-  Weekly and monthly email reports

85

ADM Value Proposition

Today’s Challenges & Pain Points

Too much data: Large data tables, aggregate data

Difficult data analysis: Technical and time consuming

Too time consuming: Limited dedicated resources

Limited visit monitoring : Too short or informal

Non-actionable data: Reports often delayed, difficult to act timely

Lack of tracking action: Action rarely tracked to completion

86

ADM Value Proposition

Today’s Challenges & Pain Points ADM Value Proposition

Too much data: Large data tables, aggregate data

Focus on concise and simply formatted data

Difficult data analysis: Technical and time consuming

Focus on simple metrics, provide written statements of performance analysis

Too time consuming: Limited dedicated resources

Leverage CommCare platform and work towards building automated reports

Limited visit monitoring : Too short or informal

Provides real-time monitoring and benchmarking of CHW activities

Non-actionable data: Reports often delayed, difficult to act timely

Provide timely reports with list of clear follow-up actions by leveraging real-time data collection

Lack of tracking action: Action rarely tracked to completion

Create a tracking system to track action from identification to completion.

87

CommCare: Proof of Concept Model + Learning Collaborative Case Studies

88

PROOF OF CONCEPT (POC) OVERVIEW

89 89

POC: New Approach to Starting Projects

90

POC #1: USAID Development Innovation Ventures

~1000 CHWs 14 states

16 projects

•  Two years of funding- emphasis on making it easier to start a project

•  Planning and sponsoring mobile technology pilots for 40 organizations in India, which includes:

•  Giving 10 free Nokia/6 free android phones

•  2 weeks on-site field support •  Ongoing remote support •  Free hosting •  Discounted packages to help

reach scale •  Currently in second round of

launches, lots of lessons learned: •  Plan for scale! •  Improve HQ function in poor

connectivity environments •  Focus on knowledge

management •  Be firm with milestones

Development and Innovation Ventures (DIV II) grant from USAID to expand and scale to new programs in health and other social sectors: Proof of Concept (POC)

91

POC #2: CORE Group Learning Collaborative

§ Supports 12 organizations to create mHealth applications around maternal and child health in 14 low-income countries

§ Started in April 2012, wrapping up this month

§ Learning Collaborative members receive:

Ø Ten free mobile phones

Ø $500 for airtime

Ø A year of remote,

technical support from Dimagi

92

POC #2: CORE Group Learning Collaborative

93

POC CASE STUDIES

94

Learning Collaborative Project #1

Project Name: Strengthening and Accessing Livelihood Opportunities for Household Impact (SALOHI) mHealth Nutrition Program

Organization: Adventist Development Relief Agency (ADRA)

Location: Amoron’i Mania region, Madagascar

Focus Area: Malnutrition for children under five

Language: Malagasy

95

ADRA’s SALOHI mHealth Nutrition Program Application Purpose: § To improve data quality of Growth Monitoring and

Promotion (GMP) § Support health volunteers in providing counseling

messages Application Details: § Tracks children’s monthly nutritional

anthropometric status § Calculates weight-for-age Z score § Record MUACs § Provides appropriate nutritional counseling

messages

96

Getting Started with mHealth 1. Designing the Prototype: describe the data points Descrip)on   Type  (Number,  Yes/no,  Single-­‐

select,  Mul)-­‐Select,  Date,  Free  text)  When  data  will  be  collected  

Full  name   Free  text   Child  registra4on  Date  of  Birth   Number   Child  registra4on  Gender   Single-­‐select  (Male/Female)   Child  registra4on  Current  Weight     Number  (decimal)   Child  registra4on  District/Village   Drop  down  list   Child  registra4on  Name  of  Mother   Free  text   Child  registra4on  Exclusive  breas_eeding  (for  infants  0-­‐6  months(  

Yes/No   Monthly  Anthropometrics  

Weight  for  age   Number  (kilograms)   Monthly  Anthropometrics  MUAC  (6  months  and  older)     Number  (cen4meters)  

Measurements  are  recorded  to  nearest  0.5  mm  

Monthly  Anthropometrics    

Illness  •  Fever  •  Diarrhea  

Yes/No   Monthly  Anthropometrics  

Recovered  from  Illness   Yes/No   Monthly  Anthropometrics  END  OF  REGISTRATION  FORM  

97

Question Tree 2. Designed Prototype and Data Outputs

1. Calculations of weight-for-age of each child by location

2.  Number of children 0-59 months of age who participate in growth monitoring promotion

3. Number of children who are Sam, Mam, and normal

98

Counseling Messages 3. Counseling messages: CHV can have complete control over which messages/audio are displayed or can be automated depending on conditions.

99

Designing the Counseling Messages

Example 1 1.   Text that should appear on the phone: Frequency of

Breastfeeding 2.   (Optional) Image that will accompany text: Breastfeed

your baby on demand, at least 10 times day and night, to produce enough milk and provide your baby enough food to grow healthy.

3.   Image file that can be played by user:

4.   When message will be given: 0-6 months of age

100

Designing the Counseling Messages

Example 2 1.   Text that should appear on the phone: Nutritional care

of infants and children with moderate acute malnutrition 2.   (Optional) Image that will accompany text: (for children

six months or older). In addition to the supplementary good that the child receives, give 1 additional bowl of food each day to help your child to recover quickly and become strong and healthy again. Breastfeed more often

3.   Image file that can be played by user:

4.   When message will be given: For children in the -2 zone (yellow zone)

101

Timeline of implementation April 22 – 30 1.  Download the final version of the application on the phone. 2.  Insert SIM cards and track which phone is given to each CHV 3.  Meet with the CHVs to complete training 4.  Distribute the phones to each CHV along with credit for Internet,

money for charging and the phone charger. 5.  Train the CHVs on the application including how images/phones be

used in counseling sessions May-July 1.  Conduct field visits with the CHVs during their SPCE session to

observe their use of the phone and answer any questions (visit each CHV twice).

2.  Track reports on CommCare HQ website to ensure that the phones are being used properly.

102

1st Round of updates to the application 1.  There should be a way to "close" the case.  This means

that right now the registration form opens a "file" for the child, and the monthly form updates details about that child.  Most applications have a "close" form that simply asks why the name is being removed (for example- because the child is too old, child moved out of the area, child died, etc.).

103

Timeline of implementation continued August 1.  Distribute evaluations (translated to Malagasy) to each

CHV regarding their involvement in the pilot program. 2.  Collect all phones and any extra charging money or

credit. 3.  Troubleshoot (i.e. if a CHV has a problem with their

phone who do they tell and how does it get fixed?) 4.  Collect feedback on how to improve the tool as well-

what was hard to use/what do the CHVs wish was there

104

Things to consider

§  Audio messages: what is typical is to have a short text phrase—sometimes just a word or two, and then a longer audio message. 

     §  Calculations for Z-Scores:  CommCare can handle math well

-- so if you are just using an equation to get the percentile, that should work very easily.  If you are looking up the answer in a table (e.g. given a child's age and height, where do they fall on this percentile chart), that is much harder to set up, but possible.

§ Multimedia:  you can upload your multimedia to our cloud through our website, CommCareHQ. You can grant people access to download the multimedia files from the website.  From there, you will have to place the multimedia files on the phone manually.

105

Things to consider cont.

§  Purchasing Phones: purchase phones locally since sometimes phones are lost/damaged and need to be replaced.   Local phones display in the local language). Nokia C-2 for most deployments, and the Samsung Galaxy Y or Samsung Galaxy Mini if you're looking to use Android phones.     

§  Word Limits:  Welcome to the land of long words! The text would be displayed on the phone, if it was too long we used shorthand.

§  Common problems in the field: Accidentally deleting things. Could also be network issues, which is separate from the application itself. If there is no network, message will be saved on the phone until network can be found. If they have unsent forms, worker can try to manually send them or wait until network pops up.

§  What happens to the phones once the program is complete? Do

the CHVs keep them?

106

Learning Collaborative

§  Sharing the reports with the mHealth Learning Collaborative: It's a good opportunity to introduce yourself to other Learning Collaborative members and to ask and answer questions.     

§ Exchange visit with Food for the Hungry, Mozambique: FH has a similar application which is to improve nutritional surveillance of children under 5 years of age within Palma district by allowing.

1.  Real time reporting of Z scores to mothers (with follow-up action, including referrals);

2.  Real-time and accurate aggregate reporting of nutrition status to FH, District, and CDC

107

Learning Collaborative Project #2

Project Name: Reducing Maternal Mortality in Panchagarh District

Organization: World Renew

Location: Panchagarh District, Bangladesh

Focus Area: Maternal Health

Language: English

108

World Renew’s Maternal Health Application

Application Purpose: § To track and provide counseling

messages to pregnant women in antenatal, birth preparation, and post-partum stages

Application Details: § Tracks and collects data about

pregnant women and babies § Provides appropriate antenatal,

birth preparation, and post-partum counseling

109

Breakout Session

110

Benefits  of  CommCare   Barriers  to  Implemen)ng  CommCare  

Improved care and quality of service delivered by FLWs

Lack of on-the-ground infrastructure (network and electricity availability)

Improved  experience/access  to  care  for  FLW  clients  

Government/health system barriers

Increased productivity/performance of the FLWs

Lack of program staff capacity

Improved supervision of FLWs   Lack of technology/computer programmer staff capacity

Increased efficiency and timeliness of reporting from the field

Lack of mHealth budgeting/finance capacity

M&E of program interventions and data analysis

Lack of interest/understanding/commitment from organizational leadership

Other   Lack  of  donor  interest/funding  

Cost of deploying technology

Inability to scale after pilot

Other

111

Breakout Session For the final part of this workshop, we will be discussing the following:

1)  What are some benefits that CommCare (or mHealth in general) could bring to your organization’s projects?

2)  What are some barriers you would foresee in implementing an mHealth solution like CommCare?

3)  Having discussed the benefits and barriers to implementing mHealth, develop some guidelines that bring up points a manager thinking about employing mHealth should consider.

112

Thank you!

Contact: [email protected] or [email protected]

Additional Videos:

CommCare Overview Video: http://youtu.be/ZpfvISKxylE

CommCare Demo Video with multi-lingual support from India: http://youtu.be/30Ftk6STM3U

Recorded Webex of CommCare Presentation given to NetHope: http://bit.ly/tiLaYy

Additional Resources:

http://groups.google.com/group/ict4chw

http://www.commcarehq.org

http://www.dimagi.com