Megan Jones, Science vs Snake Oil: Evidence-based Mental Health Technology

Science vs. Snake OilEvidence-based mental health technology

Megan Jones

@golantern golantern.com

@jmegjones

Hi.

Who am I?A lucky, privileged person who experienced mental illness and recovered.

A family member and friend of individuals who live with mental illness.

A concerned and engaged community member.

A committed advocate, researcher, therapist.

Conflict of Interest Statement

I, Megan Jones, PsyD, am an employee and shareholder of Lantern (Thrive Network, Inc.). Lantern is an online program designed to help individuals improve emotional wellbeing.

My bias.

A word about therapy.Therapy is critically important and essential.

It helped me save my life.

450M Adults suffer from a

mental disorder each year worldwide - World Health Organization

1 in 10 people get the therapeutic help they need

1. STIGMA2. ACCESS3. COST

Barriers to Mental Health Care

1. STIGMA2. ACCESS3. COST4. Effectiveness

Barriers to Mental Health Care

U.S. adult population

Distress Level

Self-HelpModerate Stress & Anxiety:

Not Targeted TodayTherapyPharma

Today’s Mental Health Market

Source: Marketdata “Overview of U.S. Self-Improvement Market”; IMS Health, “Medicine Use and Shifting Costs of Healthcare: A Review of the Use of Medicines in the United States in 2013”; SAMSHA; Note: Therapy data includes SAMSHA-reported mental health spend to physicians, other professionals (e.g., social workers), and specialty mental health organizations, and excludes hospital and specialty in-patient hospital spend

The need to reboot.

Innovation is needed to fill the gap between self-help and therapy.

How Can Technology Help?

85% mobile access

There are endless possibilities when it comes to marrying mobile and healthcare.

Integrating technology & therapy could…

• Enable better performance assessment through timely access to precise data

• Inform and facilitate targeted policy and initiatives (understand and document the practical needs and evidence)

• Optimize expenditures to reduce redundancy, promote self-management and ensure “right help at the right time”

Source: Young and Well Cooperative Research Center (October 2014). Innovative technologies in e-mental health: Briefing paper for the National Mental Health Commission.

Beyond a complement to face-to-face therapy

• Reduce complexity

• Remove inefficiency

• Promote accountability and measure effectiveness

• Provide consumer choice

• Reduce disparities in access to care

Source: Young and Well Cooperative Research Center (October 2014). Innovative technologies in e-mental health: Briefing paper for the National Mental Health Commission.

Pivot toward user-centered system• User-centered design places the individual at the center of a

system of care

• Technology that is “one size fits all” does not resolve key issues of real-world efficacy and disparity in access

• Opportunity: allow users to control their data, peer- and family-support, enhanced self-management, customizable treatment pathways

Evidence for behavioral health technology

Online interventions

• Evidence for online prevention, self-help, treatment for wide range of mental health problems

• Effect sizes comparable to face-to-face treatment (med-large)

• Evidence-based programs for depression, anxiety, substance use, eating disorders, adjustment disorders, resilience

• Increasingly lower barrier to development and market intro

• Very few high-caliber studies, early days of evidence-base

• Clear guidelines are needed for development and reporting

Self-management apps & resources

Opportunities• Self-monitoring is key to many evidence-based therapies and

well facilitated by technology

• Many evidence-based treatments are amenable to digital delivery: peer support, psychoeducation, talk therapy, CBT

• Meet people where they are - many individuals already use social media and online tools to help manage conditions and seek support

• Allow flexibility, self-management, choice

Categories of technology• Self-monitoring or goal setting, no social or support features

• Information

• Social and support features (connect and share)

• Connected to larger healthcare or support ecosystem

Technology Use Case Examples

www.nationaleatingdisorders.org

Screening• Empowering and quick

way to receive feedback

• Should not provide diagnosis

Assessment• Validated measures

• Adaptive design to reduce response burden

• Feedback designed to educate and empower user

• Link to appropriate level of care and resource

Medication Monitoring

• Medication scheduling

• Alerts for interactions

• Information about side effects

• Game-like incentives to establish and reach goals

Leveraging Data• Patient-powered online

community

• Linked to research that helps generate actionable, real-world data about the nature of disease

Reminders• Just-in-time in vivo interventions

• Ecological momentary assessment

• Psychoeducational reminders

• Scheduled reminders

Personalized self-management

1:1 Coaching Daily Sessions Relaxation Techniques

Tele-mental health• HIPAA-compliant

platforms linking licensed providers with clients in their states

• Research supports efficacy of treatments delivered via videoconference

Social, Sharing, Support

• Peer-to-peer support

• Real-time access to supportive community

• Empowering

Crisis Intervention

• Easy, quick access to crisis support

• User-centered product design

Training & Education! Online, adaptive learning

! Webinars and eTraining institutes for psychotherapy and telemental health

! Training non-professionals to deliver help & support

Progress has been made. Key challenges remain.

Challenges - Research • Research funding has been piecemeal and lacking

coordinated vision and strategy.

• This leads to unacceptable levels of redundancy and uncoordinated development, duplication and waste.

• Dissemination of narrowly focused products does not extend evidence-base nor allow interoperability to improve overall system and coordination of care.

Challenges - Risks and side effects

• Need to understand and meet the needs of a vulnerable population

• Consider implications of reliance on the technology and potential for user abandonment

• Potential for symptom severity increase or development of new symptoms for which technology is not suitable

Stigma

• Potential that technology is only source of help/support.

• How is privacy maintained?

• How does the product/service marketing impact stigma?

• Does the product empower users?

Equality of access

• Digital divide is shrinking but still exists

• Some individuals may not be able to afford even low-cost technology-based services

• Few behavioral health technologies are reimbursable by insurance

Challenges - Piggybacking vs. novelty

• Competitive environment fuels differentiation based on branding rather than novelty and true difference

• Nice branding and design can disguise absence of credibility and evidence-base

“Disruption” is great, but where do we draw the line?

Lack of Evidence• “Many health apps are based on flimsy

science at best, and they often do not work” - By Rochelle Sharpe, New England Center for Investigative Reporting, Washington Post, November 12, 2012

• “FDA regulation can’t keep pace with new mobile health apps”- By Joshua Barajas, PBS.org, July 7, 2014

The need for ETHICS & EVALUATION

in behavioral health technology

Key questions• Which behavioral health technologies should organizations

recommend or endorse?

• How should technologies be evaluated?

• How should concerns over safety, privacy, and effectiveness be addressed?

• What evidence is required? How to differentiate effective or evidence-based technology from thousands of options?

The goal should be:• Flexible and helpful standards and evaluation framework

• Articulation of industry best practices

• Raise standards to improve market “pull” for technologies that are evidence-based, address safety, privacy, and security, and user-centered

• Increase adoption of effective technology in practice

Evaluation & use case match

• Behavioral health technologies vary in aims:

• Clinical benefit

• Tool

• Need to spend research time and funding wisely

• Not all technologies need to be held to the same standard

Factors that always apply• Safety

• Privacy

• Security

• Ethics and legality

• Truth in advertising

Evaluation & regulatory frameworks

1. Safety, security privacy: What data is collected, ownership, what is done with it?

2. Source: Who developed it, who profits, who was involved?

3. Access: How to get it, cost, devices needed?

4. Relevance, content, accuracy: Need or priority addressed? Based on good practice or change theory?

5. Impact: Who benefits? Measurement? Adapted from the National Institute for Health Research (NHS) MindTech Healthcare Technology

Key questions

Product - user fit

• Which symptoms is the technology appropriate for?

• What level and range of symptom severity is a fit?

• Which users?

• For whom is the technology not a fit?

• User value: Traction (audience base, user acquisition, retention, LTV); Meaningful engagement with product

• Intended value: Is product achieving intended purpose and demonstrating place in mental health system

• Social value: Effectiveness (mental wellbeing outcomes)

Product - value

FDA: Mobile Medical App

• Used as an accessory to a regulated medical device; or

• To transform a mobile platform into a regulated medical device

• Intended use is for diagnosis of disease, mitigation, treatment or prevention of disease it is a medical device.

FDA: Regulated Mobile Medical App• Mobile apps that extend other medical devices through patient

monitoring or analyzing medical device data (e.g., app that controls delivery of insulin)

• Transform mobile platform into medical device (e.g., blood glucose strip reader)

• Patient-specific analysis, diagnosis, or treatment recommendations

FDA: Apps - Enforcement Discretion

• Provide supplemental clinical care by coaching or prompting, to help patients manage care in daily environment (e.g., apps that coach patients with adherence to medication)

• Provide simple tools to organize and track information (e.g., patients with anorexia to track eating behavior)

• Provide easy access to information (e.g., use diagnosis to provide clinician with best practice treatment guidelines or look-up drug interactions)

Examples: • Help patients document, show, and communicate to providers

(e.g., videoconferencing tools to enhance communication)• Perform simple calculations routinely used in clinical practice

(e.g., Body mass index)• Enable individuals to interact with health records systems• Help patients with diagnosed psychiatric conditions (e.g.,

depression) maintain behavioral coping skills by providing behavioral technique or audio messaging

• Motivational guidance to patients recovering from addiction• Use behavioral risk factors to provide personalized screening,

counseling, or preventative recommendations

Intended use (FDA definition)

• Shown by labeling claims, advertising materials, or oral or written statements by manufacturers or their representative.

• When intended use of mobile app is for diagnosis of disease or other conditions, or the cure, mitigation, treatment or prevention of a disease, or is intended to affect the structure or any function of the body of man, the mobile app is a device.

Ethics - truth in advertisingAmerican Psychological Association Ethics Code: “Psychologists do not make false, deceptive or fraudulent statements concerning (1) their training, experience or competence; (2) their academic degrees; (3) their credentials; (4) their institutional or association affiliations; (5) their services; (6) the scientific or clinical basis for or results or degree of success of, their services; (7) their fees; or (8) their publications or research findings.”

Contextualizing the product/service

• APA Ethics Code: 8.11 Plagiarism

• “Psychologists do not present portions of another's work or data as their own, even if the other work or data source is cited occasionally.”

• What does this mean for how you describe the research-base of a product?

Presenting outcomes and research

• How does the cited research relate to the actual product?

• How is data represented? Is a sample size provided?

• Controlled vs. uncontrolled comparisons

• Consent of participants/users? Human subjects approval (if presented/published)

Problems:

Scale is misleading (better if: 50% or 100%)

No sample size (better if: N=##)

Clients implies potential treatment relationship (better if: “user” or “participant”)

Problem:

Critics have raised concern about recommending text-only “therapy” as a treatment without evidence.

No mention of actual studies or links to abstracts.

How does this impact stigma and informed consent?

Accuracy in endorsements

• Is there reciprocity in endorsements across organizations?

• Are the endorsements from an institution or individual without conflict of interest?

• Are endorsements from an organization that has a thorough and appropriate review process for behavioral health tech?

Technology in clinical practicePractitioners who integrate technology into practice or deliver therapeutic

services via technology should adhere to minimum standards and practices to work in an ethical manner

• Encryption of records and communication• Backup systems for records• Complete password protection for hardware, files, software,

etc. and use firewalls• Protect computers and data on computers via virus protection• Knowledge of hardware and software• Trust and careful selection of third-party services

Basics of using technology in practice

Source: Anthony, K. & Nagel, D.M. (2009). Therapy online: A practical guide. Sage Publishing: London.

Scope of practice guidelines

• Indicates the specific area of practice• May include geographic location of practice, ability to practice

across geographical boundaries and the parameters of practice

• Boundaries and limitations of discipline (e.g., psychological diagnosis and provision of treatment)

• Laws and ethics of discipline and geographic region

Source: Anthony, K. & Nagel, D.M. (2009). Therapy online: A practical guide. Sage Publishing: London.

Training, knowledge, supervision

• Core to “best practice”

• Formal and informal training, current literature

• Clinical and peer supervision and/or consultation

Source: Anthony, K. & Nagel, D.M. (2009). Therapy online: A practical guide. Sage Publishing: London.

Key competency areas

• Online therapy• Online clinical and peer supervision• Text-based therapy• Telehealth• … and many many more!

Source: Anthony, K. & Nagel, D.M. (2009). Therapy online: A practical guide. Sage Publishing: London.

Guidelines for websites

• Crisis intervention information (e.g., Befrienders)• Practitioner contact information• Practitioner education, license and certification information• Terms of use and privacy policy• Encrypted transmission of therapeutic and payment

information

Source: Anthony, K. & Nagel, D.M. (2009). Therapy online: A practical guide. Sage Publishing: London.

Informed consent in online therapy• Advantages and disadvantages of online therapy• Explanation of encryption and for what data it is used• Therapist as owner of the record• File storage procedures• Privacy policy (Notice of Privacy Practices and HIPAA)• Practitioner’s geographic jurisdiction• What to do in context of technology interruption/breakdown• Emergency contact• Cultural and location specifics that may impact services• Dual relationships• Insurance and payment information

Source: Anthony, K. & Nagel, D.M. (2009). Therapy online: A practical guide. Sage Publishing: London.

Best practices for behavioral health tech?• Best practices have been developed for face-to-face

treatments and services, but often not for technology-facilitated care.

• How should existing best practices be applied to behavioral health technology?

Conclusions• Before it’s too late, digital health companies need to define their

responsibility and obligation to their users and their employees.

• While other technology companies can rally around mottos like “fail fast, fail often” and say they're “disrupting” the market, in healthcare, failing too often (or at all) can directly and seriously impact a user’s life.

• And breaking the rules can cost some employees – like health services providers – their careers.

Recommendations• Create products that are for everyone, including the toughest

use cases

• Objectively measure outcomes (including side effects)

• Pro-actively apply best-practices and ethical/legal standards

• Follow scientific and clinical standards for informed consent, privacy, and data representation and reporting

Science vs. Snake OilEvidence-based mental health technology

Megan Jones

@golantern golantern.com

@jmegjones