Patricia Flatley Brennan, RN, PhD, FAAN University of Wisconsin- Madison

1

Linking lay people to the professional literature

An application of natural language processing to

free-text e-mail

Patricia Flatley Brennan, RN, PhD, FAAN University of Wisconsin- Madison

Supported by Grants from the National Library of Medicine (LM 6249); Intel Corporation (Advanced Technologies for

Health@Home), and Wisconsin Alumni Research Foundation (The Kellet

Professorship )

2

Plan for the talk• Provide an update of the final results

of the HeartCare randomized field experiment

• Apply NLP tools to decode patient messages

• Describe current work in two areas:– Community capacity building– Infrastructure-building

3

The HeartCare Team• Investigators

– University of Wisconsin-Madison• Patti Brennan• Barrett Caldwell (Now Purdue)• Mary Ellen Murray• Dave Gustafson

– Case Western Reserve University• Shirley Moore • Sree Sreenath

– Cleveland Clinic Foundation• Ralph O’Brien

• Undergraduate, Graduate, and Post-Doctoral trainees

4

Meeting the Challenges of CABG

Recovery• Monitor, Manage, Mend, Motivate

• Demands in the discharge encounter

• Patient-centered, tailored information

5

HeartCare

Recovery requires communication andtailored health information

• Peer and professional communication• Information sequenced over time and

tailored to the patient’s needs– Weeks 1-2: Symptom Management– Week 3-6: Resume physical activity– Weeks 6-12: Return to prior function– Weeks 12-26: Adopt healthy behaviors

6

HeartCare Evaluation • Randomized Field Evaluation

– 6 Months experimental period– 140 adults recovering from CABG surgery

• Mean age: 63; 35% Female; 19% Non-majority– Outcome Measures

• Symptom Inventory, Sickness Impact Profile, POMS (Depression), Family function, Health Behavior change

• Three Groups– Usual Care – CHIP, An Audiotape Intervention – HeartCare: WWW-based recovery support

8

9

11

Does access to HeartCare improve

recovery from CABG?

Yes!

12

0

5

10

15

20

25

30

35

0 1 2 3 4 5 6

Sickness Impact Profile (SIP)

SIP

Months Since Surgery

CHIP

HeartCare

13

0

2

4

6

8

10

12

14

0 1 2 3 4 5 6

POMS Depressive Symptom ScaleD

ep

ress

ion

Months Since Surgery

CHIP

HeartCare

14

Summary• Participants in the HeartCare group

recovered faster, with fewer symptoms, than those using the CHIP intervention.

• Participants use HeartCare intensively during the early recovery phase.

• E-mail used more often than public forum

• Information reviewed on most encounter

15

What’s needed to make HeartCare-like interventions

Scalable?• Strategies to understand information

needs• Characterization of the ways lay people

organize health information• Sustainable knowledge management

approaches• Alignment of the CHI investments with

the community’s health information assets

• Robust health information infrastructure

16

What’s needed to make HeartCare-like interventions

Scalable?• Strategies to understand information

needs• Characterization of the ways lay people

organize health information• Sustainable knowledge management

approaches• Alignment of the CHI investments with

the community’s health information assets

• Robust health information infrastructure

17

Message to the NurseDear Connie, I've been out of the loop for a few weeks. I had a setback with the appearance of a blood clot 2 weeks ago and was back in the hospital for a week. I was released a week ago Friday and now am on several new medications. With all these new meds, I feel nauseous almost all the time and frequently dizzy. I have a visiting nurse coming to see me 3x a week, and she monitors my blood pressure, temperature and checks my legs for possible clots. But nothing seems to help the nauseous feeling and I have little appetite. The medication I am now taking are … I suspect the Lasix may be the culprit, since I had been on it a LONG time ago and it made me nauseous, but I don't know. Do I really need to be on all of these now? I take alot of them at the same time (meal time), but should I change this and stagger them? What order should I take them, or are there alternatives to this medication for now? Any advise you could give me before I go back to see my internist on Tuesday would be helpful, then I could discuss it with him again. I see the cardiologist on Thursday and hope to be cleared to start cardiac rehab after that. Right now, however, it is slow going and discouraging. Thanks, Bill

18

Can existing

UMLS lexical tools decode

patient information needs?

19

SPECIALIST lexicon

Dear Connie, I've been out of the loop for a few weeks. I had a

setback with the appearance of a blood clot 2 weeks ago and

was back in the hospital for a week. I was

released a week ago Friday and now am on

several new medications.

Mapped terms

that can launch

searches of

electronic resources

20

Background

• Federal initiatives to meet lay people’s information needs

• Most common stimulus: query phrases• But…

– Consumers’ don’t speak UMLS– Information need arise in colloquial conversations

• However,– The UMLS and its lexical tools exist-- exploitable?– Electronic resources applying machine-readable

indexing approaches

22

USUAL APPROACHES

• Human intermediary • Natural language interpretation

– Awakening from the dream stage• Terminological strategies

– Query terms– Indexing Initiative

23

Can NLP tools built to manage professional

vocabularies help patients?

• Source document– 241 messages sent from patients to nurses in

the HeartCare project• Pass thru Metamap

– Parses text of electronic bibliographic databases

– Strips capitalization, ignores word order– Assigns candidates from UMLS– Scores the adequacy of the concept match

24

Approach

• Stimulus text acquired– Sanitizing process – Human Subjects’ issues

• Preliminary Structuring– Demarked units– Title of message ---> Citation Title– Body of Message ---> Abstract

25

Preliminary Results

• 241 Messages (1976 Utterances)• 15566 Phrases• 11,373 Candidate UMLS Concepts

found– (mean 32.91 ; sd 42.7741)

– 9903 phrases had no candidates

• 7143 Mappings found (1.13; s.d.1.79)

26

Observations

• Diagnosis, symptom and findings recognized

• Health service elements not recognized (appointments, medication renewals)

• No tolerance for mis-spellings• Idioms choke the system• Full UMLS may be too rich

– Metamorph – Post-processing to remove selected

components

27

Preliminary Thoughts

• Promising but sparse; PARSING is key• Efficiency/interpretation tradeoff • Early work in a highly professional,

highly controlled stimulus had a 70% mapping

• Most messages deal with managing a health problem in the home ---> Nursing!

28

Vocabularies used in the TestThe Six Nursing Vocabularies

Nursing Plus:International Classification of Primary Care (ICPC2E)International Classification of Primary Care- Am English (ICPC2AE)Micromedex DRUGDEX (MMX01)National Drug Data File (NDFF01)Thesaurus of Psychological Terms (PSY2001)WHO Adverse Drug Reaction Terminology (WHO97)

NursingPLUS + Medical Subject Heading 2003 (MSH_2003)

NursingPLUS + SNOMED International Version 3.5 (SMNI98)

29

Vocabulary performance on a single message

NursingOnly

NursingPlus NursingPlus +MeSH

NursingPlus+SNOMED

CandidatesConcepts

15 54 85 114

MappedConcepts

13 42 57 70

Phrases w/ oneor more

maps

12 43 50 57

Mean concepts/phrase

1.08 .98 1.14 1.22

Errors 3 23 37 39

30

How did the vocabularies perform?

NursingOnly

NursingPlus NursingPlus +MeSH

NursingPlus +SNOMED

CandidatesConcepts

1016 3734 5786 7366

Mapped Concepts 948 3094 4439 5078

Phrases w/ one ormore maps

871 2863 3995 4383

Mean concepts/phrase

1.09 (0.28) 1.08 (0.30) 1.11 (0.35) 1.16 (0.38)

31

Mapping Adequacy

• Findings– True Positives– False Positives– Missing

• Trade-off of recall and precision• Zeng’s Model of Mapping:

– Lexical– Semantic– Mental Model

32

The Contexts of Care• Living Environments

– Homes– Communities

• Social Environments– Families– Cultural Groups

• Psychological Environments– Illness representations– Human Information Processing

• Technological Environments– Telecommunication– Consumer Electronics

• Health Service Environment– Clinical Care Practices– Financing & Delivery Institutions

33

What’s needed to make HeartCare-like interventions

Scalable?• Strategies to understand information

needs• Characterization of the ways lay people

organize health information• Sustainable knowledge management

approaches• Alignment of the CHI investments with

the community’s health information assets

• Robust health information infrastructure

34

The Dodge-JeffersonHealthier Communities

Partnership

35

Develop a model to generate design criteria

for health-related IT solutionsfrom an understanding of

citizen health information managment

behaviors and

community resources

36

49 Households in Central Wisconsin

• Housing type– 39 Single Family, 9

Apartments, 1 Mobile Home

• Most of those interviewed live alone

• Over half of the 1 & 2 person families had one person over age 65

• Electronics– Phones: 49– Cable: 42– Internet: 26

37

Health of the Household

• Respondents:– 7 Excellent; 12 Very Good; 10 Good; 3 Fair

• No one indicated Poor• Respondent’s assessment of household generally matched

• Health Concerns– Cancer, Cardiovascular disease, Hypertension, Arthritis

• Also: depression, memory problems, nutrition, wellness

• Income adequate• ? Health Insurance coverage• ? Health Care Provider

38

Where Do People get Health Information?

0 5 10 15 20 25 30 35

Physician Clinic/Hospital Public Health Nurse

Library News Health Magazines

Internet Hotlines Reference Books

Alt Med Sources Family/Friends School

Classes Other

Physician

Family

39

Appointments Contact Info Med Schedule Treatment

Medical History Insurance Labs X-Rays

Immunizations Birth/Death Records Self-Care Procedure Info

Self-Monitoring Provider info Literature Poison Control

Clinic/Hospital Info Other

Information Managed in the Home:

40

Information Management in the home

• Information types named by at least 20 respondents– Appointment & Contact

Information– Medication– Treatment– Birth/Death records

• Household experiences– Average 10.2 (sd. 3.3)

information types– Number and variety

unrelated to age of respondents or presence of children

Where do they put all of this information?

41

42

43

45

What’s needed to make HeartCare-like interventions

Scalable?1. Strategies to understand information

needs2. Characterization of the ways lay people

organize health information3. Sustainable knowledge management

approaches4. Alignment of the CHI investments with

the community’s health information assets

5. Robust health information infrastructure

Assessment of Community Health Information

Resources(ARCHIR)

PKI Approaches to

Secure E-Mail among

Health Professionals

Personal Consumer

Health Information Exchange(P-CHIE)

Community-Partnership Digitial

Library Project

Dodge-Jefferson Healthier Communities

Partnership

46

Community-Centered Information System

Consumer Health Information

Network

Clinic

Hospital

Public Library

Pharmacy

FurtiveRecords

Dentist

State Health Dept

clinician

patient

trainee

47

Web Site:healthsystems.engr.wisc.e

[email protected]