Teaching Clinical Effectiveness

Teaching Clinical EffectivenessRobert Baldor, M.D.

Frank J. Domino, M.D.Family Medicine and Community Health

University of Massachusetts Medical SchoolWorcester, MA

Clinical Effectiveness…

…. is the extent to which specific clinical interventions do what they are intended to do, i.e. maintain and improve the health of patients, securing the greatest possible health gain from the available resources.

B

To do….

• the right thing• at the right time• in the right place• with the right result

B

Our Goal

To appreciate the shift to ‘Comparative Effectiveness’

as it relates to teaching ‘Clinical Effectiveness’.

B

By the end of this session, you will..• Gain an appreciation for the recent emphasis on

‘Comparative Effectiveness‘ • Understand the basic concepts of comparative

effectiveness and the relationship to our current understanding of evidenced based medicine

• Discover where to find on-line resources to assist with teaching

• Discuss methods for incorporating comparative effectiveness into the curriculum for your students and residents

B

The more things change, the more they stay the same…

• HMO’s• Managed care• Capitation• Evidenced-based

medicine

• Medical Homes• Care management• Global fees• Comparative

effectiveness?

B

Soaring health care costs….

• $2.2 trillion – 16% of GDP in 2007• Without any changes in federal law, it will rise

to 25 % of GDP in 2025.

B

The American Recovery and Reinvestment Act (ARRA) of 2009

• Growing concern that we have little solid evidence of the value of many treatments

• Hope to save money by discouraging the use of costly, ineffective treatments

B

Comparative Effectiveness Research

• Congress tasked the IOM to recommend priorities for research to be addressed by CER and supported by ARRA funds

B

Perspective research…• Is it better to treat neck pain with surgery or PT & meds?

• What is the best combination of counseling & meds to

treat depression?• How do drugs compare with surgery to treat intermittent

claudication? • Is it better to treat CHF by meds alone or by including

home monitoring of BP and weight?

B

A Teaching Perspective

• Your MS III has just seen a patient and believes they have new onset depression. “I think he is pretty depressed and needs medication. On my Psychiatry rotation last month, citalopram was the SSRI of choice; they thought it worked better.”

• What would you use for an SSRI and why?

B

CER is done elsewhere…

• Britian, France and others have official bodies that assess health technologies and compare the effectiveness, and sometimes the cost, of different treatments.

B

The CER legislation became a lightning rod …

• Republicans complained that the government would intrude in a person’s health by enforcing clinical guidelines.

• Pharmaceutical and medical-device lobbyists fear the findings will be used to deny coverage for more expensive treatments and, thus, to ration care.

B

Rationing ???

• Agreement that researchers should compare the clinical merits of different treatments. – Whether they should also consider cost is hotly debated.

• The House Appropriations Committee inadvertently stoked rationing concerns in a report that stated

‘research comparing different treatments could yield significant payoffs because less effective, more expensive treatments will no longer be prescribed’

Sounds like you’re trying to kill Granny to me!

B

Then of course there is Rush…

• Bureaucrats “will monitor treatments to make sure your doctor is doing what the federal government deems appropriate and cost-effective”.

What he really meant was to keep your hands off my drugs!

B

The plan

• The FDA regulates drugs and devices, but the goal is to establish if a treatment is safe and effective, not if it was better than the alternatives. The plan is for – systematic reviews of published scientific studies– clinical trials of head-to-head comparisons

F

IOM definition

• CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.

How is this different from EBM?

F

Your MS III has just seen a patient…

...and believes they have new onset depression“I think he is pretty depressed and needs medication. On my Psychiatry rotation last month, citalopram was the SSRI of choice; they thought it worked better.”

• What would you use for an SSRI and why?

F

How do you teach “what works better? Where would YOU look to find this data?

• Pubmed/Clinical Queries?

• In Cochrane?

• In UpToDate?

F

Will CER help us to answer those questions?

B

IOM prioritization – a political process

• Consulting stakeholders…• A public session - consumers, advocacy groups,

provider groups, insurers, manufacturers, and academia.

• A web-based questionnaire received more than 2,600 topics.

B

The selection process:How an Idea becomes a CER

B

No

No

Yes

Yes

Anybody in this room provide input?

• What’s on your top 10 list? – For your learners?Is this different than your list – as practitioners?– as patients?

F

Research methodology

• RCTs felt to be the most appropriate methodology for ½ of the topics

• Prospective observational studies, database reviews, and systematic literature reviews for the rest

F

EBM Informational Pyramid

Systematic Review Evidence Based Review Randomized Clinical Trial Cohort Study Prospective Case Control Study RetrospectiveTextbook (Harrison, Uptodate, etc.)

F

CER USE many forms….

1. Systematic literature reviews to summarize evidence, identify gaps, and generate new ideas for research

2. Randomized controlled trials - considered the gold standard of evidence, they also have shortcomings

3. Observational research using prospective registries and large established databases, including EHRs, are used for cohort studies to understand the natural progression of disease and the factors that influence clinical outcomes.

F

The Forest Plot

Estimate and confidence interval for each study

Estimate and confidence for the meta-analysis

Direction of effect

Scale (effect measure)

Line of no effect

Estimates with 95% confidence intervals

0.2 1.0 5

Favours LR Favours control

Risk ratio

Kennedy 1997

Locke 1952A

Lopes 1997

Reynolds 1998

Seiberth 1994

Forest Plot: If <> to your Left, Intervention was Effective at Lowering Risk of Outcome.

F

FOREST PLOT

FMeta Analysis: Vitamin D on Falls 400-800 IU per day

100 topics, ranked by quartile…..

• Collectively address broad societal needs. • Conditions with the greatest aggregate effect

on the health of the U.S. population• Less common conditions that severely affect

individuals in vulnerable subgroups of the population. (24 of the 100 topics)

B

1st Quartile medical examples……• Medical Treatments– low back – ADHD treatment strategies– a-fib (surgery v catheter ablation v drugs)– ACEi/ARBs for CAD

• Medical Diagnostics– Use of PET, MRI, CT for cancer diagnosing, staging – Use of genetic and biomarker testing

B

1st Quartile prevention home, community, delivery

• Unintended pregnancies • Obesity, hypertension, diabetes in at-risk populations • Primary prevention (exercise and balance training) v clinical

treatments in preventing falls • Management strategies (e.g., pharmacologic treatment,

social/family support) for dementia in home and institutional settings.

• Compare the medical home v. usual care in managing those with chronic disease.

B

4th quartile examples….

• Compare the effectiveness of:– long-term treatments for acne.– diagnostic imaging performed by non-radiologists

and radiologists.

B

Back to our student…

• Your MS III has just seen a patient and believes they have new onset depression. “I think he is pretty depressed and needs medication. On my Psychiatry rotation last month, citalopram was the SSRI of choice; they thought it worked better.”

• What would you use for an SSRI and why?• How do you teach “what works better”???

F

Lets answer that question together

• What resources would your learners use…

• What resources would YOU use?

• What kind of a study is found (ie RCT, SR?)

• So what’s the answer?

F

OK lets take a break

• When we come back – you’ll pick a clinical question related to clinical effectiveness and search for an answer

Appreciating EBM & CER

1. Pick a clinical question; either one you have on your mind or one from this list:

• Treatment for Localized Prostate Cancer• Best Oral Rx for Type II DM• Choosing An Anti Depressant• ACEi or ARB for Hypertension• Best Pain Control in Osteoarthritis• Fracture Prevention in Osteoporosis• Best Management of Gestational Diabetes

Teaching Method for use of CER and EBM to young (and old) Learners

• Pick a topic of their interest• Search using conventional methods of

information acquisition• Find a high level of evidence Systematic

Review on the topic• Look for a Comparative Effectiveness Review

on the topic

Appreciating EBM & CER

1. Read a summary or review article on your topic (AFP, eMedicine, etc.)

2. Find a Randomized Controlled Trial using PubMed Clinical Queries Filterswww.pubmed.gov use CQ*

3. Find a Systematic Review on your question4. Try to find a Comparative Effectiveness Review

(AHRQ.gov**)

Using Pubmed Clinical Querieswww.pubmed.gov

Pubmed Clinical Queries

Prostate cancer AND local

Appreciating EBM & CER

1. Read a summary or review article on your topic (AFP, eMedicine, etc.)

2. Find a Randomized Controlled Trial using PubMed Clinical Queries Filterswww.pubmed.gov use CQ*

3. Find a Systematic Review on your question4. Try to find a Comparative Effectiveness

Review (AHRQ.gov**)

Comparative Effectiveness Reviewswww.ahrq.gov

After our exercise…

• What’s the curriculum for our students and residents?

• What are the best methods to teach about clinical effectiveness?– Lecture? – On-line curriculum?– Workshops?– Others?