Evidence-Informed Public Health Decisions Made Easier: Take it one Step at a Time

Evidence-informed public health decisions made easier: Take it one step at a time

TOPHC, Toronto, ON

April 7, 2011

2

Objectives

1. To understand the process of evidence-informed

decision making in public health.

2. To clearly frame an answerable question.

3. To know where to find high quality relevant research.

4. To practice integrating the research evidence with

other important factors that contribute to public health

decisions.

5. To plan for implementation and evaluation

3

Purpose

Describe the steps involved in evidence-informed public health, and apply this process to a practice scenario relevant to public health

Who are we? One of six National Collaborating

Centres for Public Health created by the Government of Canada to renew and strengthen public health.

Aims of the National Collaborating Centres

• Increase awareness of new and existing knowledge.

• Identify and help address public health priorities. • Collaborate with established public health

networks. • Identify gaps in knowledge and relevant applied

research.

NCC Aboriginal HealthPrince George, BC

NCC Environmental HealthVancouver, BC

NCC Infectious DiseasesWinnipeg, MB

NCCMethods and ToolsHamilton, ON

NCC Healthy Public PolicyMontreal and Quebec City, PQ

NCC Determinants of HealthAntigonish, NS

http://www.nccah-ccnsa.ca/

http://www.ncceh.ca/

http://www.nccid.ca/en/home

http://www.nccmt.ca/

http://www.healthypublicpolicy.ca/2/Home.htm

http://www.nccdh.ca/

Methods and tools for what?

sharing what works in public health

8

NCCMT products and services

• DialoguePH• Public Health+• Workshops• Online learning modules• Webcasts• Webinars• Searchable Registry of Methods and Tools• Online Health Program Planner

sharing what works in public health

9

Health Evidence

• Registry of systematic review evidence

• Capacity building & knowledge translation

• Research evaluating KT strategies

10

Health-Evidence.ca database

o2100+ reviewsoComprehensive & transparent processoQuality appraisedoEvidence-summaries (86 well-done

reviews)

11

Health Evidence – engagement

• 40, 000 visitors annually (~110/day)• 5,000 registered users• Tailoring options• Feedback and user queries• Knowledge brokers• HE-CBPP simultaneous searches• Social media strategy in development

o YouTubeo Twitter

12

Complementary Activities

• National in scope

• Knowledge brokers functioning at different levels within overall PH system to build capacity

• Complementary services – HE: Access to content & tailoring through

engagement– NCCMT: EIDM

13

EIPH Workshop facilitators

• Jackie Muresan • Kara DeCorby• Donna Ciliska• Pam Forsyth

Evidence Transfer Gap

Evidence of Gap in Acute and Primary Care

– Consistent evidence of failure to translate research findings into clinical practice

– 30-40% patients do not get treatments of proven effectiveness

– 20-25% patients get care that is not needed or potentially harmful

Schuster, McGlynn, Brook (1998). Milbank Memorial Quarterly

Grol R (2001). Med Care

16

What is

Evidence-Informed Decision Making in Public Health?

In addition to the best available research evidence, evidence-informed decisions in public health consider valuable evidence from a variety of sources:

• community health issues and local context• existing public health resources• community and political climate

17

What is

Evidence-Informed Decision Making in Public Health?

Decision makers must rely on their public health expertise to integrate all relevant factors into any conclusions or recommendations.

18

What is Evidence-Informed Public Health?

… the process of distilling and disseminating the best available evidence from research, context and experience, and using that evidence to inform and improve public health policy and practice.

19

What is Evidence-Informed Public Health?

Put simply, it means finding, using and sharing what works in public health.

20

Why implement Evidence-Informed Public Health?

• Effectively transfer knowledge from both quantitative and qualitative research into practice and policy.

• Efficiently use limited health care resources.

21


• Create programs and actions that are both effective and appropriate for our communities and target populations.

• Improve client satisfaction.• Strengthen public health in Canada.

22


Do what works … and don’t do what does not work!

23

Barriers to Evidence-Informed Public Health

• Lack of time• Difficulty accessing research• Lack of skills or confidence with critically appraising

evidence• Difficulty interpreting results • Challenges with applying research findings to local

context and setting• Insufficient power or support to make a change• Environment/organizational culture

(Keifer et al.,2005; Retsas, 2000; Thomas et al., 2004)

Stages in the process of

Evidence-Informed Public Health



Clearly define the question or problem

26

Scenario

You are a public health nurse working on a team that focuses on healthy living and physical activity. Despite walking being free and relatively safe, a recent survey of adults in the community indicated few people were walking regularly or engaging in any exercise. In a team meeting, you discuss whether a community-wide strategy should be implemented to promote walking and increase physical activity among sedentary adults. You are unsure which intervention would be most effective, and whether specific subgroups should be targeted (vs. a broader approach). Your supervisor has asked you to identify the best available research about the effectiveness of interventions to promote walking, and share with the team.

27

Define the question

P opulation

I ntervention

C omparison

O utcome

(T) ime



Efficiently search for research evidence

SearchingStart here with a question

(DiCenso et al., 2009; Haynes et al. 2005; Robeson et al., 2010)

6S Pyramid

6S pyramid of pre-processed research evidence


SearchingStart here with a question

(program* OR intervention*) AND (walking)


0

174

18

329

Google Scholar 103,000Google 38,200,000

3

32

health-evidence.ca

• Online registry of reviews evaluating effectiveness of public health interventions.

• Provides quality rating for every review• Summary statements are available for some high quality

reviews and include:– PICO and main findings of the study– Methodological strengths and weaknesses– Implications for policy/practice

37

What is a systematic review?

• A rigorous summary of all primary research evidence relevant to a focused question.

• Evidence is consolidated through a systematic process to avoid possible sources bias at various stages in the review process.

(Haynes, 2007)

Search of personal files

Systematic manual search of key journals

Computerized databases

Review of reference lists of

articles

Consultation with experts

Identify studies

Review for relevance

Relevant

Evaluate methodological quality

Extract data

Analyze data

Draw conclusions

RejectNot Relevant

Steps of a Systematic OverviewDefine the question

39

What is a meta-analysis?

• A method for combining the results of several independent studies that measure the same outcomes so that an overall summary statistic can be calculated.

(Dawson-Saunders & Trapp, 1994)

Royal S, Kendrick D, Coleman T. Non-legislative interventions for the promotion of cycle helmet wearing by children. Cochrane Database of Systematic Reviews 2005, Issue 2

Richardson, C.R., Newton, T.L., Abraham, J.J., Sen, A., Jimbo, M., Swartz, A.M.(2008). A meta-analysis of pedometer-based walking interventions and weight loss. Annals of Family Medicine, 6(1), 69-77.

42

Narrative summary vs. Meta-analysis

• Clinical heterogeneity• Multiple outcomes• Availability of data

* Review authors need to justify the method they choose

43

Is the summary statement useful?

• What is the “actionable message”?

• What if you do not find an answer in health-evidence.ca?

45

http://www.guideline.gov/

http://www.ncbi.nlm.nih.gov/pubmed/

Public Health+

www.nccmt.ca

http://cbpp-pcpe.phac-aspc.gc.ca/



Critically and efficiently appraise the research methods

53

Practice appraisal

• Refer to NCCMT’s online learning modules for additional practice: http://www.nccmt.ca/modules/index-eng.html

• Week long critical appraisal course at McMaster

http://www.nccmt.ca/modules/index-eng.html




Interpret information; understand how to prioritize conflicting results



Adapt the information to the local community



Decide whether (and plan how) to implement the

change in practice or policy

58

• Situational assessment – Identify barriers and facilitators in the

organization, the people, and the change you are trying to make.

• Identify key people.

• Obtain approval from appropriate leadership.

• Create a project timeline.

Implementation

(Dobbins et al. 2005; Fineout-Overholt & Johnston, 2006)

59

http://www.rnao.org/Page.asp?PageID=924&ContentID=823

Related resources:DiCenso A et al. A toolkit to facilitate the implementation of clinical practice guidelines in

healthcare settings. Hospital Quarterly 2002;5(3):55-60.

Dobbins M et al. Changing Nursing Practice: Evaluating the Usefulness of a Best-Practice Guideline Implementation Toolkit. Nursing Leadership 2005;18(1):34-45.

Implementation Toolkit

http://www.rnao.org/Page.asp?PageID=924&ContentID=823

60

• What is the message?• To whom (audience)?• By whom (messenger)?• How (transfer method)?• With what expected impact (evaluation)?

Implementation

(Institute of Work & Health with J. Lavis, 2006. www.iwh.on.ca)

http://www.iwh.on.ca/

61

Identify Strategies to Disseminate Information

• Policy change• Education • Academic detailing/outreach visits• Audit and feedback• Opinion leaders• Knowledge broker• Champions• Reminders: prompts; patient reminds staff• Interactive educational meetings/workshops • Multiple interventions

http://www.thcu.ca/ohpp/



Assess the effectiveness of the change in

practice or policy

64

Evaluation

• How will you know if people are using the evidence? – Have they changed their practice? – Does it make any difference to patients?

• Decide on indicators (structure, process, outcome)

• Gather baseline data

















73

Step ‘0’Reflecting on areas of uncertainty

• Examine practice critically.

• Acknowledge uncertainty in your practice.

• Formulate a focused clinical questions through the process of reflective practice.

• Will contribute to professional practice requirements.

(Johnston & Fineout-Overholt, 2005; Witmer & Cullum, 1999)

74

Road ahead

• Individual skills

• Organizational

capacity, leadership

resources

and culture

75

How can I continue to develop the skills needed for EIPH?

• On-line learning moduleshttp://www.nccmt.ca/modules/index-eng.html

• Dialogue PHOn-line support and discussion

• EIDM workshops offered by Health Evidence or NCCMT


76

NCCMT future directions

• Renewed website• Upcoming conferences with networking opportunities:

CHNC, CPHA, • New modules• Web casts• Digital stories..share yours with us!• Discussion forums linked to CHNETworks

77

HE future directions

• More summary statements are on the way!

• Plans to disseminate

• Continuing work with public health units

• Improving health-evidence.ca (watch for our

survey!)

Feedback• Your input is needed to determine

whether this workshop was effective in meeting your learning needs.

• Please take a minute to complete the evaluation form in your packages and provide your comments or suggestions.

Contact Info

Jackie Muresan, RN, MScKnowledge BrokerNational Collaborating Centre for

Methods and Tools (NCCMT)McMaster University905-525-9140, x 20453

[email protected]

Donna Ciliska, RN, PhDProfessor and Scientific Director

905-525-9140, x [email protected]

Kara DeCorby, MScResearch CoordinatorHealth Evidence

McMaster University905-525-9140, ext. [email protected]

mailto:[email protected]



Evidence-Informed Public Health Decisions Made Easier: Take it one Step at a Time

Health & Medicine

worksin public health

public health priorities

health evidence engagement

research evidence

process of evidence

knowledge translation

relevant applied research

existing knowledge