NCCMT Webinar: Injury-Prevention-Focused Methods and Tools to Support Evidence-Informed Decision-Making

Follow us @nccmt Suivez-nous @ccnmo

Funded by the Public Health Agency of Canada | Affiliated with McMaster University The views expressed here do not necessarily reflect the views of the Public Health Agency of Canada.

Injury-Prevention-Focused

Methods and Tools to Support

Evidence-Informed Decision-Making

Susan Snelling, PhD

Senior Knowledge Translation Specialist, NCCMT

June 22, 2016


Poll Question #1

2

How many people are watching today’s webinar with you?

a) Just me b) 1 to 3 c) 4 to 5 d) 6 to 10 e) More than 10


Housekeeping

• TC line will be muted during the webinar

Use Q&A to post comments / questions during

the webinar

• ‘Send’ questions to All (not privately to

‘Host’)

Connection issues

• Recommend using a wired Internet

connection (vs. wireless),

• WebEx 24/7 help line: 1-866-229-3239

Q&A

3


Housekeeping

Experiencing feedback?

• Click Audio Broadcast at the bottom of the

Participants panel to join or leave the audio

broadcast.

4


Availability of the Slides and Audio

Recording

• The PowerPoint presentation (in English and

French) and English audio recording will be

available at:

• PowerPoint: http://www.slideshare.net/nccmt

• Audio Recording: https://www.youtube.com/user/nccmt

5

http://www.slideshare.net/nccmt

https://www.youtube.com/user/nccmt


Follow us @nccmt Suivez-nous @ccnmo 6

Outline of Today’s Webinar

• Introduction to

NCCMT

• A model for

evidence-informed

decision making

• Steps in the process

of evidence-informed

public health

With reference to

methods & tools

relevant for

injury prevention

practitioners


National Collaborating Centre for

Methods and Tools (NCCMT)

Vision

• The effectiveness and efficiency of Canada’s public

health system will be guided by the production,

sharing and use of high quality evidence.

Mission

• The NCCMT will enhance evidence-informed public

health practice and policy in Canada by providing

leadership and expertise in supporting the uptake of

what works in public health into practice.

7


NCCMT Registry of Methods and Tools

Resources to support use of research evidence in

public health

http://www.nccmt.ca/resources/registry

A free, searchable, online collection of knowledge

translation methods and tools for public health

• Summary statements

• Web links (when available) to Methods and Tools

8





NCCMT Registry of Methods and Tools for Knowledge Translation

9


What are methods and tools?

A process or series of steps to organize a Knowledge Translation activity

Dissemination framework

Method An instrument to carry out the steps of a Knowledge Translation activity

Dissemination checklist

Tool

10


Evidence… of what?

11


Evidence of the Problem versus

Evidence of Effective Interventions

• Injury Prevention has a lot of data on the

problem: rates of injury, causes, burden…

• Knowing the nature and magnitude of the

problem helps to focus efforts.

12


Example: www.parachutecanada.org

13

Evidence of the

problem comes in

reports on cost,

surveillance

reports…

http://www.parachutecanada.org/


Evidence of the Problem versus

Evidence of Effective Interventions

• Injury Prevention has a lot of data on the

problem: rates of injury, causes, burden…

• Knowing the nature and magnitude of the

problem helps to focus efforts.

• Knowing the problem doesn’t tell us what

effective interventions are.

• We also want to find the best available

evidence for effective interventions.

14


A Model for Evidence-Informed

Decision-Making in Public Health

15


What is

Evidence-Informed Public Health?

… the process of distilling and disseminating

the best available evidence from research, context

and experience, and using that evidence to inform

and improve public health policy and practice.

16


What does it mean for an intervention

to be ‘effective’?

Recommended resource:

http://www.nccmt.ca/resources/search/129

17

Centers for Disease

Control and

Prevention (2011).





Strength of evidence and Effectiveness

18


Stages in the process of

Evidence-Informed Public Health

19


Scenario: Setting the Stage for

Evidence Use in Injury Prevention

• Falls prevention for seniors living

at home

• Opportunity to offer some

programming in partnership with

several stakeholders

How can the steps of Evidence-

Informed Public Health be applied

to this scenario?

20



Clearly define the question or

problem


Frameworks can help:

BEEEACH Model of Categories and

Settings of Interventions

22


Define the question

PICO

P population

I intervention

C comparison

O outcome

23


Define the question: Apply our scenario

PICO

P population: community-dwelling seniors

I intervention: [we’ll leave unspecified]

C comparison: no intervention or usual

care

O outcome: prevention of falls

24


Recommended Tool: Define

Developing an Efficient Search Strategy Using

PICO


Use it to develop an evidence-search question

using the PICO format.

25

http://www.healthevidence.org/practice-tools.aspx#PT2








Efficiently search for research

evidence


Recommended Tool: Search

NCCMT Search Pyramids http://www.nccmt.ca/resources/search/181

http://www.nccmt.ca/learningcentre/EN/index.php

• Pre-populated with good sites for finding evidence in public health, and for certain specific topics.

• Start at the top for the most synthesized evidence.

• Saves time

• Systematic approach to searching

28




29


Recommended Tool: Search

National Guideline Clearinghouse

31

32

33

Recommended Tool: Search www.healthevidence.org

http://www.healthevidence.org/

34

35

36

37

38

39

40

41


Tips for “Search”

• Choose the highest possible level on the Pyramid

• Pre-appraised sources make your search more

efficient

• NCCMT online module on Searching through

Learning Centre

42



Critically and efficiently appraise

the research methods


Recommended Tools: Appraise

44

Critically Appraising Practice

Guidelines: The AGREE II

Instrument

http://www.nccmt.ca/registry/view/eng/100

Tools to Assess the Methodological

Quality of Systematic Reviews:

AMSTAR

http://www.nccmt.ca/registry/view/eng/97

Critical Appraisal Skills Programme:

Checklists


http://www.nccmt.ca/pubs/CompendiumToolENG.pdf

http://www.nccmt.ca/registry/view/eng/100.html

http://www.nccmt.ca/registry/view/eng/97.html



Tips for “Appraise”

• Appraise is a filtering step, looking for sources

of bias

• Answers the question: is this research/review of

good enough quality that I should read further?

• All research has flaws! But some flaws are

more serious than others.

45


Additional Resources

• Refer to NCCMT’s online learning modules for

additional practice:

http://www.nccmt.ca/professional-development/modules

• Week-long course at McMaster University: “What

is Evidence-Informed Decision Making (EIDM)?” http://ccebn.mcmaster.ca/workshop

46





http://ccebn.mcmaster.ca/workshop.html



Interpret information;

understand how to prioritize

conflicting results


Recommended Tool: Synthesize

Briefing Note: Decisions, Rationale and Key Findings

Summary


Use it to create briefing notes to inform decision-

makers. Use ‘actionable messages’ to inform program

and policy development.

48




Summarize Key Findings from Stubbs

• exercise reduces falls (including the rate, risk,

and odds of falling), with 13/14 pooled analyses

(93%) from 7 meta-analyses demonstrating a

significant reduction.

• multifactorial interventions reduce falls (5/6,

83% reported significant reduction).

• conflicting evidence regarding the influence of

vitamin D supplementation (7/12, 58.3%

reported significant reduction).

49


Bringing it all together…back to the

framework!

50

51


Tips for “Synthesize”

• Bring good quality evidence together to

determine results and implications

• If results ‘conflict’, prefer sources that are

highest quality, most recent, most relevant to

your question

• Identify actionable findings that can guide

decisions

52



Adapt the information

to the local community


A Model for Evidence-Informed

Decision-Making in Public Health

54

Recommended Tools: Adapt - Applicability and Transferability of Evidence

55








Online Learning Module Assessing the Applicability and Transferability

of Evidence

56


Tips for “Adapt”

57

• In developing a local approach, consider:

• Research evidence

• Stakeholder perspectives

• Media coverage – what’s going on in community

• Political context

• Local and regional data reports

• Financial and human resources

• How can you adapt potential interventions to be

appropriate for your community?



Decide whether

(and plan how) to implement

the change in practice or policy


https://www.mcmasteroptimalaging.org/

59

Which is more likely to meet the need:

Communicating evidence to partners and community members





60

Recommended Tool: Implement






Assess the effectiveness

of the change in

practice or policy


Recommended Tool: Evaluate

Improving Future Decisions: Optimizing the

Decision Process from Lessons Learned


Use it to evaluate your use of evidence in

making decisions.

A reflective practice tool.

62


63


Tips for “Evaluate” Determine how effective you were at using an evidence-informed approach:

• What process did you follow?

• What can you learn from your process?

• How could you improve your use of evidence?

• How will you know if people are using the evidence? Have they changed their practice?

• Was the intervention adapted effectively at your local level?

You may need to gather baseline data before you start if you want to assess change.

64




65


Poll Question #2

66

How familiar are you with the methods and tools described today?

a) I am not familiar with any of the

methods/tools b) I have heard of one or more of the

methods/tools c) I have used one or more of the methods/tools


Recommended tool: Evidence-

Informed Decision Making Checklist http://www.nccmt.ca/resources/search/237

67

EIDM Phases & Steps File Path / Link to

Saved Location on: (insert name of your

network here)

1. Define question Was a clear answerable search question developed?

What was the question?

P

I

C

O

2. Search for relevant evidence Was a comprehensive search strategy employed to find the best available evidence to

address this question?

PICO search terms table (See: Developing an Efficient Search Strategy Using PICO)

Years searched:

Pyramid results (See: Levels & Sources of Public Health Evidence)

Search results (See: Keeping Track of Search Results: A Flowchart)

References saved in reference management software database (e.g. Reference Manager / RefWorks)

What relevance criteria were used to determine evidence for quality assessment:

Primary

Titles and abstracts as found in reference management database Save as separate reference management database.

Secondary

Relevance assessment of full document versions Save as separate reference management database.

How many papers remained following relevance assessment(s)? (See: Flowchart, above)

3. Appraise Was quality assessment conducted on relevant evidence?

How many papers remained after quality assessment? (See: Flowchart, above)

4. Synthesize What were the results of the review of the evidence?

What were the actionable messages from the evidence?

5. Adapt.

NCCMT’s Applicability and Transferability tool; AHRQ’s Will It Work Here? A Decisionmaker's Guide to Adopting Innovations

6. Implement.

7. Evaluate.



http://www.nccmt.ca/eiph/index-eng.html

http://www.healthevidence.org/documents/practice-tools/HETools_DevelopingEfficientSearchStrategyUsingPICO_18.Mar.2013.doc

http://www.healthevidence.org/documents/practice-tools/HETools_Levels&SourcesPublicHealthEvidence_18.Mar.2013.doc

http://www.healthevidence.org/documents/practice-tools/HETools_KeepingTrackSearchResultsFlowchart_18.Mar.2013.ppt

http://www.nccmt.ca/publications/9/view-eng.html

http://www.innovations.ahrq.gov/guide/guideTOC.aspx?utm_source=issueanc&utm_medium=email&utm_campaign=20110316


Poll Question #3

68

To what extent will the methods or tools described today be useful in your practice?

a) Very useful b) Somewhat useful c) Not at all useful d) Don’t know


• Use Q&A to post comments

and/or questions

• ‘Send’ questions to All (not

privately to ‘Host’)

Q&A

Your Comments/Questions

69


Poll Question #4

70

What are your next steps? (Check all that apply)

a) Access a method/tool referenced in the presentation

b) Read an NCCMT summary about a method/tool described today

c) Consider using the methods/tools in practice

d) Tell a colleague about the methods/tools


Availability of the Slides and Audio

Recording

• The PowerPoint presentation (in English and

French) and English audio recording will be

available at:

• PowerPoint: http://www.slideshare.net/nccmt

• Audio Recording: https://www.youtube.com/user/nccmt

71

http://www.slideshare.net/nccmt




Your Feedback is Important

Please take a few minutes to share your thoughts

on today’s webinar.

https://nccmt.co1.qualtrics.com/SE/?SID=SV_9EIx

wibAzI4IgcJ

72

https://macmail.mcmaster.ca/owa/redir.aspx?REF=ubfvycDUq47amHQ6jORtLnXZYXJpLtE5H257MAn4zGe0ev7qMJnTCAFodHRwczovL25jY210LmNvMS5xdWFsdHJpY3MuY29tL1NFLz9TSUQ9U1ZfOUVJeHdpYkF6STRJZ2NK

https://macmail.mcmaster.ca/owa/redir.aspx?REF=ubfvycDUq47amHQ6jORtLnXZYXJpLtE5H257MAn4zGe0ev7qMJnTCAFodHRwczovL25jY210LmNvMS5xdWFsdHJpY3MuY29tL1NFLz9TSUQ9U1ZfOUVJeHdpYkF6STRJZ2NK


For more information about the

National Collaborating Centre

for Methods and Tools:

73

NCCMT website www.nccmt.ca

Contact: [email protected]

http://www.nccmt.ca/

mailto:[email protected]

NCCMT Webinar: Injury-Prevention-Focused Methods and Tools to Support Evidence-Informed Decision-Making

Healthcare