Chapter 019

1Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Chapter 19

Evidence Synthesis and Strategies for Evidence-Based Practice

2Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

EBP and Best Research Evidence

Evidence-based practice (EBP)—the conscientious integration of best research evidence with clinical expertise and patient values and needs in delivery of quality, cost-effective health care

Best research evidence Conduct and synthesis of available research

studies in an area

3Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Benefits and Barriers Related to Evidence-Based Nursing Practice Some nursing interventions are evidence-

based or supported by best research knowledge; others are under-researched

Some nurses readily use research-based interventions, and others are slower to make changes

Some clinical agencies are supportive of EBP; others have limited support for EBP process

4Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Benefits of Evidence-Based Practice in Nursing

Greatest—improved outcomes for patients, providers, and healthcare agencies

Research syntheses provide basis for developing evidence-based guidelines for practice (gold standard for patient care)

Magnet status through American Nurses Credentialing Center (ANCC)

5Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Quality Care, as Defined by ANCC

Current literature is available, disseminated, used to change practice

Nurses are involved in human subjects committee work

Research consultants shape nursing research infrastructure, capacity, mentorship

Budget line items for allocation and utilization of resources for nursing research

6Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Quality Care, as Defined by ANCC (Cont’d)

Ongoing nursing research activities are compiled and documented

Education and mentoring activities engage staff nurses in research, EBP

Resources are available to support participation in nursing research, utilization

7Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Barriers of Evidence-Based Practice in Nursing

Lack of research evidence available regarding effectiveness of many nursing interventions

EBP requires synthesizing research evidence from experimental or quasi-experimental research Randomized controlled trials (RCTs) Other interventions studies

8Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Barriers of Evidence-Based Practice in Nursing (Cont’d)

Bolton, Donaldson, Rutledge, Bennett, and Brown (2007) Reviewed of systematic/integrative reviews and

meta-analyses Limited association found between nursing

interventions/ processes and patient outcomes in acute care settings

A concern—the “one size fits all” application mindset

9Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Barriers of Evidence-Based Practice in Nursing (Cont’d)

Best research evidence Generated mainly from experimental and quasi-

experimental research Limited focus on contributions of descriptive-

correlational studies, qualitative research, mixed-methods studies, and theories

Concern over “cookbook” approach Lack of support from healthcare

agencies/administrators

10Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Guidelines for Synthesizing Research Evidence

EBP organizational websites for nurses (e.g. Cochrane Collaboration Library, Joanne Briggs Institute)

11Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Guidelines for Implementing and Evaluating Systematic Reviews

Structured, comprehensive synthesis of research literature

Intent: determine best research evidence available

Identify, locate, appraise, synthesize quality research evidence

Often two or more researchers 10-step process

12Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 1: Formulate a Relevant Clinical Question to Direct the Review

PICO(S) often used to develop a relevant clinical question to guide review

P—Population or participants of interest I—Intervention C—Comparisons of the intervention with

control, placebo, standard care, variations of the same intervention, or different therapies

O—Outcome (desired) S—Study design

13Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 2: State the Purpose and Objectives or Aims of the Review

Identifies major goal or focus of the review “To collate, to present evidence, to analyze

____________”

14Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 3: Identify the Literature Search Criteria and Strategies

Identify inclusion/exclusion criteria to direct search

Search often excludes “gray literature”—limited-distribution items, such as theses, dissertations

15Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 4: Conduct a Comprehensive Search of the Research Literature

Provide extensive focused search Document exactly what was found where,

and through which database

16Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 5: Selection of Studies for Review

Whatever databases are used, expect duplications

Even if references are searched, expect eventual exhaustion of leads

Retrieve full-text citations Enter into a table and make notes

17Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Flow Diagram of Selection Process

18Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Four Phases of Selection

Identify sources Screen sources based on set criteria Determine whether sources meet eligibility

requirements Edit down to final list

19Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 6: Critical Appraisal of the Studies Included in Review

Initial brief critical appraisal Thorough critical appraisal after list is

finalized Two or more experts should independently

review and judge studies’ quality Studies often rank-ordered, based

contribution level

20Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 7: Conduct a Meta-analysis if Appropriate

Usually provides strong, objective information about effectiveness of intervention or solid knowledge about a clinical problem

21Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 8: Results of the Review

Tabular form often useful Items in table might include authors, year,

title, journal (or other source), design, sample, variables, measurement method, major finding, level of significance

22Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 9: Discussion Section of the Review

Overall evaluation of types of interventions implemented and outcomes measured

Methodological issues or limitations of review Theoretical basis Recommendations

23Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Step 10: Development of the Final Report for Publication

Title identifying systematic review or meta-analysis

Include abstract identifying: background, purpose, data sources, review methods, results, and conclusions

Body of the report: content discussed in previous nine steps

Checklist: http://www.prisma-statement.org

24Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Critical Appraisal of a Published Systematic Review

Checklist Table 19-2 Provide comments/rational for appraised

strengths/limitations of review Currency of literature synthesis important

Obsolete in 1 to 10 years, depending on extent of literature

25Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Conducting Meta-analyses to Synthesize Research Evidence

Considered high-level evidence Needs to be homogeneous—same general

population, intervention, variables Statistically combines data from several

studies Results in a large sample size Increased power Used to determine true effect of specific

intervention on a particular outcome

26Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Conducting Meta-analyses to Synthesize Research Evidence (Cont’d)

Goal is to determine if an intervention Significantly improves outcomes Has minimal/no effect on outcomes Actually increases risk of adverse events

Effective way to Average conflicting study findings and controversies

Results in some sort of overall impression

27Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Downsides to Meta-analysis

Rare to find same-population, same-intervention studies for pooling

Uses “second-hand” data If information is incomplete or flawed, meta-

analysis is flawed

28Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Clinical Question for Meta-analysis

“What is the effectiveness of a selected intervention?”

PICOS format could be used to generate the clinical question

29Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Purpose and Questions to Direct Meta-analysis

Cochrane Collaboration identified four basic questions: What is the direction of effect? What is the size of effect? Is the effect consistent across studies? What is the strength of evidence for the effect?

30Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Search Criteria and Strategies for Meta-analyses

Methods similar to those for systematic reviews

Search is for similar-method studies Goal is meaningful pooled data

31Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Possible Biases for Meta-analyses and Systematic Reviews

Publication Time-lag Location bias Citation bias Language bias Outcome reporting bias

32Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Funnel Plot

Used to assess for biases in a group of studies

Provide graphic representations of possible effect sizes (ESs) or odds ratios (ORs) for interventions in selected studies

33Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Results of Meta-Analysis for Continuous Outcomes

If continuous outcomes are measured in all studies (ratio or interval data) May calculate weighted average of outcomes Answers the question, “Was there a change?” Standardized mean difference Disadvantage: conclusion will be “on average,

there was a change” (can mean next to nothing)

34Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Results of Meta-Analysis for Dichotomous Outcomes

Dichotomous outcomes (Yes-No) Can be characterized as (Yes, improved, and No,

not improved) Risk ratios, odds ratios, and risk difference

usually calculated to determine effect of intervention on measured outcome

35Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Risk Ratio or Relative Risk (RR)

36Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Odds Ratio (OR)

“The likelihood of…..”

37Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Risk Difference (RD)

Absolute risk reduction Risk of an event in experimental group minus

risk of the event in control or standard care group

38Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Forest Plot

39Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Forest Plot (Cont’d)

The black diamond in the plot is the summary of effect of all studies

If the diamond is left of the vertical line that is positioned at 1, then the results favor intervention or treatment

40Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Conducting Meta-synthesis of Qualitative Research

Systematic review/formal integration of findings from qualitative studies

Still in developmental phase Variety of synthesis methods: meta-synthesis,

meta-ethnography, meta-study, meta-narrative, qualitative metasummary, qualitative meta-analysis, aggregated analysis

41Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Conducting Meta-synthesis of Qualitative Research (Cont’d)

No agreement about method to use, or if one method is enough to accomplish purpose

Cochrane Collaboration Cochrane Qualitative Methods Group

42Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Meta-synthesis

Most prevalent method for at least the past 10 years

The systematic compiling/integration of qualitative results

Like telling a story about a group of stories Metasummary

43Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Framing a Meta-synthesis Exercise

Providing a focus, and defining scope of exercise

Difficult to accomplish

44Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Searching the Literature and Selecting Sources

No different from quantitative meta-analyses when there is not much literature

All sources acceptable May choose like-methods research

45Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Appraisal of Studies and Analysis of Data

Use critical appraisal guidelines in text Various recurrent themes/essences either

noted or tallied Comparative analysis

46Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Discussion of Meta-synthesis Findings

May result in a larger number of themes If grounded theory, could result in the

beginnings of the theory Report often concludes with

recommendations for further research and possibly implications for practice, policy development

47Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Mixed-methods Systematic Reviews

Includes only quantitative, of mixed types, or both quantitative and qualitative

Used when there is a shortage of single-method studies

No definitive results can be obtained Multilevel synthesis Parallel synthesis

48Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Models to Promote Evidence-based Practice in Nursing

Stetler model: facilitate EBP The Iowa model: promote quality of care

49Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Model of Research Utilization

Developed by Stetler and Marram in 1976 Expanded/refined by Stetler in 1994 and 2001 Institutional level: synthesized research knowledge

used to develop/update protocols, algorithms, policies, procedures, or other formal programs

Individual level: those in leadership positions summarize research and use knowledge to influence educational programs, make practice decisions, impact political decision making

50Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Model of Research Utilization (Cont’d)

51Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Modelphase I: Preparation

Determining purpose, focus, potential outcomes of making an evidence-based change in a clinical agency

52Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Modelphase II: Validation

Research reports critically appraised to determine scientific soundness

53Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Modelphase III: Comparative Evaluation/Decision Making

Substantiation of evidence Fit with healthcare setting Feasibility of using research findings Concerns with current practice Benefits/risks assessed

Use it Consider it Don’t use it

54Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Modelphase IV: Translation/Application

Planning for/using research evidence in practice

Determining exactly what knowledge will be used/applied

55Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Modelphase IV: Translation/Application (Cont’d)

Three levels of application: Cognitive application Instrumental application Symbolic or political application

56Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Stetler Modelphase V: Evaluation

Evaluates effect of the evidence-based change on some kind of outcomes

Everything from satisfaction surveys through QI projects

57Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

IOWA Model of Evidence-Based Practice

Provides direction for development of EBP Formerly not EBP-focused but used evidence

when available Identifies triggers for change Evaluated/prioritized based on clinical agency

needs Best evidence is used

58Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Development of Evidence-Based Guidelines

Agency for Healthcare Research and Quality (AHRQ)

Panels address various patient problems Early topics were symptom management Literature review, when possible Published as booklets National Guideline Clearinghouse (NGC)

59Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Evidence-based PracticeCenters (EPCs)

In 1997, AHRQ established 12 EPCs in Us and Canada

Develop evidence reports and technology assessments on healthcare organization and delivery issues

5-year contracts

60Copyright © 2013, 2009, 2005, 2001, 1997 by Saunders, an imprint of Elsevier Inc.

Introduction to Translational Research

An evolving concept Defined by the NIH as translation of basic

scientific discoveries into practical applications

For testing applications of basic research (with humans)

CTSA consortium