Changing physician behavior: an exercise in futility? Jeremy Grimshaw Canada Research Chair in Health Knowledge Transfer and Uptake Director, Clinical.

Changing physician behavior:

an exercise in futility?

Jeremy GrimshawCanada Research Chair in Health

Knowledge Transfer and UptakeDirector, Clinical Epidemiology Program

Ottawa Health Research Institute

Personal perspective

Focus has been on: professional and organizational behavior change. improving technical aspects of care ie how do we

ensure patients get the right (evidence based) treatments at the right time.

populations of physicians (and health care organizations).

My life as an atheoretical empiricist (and an unashamed positivist)

Background

Research is consistently producing new findings that may contribute to effective and efficient patient care

Watt observed that 50% reduction in mortality in ischemic heart disease over past 50 years has been due to advances in clinical care and suggested we were entering a post McKeown era where evidence based health services can improve population health

The findings of such research will not change population outcomes unless health services and health care professionals adopt them in practice.

Grimshaw, Ward, Eccles (2001). Oxford Handbook of Public Health.Watt (2002) Lancet

Background Consistent evidence of failure to translate research

findings into clinical practice 30-40% patients do not get treatments of proven

effectiveness 20–25% patients get care that is not needed or

potentially harmful IOM round table identified failure to translate

research findings as one of the major blocks to improved health outcomes

‘Evidence based medicine should be complemented by evidence based implementation’

Schuster, McGlynn, Brook (1998). Milbank Memorial QuarterlyGrol (1997) BMJ

Sung et al (2003). JAMA

Grol R (2001). Med Care

Background

Local health care organisations have relatively few resources for implementation activities and decision makers need to consider how best to use these to maximise benefits

Dissemination and implementation strategies are not without costs.

In some circumstances, the costs of development and introduction are likely to outweigh their potential benefits. In other circumstances, it may be more efficient to adopt less costly but less effective dissemination and implementation strategies.

Background

If decision makers are to make decisions aboutimplementation strategies informed by research evidence, they need information on: likely effectiveness of interventions (direction of

effect, predicted effect size of intervention and relative effectiveness of different interventions);

effect modifiers (practice environment, potential adopter, characteristics of behaviour);

and, resources need to deliver interventions.

Rigorous evaluations (mainly randomised controlled trials) provide the best evidence of effectiveness of different interventions because: Effects of interventions are modest Limited understanding of likely confounders Substantial opportunity costs if ineffective or

inefficient dissemination and implementation strategies used

Eccles (2003) Quality and Safety in Health Care

Background

Background

Systematic reviews of rigorous evaluations of implementation interventions should inform decisions because they: identify and summarise evidence on the

effectiveness of interventions allow the generalisability and consistency of

research findings to be assessed allow exploration of data inconsistencies to be

explored. Mulrow (1994) British Medical Journal

Grimshaw (2003) Quality and Safety in Health Care

Overview of previous systematic reviews

Generally ineffective

Mixed effects Generally effective

Dissemination of printed educational materials

Audit and feedback Reminders

Didactic educational session

Local opinion leaders Educational outreach

Multifaceted interventions

Bero et al (1998). BMJ

Grimshaw et al (2002). Medical Care

Problems with previous systematic reviews

Use of vote counting techniques: Problems handling studies where statistical significance

of comparison is uncertain (eg studies with unit of analysis errors)

Don’t provide any estimate of effect size Inadequate consideration of quality of primary

studies Conflation of multifaceted interventions into single

intervention category eg educational outreach

Systematic review of guideline dissemination and implementation strategies

Objective Systematic review of the effectiveness and

efficiency of guideline dissemination and implementation strategies to promote improved professional practice.


Inclusion criteria Study designs – RCTs, CCTs, CBAs, ITS Participants – medically qualified health care

professionals Interventions – guideline dissemination and

implementation strategies Outcomes – objective measures of provider behaviour

and/or patient outcome

Search strategy Cochrane Effective Practice and Organisation of Care

group’s search strategy

Results – included studies

Search strategy identified approx 150,000 hits

5,000 hits identified as potentially relevant Full text 863 reports retrieved Included 285 reports of 235 studies, yielding

309 separate comparisons

Results – methodological quality

Overall methodological quality poor (eg unit of analysis errors common)

Poor description of interventions Only 27% of studies used theories and/or

psychological constructs 29.4% comparisons reported any economic

data


Intervention Number of CRCTs

Median effect size

Range

Educational materials

5 +8.1% +3.6%, +17.0%

Audit and feedback

5 +7.0% +1.3%, +16.0%

Reminders 14 +14.1% –1.0%, +34.0%

Results – single interventions


16 28 46 63 56 N =

Absolute effect size

Number of interventions in treatment group

>4 4 3 2 1

80%

60%

40%

20%

0%

-20%

-40%

-60%

-80%


Multifaceted interventions including educational outreach13 RCT

Median effect +6.0% (range –4% to +17.4%)


Conclusions Imperfect evidence base for decision makers Many current rigorous evaluations have

methodological weaknesses (eg unit of analysis errors)

Poor reporting of study settings, barriers to change, content and rationale of intervention

Generalisability of study findings is frequently uncertain


Conclusions Improvements in direction of effect in 86% of

comparisons Reminders most consistently observed to be effective Educational outreach only led to modest effects Dissemination of educational materials may lead to

modest but potentially important effects (similar effects to more intensive interventions)

Multifaceted interventions not necessarily more effective than single interventions

UK MRC Framework for Evaluating Complex Interventions

Theory Modelling Exploratory trial Definitive RCT Long termimplementation

Pre-clinical

Phase I

Phase II

Phase III

Phase IV

Continuum of increasing evidence

My life as an unashamed positivist

Innovation

Potential Adopters

Practice Environment

Interventions

Adoption

Outcomes

Ottawa Model Of Research Use

Assess + Monitor + Evaluatebarriers & supports interventions outcomes & degree of use

Copyright, Logan, J. & Graham, ID 2003

Towards evidence based practice

Ferlie and Shortell suggested four levels at which interventions to improve the quality of health care might operate:

the individual health professional; health care groups or teams; organisations providing health care (e.g., NHS

trusts); the larger health care system or environment in

which individual organizations are embedded. Ferlie, Shortell (2001). Milbank Quarterly


Levels of engagement Policy

Macro (national/provincial) Meso (organisational)

Managerial Provider Consumer


Most clinical care occurs within the context of a provider – patient dyad.

Interventions at other levels are mediated through the provider – patient dyad.

Provider behaviour is most proximal determinants of evidence based practice.

Changing provider behaviour is (one of) the right objective(s) to promote evidence based behaviour.

Implications for KU research

Further research is required to develop a better theoretical understanding of professional behavior change by exploring determinants of provider and organisational behaviour to better identify modifiable and non modifiable factors Incremental predictive or action orientated models Emphasis should be on developing standard

methods for operationalising theories and then testing their utility


Further research is required to estimate the effectiveness and efficiency of dissemination and implementation strategies in the presence of different barriers and effect modifiers. Rigorous designs (pragmatic RCTs) Maximise informativeness by exploring causal

mechanisms (based on theoretical constructs) and economic evaluations


Further research is required to develop practical methods of identifying barriers and facilitators to change outside an academic context.


Barriers to progress Lack of consensus around overarching

frameworks and candidate theories Measurement issues Lack of interdisciplinary approach Lack of programmatic funding and

approaches Lack of cohesion in KU research community

Summary

KU research is about saving lives, improving health outcomes and enhancing the quality of health services.

The good news – changing physician behavior is possible though current efforts only achieve modest effects.

I am currently more optimistic than ever that with concerted research effort within increasingly cohesive research community, it should be possible to improve incrementally KU.

Implications for KU trainees (and researchers)

Identify an research area that can engage you over a 3 – 5 year period

Avoid dogma Avoid jargon Keep focused on the larger picture Challenge your supervisors (we don’t know

what we’re doing either!)

Changing physician behavior: an exercise in futility? Jeremy Grimshaw Canada Research Chair in Health Knowledge Transfer and Uptake Director, Clinical.

Documents

health care background

research evidence

health outcomes evidence

health care professionals

health care organizations

health services

population health

med care slide