Building and Assessing Competence David C. Leach, M.D. Executive Director ACGME September 12, 2002.

Building and Assessing Competence

David C. Leach, M.D.

Executive Director

ACGME

September 12, 2002

Objectives

• To clarify what you have known all of your professional life about competence

• To explore how residents learn to make good clinical judgments

• To define specific steps that can be taken to respond to the ACGME Outcome Initiative

Reasons this is hard

Yet can also be immensely satisfying.

Reason Number OneCompetence is a Habit

Competence

“…the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions,

values, and reflection in daily practice for the benefit of the individual and

community being served.”Epstein and HundertJAMA, Jan. 9, 2001

Why worry about it?

• Public concerns with safety

• Variability in patterns of care that are not based on science

• Poor customer service

ACGME Outcome Project

• The Project A long term initiative

• The Visionto enhance residency education

• The Process through educational outcome assessment

Reason Number TwoACGME/RRCs judge

competence of ProgramsBoards judge individuals

A slight problem …

• We accredit programs

• Programs don’t exist

• The only things that are real are the humans and the relationships between humans in so-called programs

• These relationships can either inhibit or facilitate learning

So what do we accredit?

• Humans?

• Sets of relationships?

• Educational outcomes?

Reason Number ThreeHumans: the important things

are hard to measure

Dee Hock’s Criteria for Hiring People

• Integrity

• Motivation

• Capacity

• Understanding

• Knowledge

• Experience

Reason Number FourKnowing the rules is not enough

Residents need to prepare for the unknown

Agreement

Certainty

+

+ -

-

Chaos

Zone Of Complexity

Control

Stacey, 1996

“Needed are a few organizing principles to have conversations

about our work.”

Marvin Dunn, M. D.

Paul Batalden, M.D.

Organizing Principles

• General competencies

• Continuum

• Measurements

• Improvement models

The General Competencies

• Patient care

• Medical Knowledge

• Practice-based Learning and Improvement

• Interpersonal and Communication Skills

• Professionalism

• Systems-based Practice

Accreditation Aside

“You must call your mother every Sunday.”

Prescription or invitation?

The Continuum

Life after competent

Dreyfus Model of Skill Acquisition

• Novice

• Advanced Beginner

• Competent

• Proficient

• Expert

• Master

Dreyfus Model Novice Rules

Advanced Beginner Rules + Situation

Competent Rules + Selected Contexts + Accountable

Proficient Accountable + Intuitive

Immediately sees what

Expert Immediately sees how

Master Develops style

Loves surprise

“To become competent you must feel bad”

Hubert Dreyfus

Between Advanced Beginner and Competent

• The number of potentially relevant details becomes overwhelming

• Exhausting to manage with rules

• Choose a perspective

• Result depends on the perspective adopted by the learner/risk taking

• Fright replaces exhaustion

Two Paths

• Go back to rules– Cycle between advanced beginner and

competent – Burn out

• Become fully involved– Feel bad when wrong and good when right

Next

• Proficient - intuition replaces reasoned

responses. - immediately sees the problem - recognizes patterns

• Expert- immediately sees how to solve

problem• Master – styles, continuous learning

Conceptual Model

Patient Care

Med Know

Practice Based Learning

Inter & Comm Skills

Profess-ionalism

System-based Practice

Novice

Advance Beginner

Comp

Proficient

Expert

Master

Reason Number Five

Residents seek practical wisdom

Aristotle

• Episteme– Cognitive knowledge, science

• Techne– Craft/Art of medicine

• Phronesis – Practical wisdom

Accreditation Aside

Minimal threshold

Do your graduates know the rules?

Can they apply them in complex contexts without supervision?

Accreditation Aside

Improvement Model

Do your graduates have the habit of accountability?

Have they acquired practical wisdom?

Reason Number Six

The quality of the program is dependent on the quality of the

relationships.

Medicine, education and management are cooperative arts

rather than productive arts.

Therefore the quality of the activity is dependent on the quality of the

relationships.

Cooperative Arts

• Medicinecooperates with the body’s natural tendency to heal

• Teaching cooperates with the mind’s natural tendency to ascend to the truth

• Management cooperates with people’s natural tendency to form communities

Microsystems: another unit of learning

Real learning (intelligent adaptation) occurs in

microsystemsAnd sometimes in macrosystems.

Health care systems consist of macrosystems and microsystems

Paul Batalden, M.D.

Substance is enduring; form is ephemeral. Preserve substance;

modify form; know the difference.

Dee Hock

Substance attracts resources; form attracts expenses.

Dee Hock

Microsystems have a high substance to form ratio;

macrosystems have the reverse.

Characteristics of High Performing Microsystems

• Integration of information• Measurement• Interdependence• Supportiveness of the larger organization• Constancy of purpose• Investment in improvement• Alignment of roles and training• Connection with community Julie Mohr,

Ph.D.

Reason Number Seven

Introducing learners can enhance or inhibit the function of a microsystem.

This is a big opportunity for improvement.

Rehearsals are good for relationships and outcomes.

A restaurant in Chicago

Simulation offers a huge opportunity for improvement.

Whatever we measure we tend to improve.

Useful Concepts about Measurement

• Life is not condensable• We use models to understand life• All models are limited, some are useful• Measurements are applied to models• Both measurements and models must be

constantly reassessed• We need structured dialogue about

measurement

Useful Concepts

• Rules and context

• Science is universal; art is always unique

• Objective and subjective

Characteristics of good assessment

• Measures actual performance

• Identifies areas for improvement

• Satisfies reasonable request for accountability

• Is practical

• Is done over time to discern growth

www.acgme.org/Outcome/

• Assessment toolbox

• References

• Table of “best methods”

• Key considerations in selecting and implementing assessment approaches

• Assessment approaches

RRC Think Tank

• Chair Gail McGuiness, M.D.

• Clarify expectations for programs

• Clarify operational issues for RRCs

• Identify PIF questions relevant to assessment for relational database

What to Do Right Away

Forming the Initial Response

• Show evidence of initial plans• Institutions build in competencies and their

assessment into internal reviews• Change the verbs• Start with Global Assessments• Organize Focused Assessments• Patient and/or Professional Associate

Assessment

What if you were crazy enough to go beyond the minimal?

What would an improvement model look like?

Accreditation Aside

Minimal threshold – hide flaws

Improvement – expose problems and show how they were addressed

Requires much greater trust

Excellent programs will use rehearsals.

Simulations are not the same as simulators

Excellent programs will develop virtuous cycles between the microsystems and education.

Excellent programs and institutions get to pick their own

indicators.

Remember the Northern New England Cardiovascular Surgery

experience.

“To teach is to create a space in which obedience to truth is

practiced.”

Abba Felix

Desert Father

What we attend to and how we attend to it defines who we are.

A Community of Practice

• General Competencies

• Open data systems

• Celebrate benchmarks across disciplines

• Build knowledge about medical education

• Build knowledge about improving patient care

• Enhance public accountability

To Teach/Learn is to create a Space/Community in which

obedience to truth is practiced.