Beyond CME and Examination for Recertification – Maintenance ...

Beyond CME and Examination for Recertification –

Maintenance of Competency

Rebecca Lipner, PhD

VP, Psychometrics and Research Analysis

[email protected]

CNG Quarterly Meeting

March 17, 2010

Presentation Outline

Background on medical board certification History and philosophy of Maintenance of

Certification (MOC) Development of MOC program Planning and Implementation of MOC program Communication with stakeholders and reactions Challenges

Background:Physician Education, Licensure, and Certification

Medical Societies (Colleges, Academies & Societies)• National membership organizations (ACP, ACC)• Promote education and provide CME• Develop clinical guidelines & publish medical journals

Licensing Boards • State governed, non-profit, federated (USMLE/NBME)• Issue and regulate medical licenses— required for practice• Varying requirements for CME for maintenance of licensure

Certifying Boards (American Board of _____)• Non-profit “oversight” organizations• “Of the Profession, For the Public” • Certification and Maintenance of Certification define “the field”

(i.e., expectations of physician knowledge and responsibilities)

American Board of Medical Specialties (ABMS)

Founded in 1917 out of concern for quality care Largest self-regulatory group of physicians in US About 725,000 practicing certified physicians

24 ABMS member boards

• Allergy and Immunology• Anesthesiology• Colon & Rectal Surgery• Dermatology• Emergency Medicine• Family Medicine• Internal Medicine• Medical Genetics• Neurological Surgery• Nuclear Medicine• Obstetrics & Gynecology• Ophthalmology

• Orthopaedic Surgery• Otolaryngology• Pathology• Pediatrics• Physical Medicine and Rehabilitation• Plastic Surgery• Preventive Medicine• Psychiatry & Neurology• Radiology• Surgery• Thoracic Surgery• Urology

ABMS Member Boards

The American Board of:

American Board of Internal Medicine (ABIM)

ABIM founded in 1936; not for profit

Independent; certified physicians are Diplomates (not members)

~230,000 physicians certified in Internal Medicine

Increasingly involved in the national health policy dialogue

ABIM’s Mission

To enhance the quality of health care by certifying internists and subspecialists who demonstrate the knowledge, skills, and attitudes essential for excellent patient care.

“Of the profession… for the patient”

• Cardiovascular Disease• Endocrinology, Diabetes

and Metabolism• Gastroenterology• Hematology• Infectious Disease• Medical Oncology• Nephrology• Pulmonary Disease• Rheumatology

• Adolescent Medicine• Clinical Cardiac

Electrophysiology• Critical Care Medicine• Geriatric Medicine• Hospice & Palliative Care• Interventional Cardiology• Sleep Medicine• Sports Medicine• Transplant Hepatology

ABIM’s Subspecialty Disciplines

Stakeholders

Patients Physicians Others interested in physician performance

• Consumers, payers and insurers, pay for performance (P4P) organizations

• Quality organizations• AHRQ, AQA, NQF, NCQA, PCPI (AMA)

• Federal government • Centers for Medicare and Medicaid Services

(CMS)• Healthcare reform

History Leading to MOC - Milestones

1970 – First time-limited certificate issued (ABFM) 1973 – Boards adopt recertification principles 1982 – Many board administer recert exams 1999 – Six general competencies (with ACGME) 2000 – Boards commit to MOC and 4 components 2006 – Boards receive approval of MOC programs 2009 – New standards to synchronize 24 boards

(developed by Committee on Monitoring Maintenance of Certification (COMMOC))

Institute of Medicine Quality Reports

Patient welfare firstPatient welfare first

Patient autonomyPatient autonomy

Social justiceSocial justice

Need for Continuous Physician Assessment

On average, clinical skills tend to decline over time Amount of clinical experience does not necessarily lead

to better outcomes or improvement of skills• Choudhry, N.K., R.H. Fletcher, and S.B. Soumerai, Systematic Review: The Relationship

between Clinical Experience and Quality of Health Care. Annals of Internal Medicine, 2005. 142(4): p. 260-273.

Fewer than 30% of physicians examine their own performance data.

• Audet, A.-M.J., et al., Measure, Learn, And Improve: Physicians' Involvement In Quality Improvement. Health Affairs, 2005. 24(3): p. 843-853.

A physician’s ability to independently and accurately self-assess and self-evaluate is poor.

• Davis, D.A., et al., Accuracy of Physician Self-assessment Compared With Observed Measures of Competence. JAMA: Journal of the American Medical Association, 2006. 296(9): p. 1094-1102.

Choudhry NK, Ann Intern Med, 2005;142:260-73

“Point in Time Mastery” is Not Enough

Lower Performance All Outcomes

MOC Guiding Principles

Its purpose will be to assure

high standards for patient care It will provide physicians with the means to

continually assess and improve their abilities It will assure stakeholders that physicians are

being assessed by reliable and valid measures It will be transparent to public and communicate

information about physicians’ competence

ABMS/ACGME Competencies

Practice-based

Learning & Improvement

1

Interpersonal &

Communication Skills

5

Professionalism

6

Medical Knowledge

4

Patient Care

2

Systems-based

Practice

3

Development of MOC Program- Four components

1. Professional standing (licensure) • Hold a valid, unrestricted medical license

2. Lifelong learning and self-assessment• Evidence of participation

3. Cognitive expertise (examination)• Covers the scope and range of the discipline• Is clinically relevant

4. Practice performance assessment• Proven scientific, educational and assessment methodology• Reflects patient care and should result in quality

improvement

ABIM Assessments (initial)

Lifelong learning• Medical Knowledge

• Medical society products accepted• Procedural Skills

Cognitive expertise• Discipline specific

Practice Performance• Practice improvement modules

• Practiced-based learning and improvement• Systems-based practice

• Interpersonal communication skills• Patient and peers

Which of the following is the most likely diagnosis?(A) Chronic bronchitis(B) Bronchiectasis(C) Asthma(D) Pulmonary fibrosis(E) Left ventricular failure

Question 20

Click icons on lung fieldsfor audio

Could this simulator be used for testing as well as training?

SimSuite® (Medical Simulation Corporation)

Practice Improvement Module (PIMSM)

Performance Report

Performance Report

Improvement

Chart review Patient survey

Impact

plan

do

study

act

Practice survey

Based on Picker patient and CAHPS

surveys

Based on Wagner’s Chronic Care Model & IHI’s Idealized

Office Design

Evidence-based guidelines

Physicians Achieving Significant Change

Target Measure Category

(Mean re-measurement N=31 patients) Number of physicians

Mean Δ

Blood Pressure or Lipid Control 52 + 28%

Medication Selection/Adherence 12 + 33%

Non-pharmacological Treatment/Self-care Support

69 + 50%

Patient Evaluation & Testing 35 + 37%

Review of Hypertension PIM re-measurement results for general internists (115) and subspecialists (53)

New 2009 MOC Standards

A more continuous & comprehensive process Integrate the patient’s voice Supports transparency to the public Address patient safety Address communication skills and

professionalism Include assessment of cognitive skills Incorporate quality improvement

New Standards - Public Benefits

Improve quality of care and patient safety Objective criteria for evaluating a

physician’s performance Improved ability of public to make well-

informed healthcare choices Give patients a voice

New Standards - Physician Benefits

Certification is a professional value Physician is helped in pursuit of lifelong learning Alignment with external environment

• Reduce redundancy - Streamline credentialing & privileging

• Reduce malpractice costs• Align with other strategies for physician accountability

(e.g., P4P)

Examine Current MOC Program

What is the value of certification? What do patients think of certification? What is the rate of participation? What do physicians think of MOC? What do physicians think of the products? What is the evidence?

Value of ABIM Certification

Although voluntary, more than 87% of U.S. doctors are certified

Almost 90% of time-limited doctors recertify

Gallup survey demonstrates that patients value certification

Other entities –health plans, hospitals, medical groups, licensing boards, other countries – are using certification and/or components

Physicians eligible to claim CME (AMA PRA Category 1 Credit™)

Public Expects Physician Evaluation

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Re-evaluatedevery so

many years

Periodicallypass a written

test ofknowledge

Have highsucces ratesfor diseasestreated most

often

Not ImportantLess ImportantNeutralImportantVery Important

How important is it that doctors have the following…

MOC Completion 1990-1998 Cohorts

Lipner et al, Ann Intern Med, 2006; 144(1):29-37.

Maintain/improve professional image

Update knowledge Maintain/improve patient

care or safety Personal preference or

interest Required for employment

Takes too much time Too expensive Not required for

employment Requirements

unreasonable No monetary benefit

Participate Not Participate

Attitudes about MOC -- Reasons to…

Physicians’ Attitudes Towards MOC

Part 2: Self-Evaluation Process was valuable learning experience• 77% Strongly agree or agree

Part 3: Satisfied with test experience • 83% Strongly agree or agree

Part 4: Performance Improvement Module

• 73% changed their practice as a result of completing the module

• 82% would recommend the PIM to a colleague

Physician Reactions After Completing Part 4 I didn’t know how I was doing. I am surprised by what I’m missing. We can do something about this! Small changes can make a big difference. Patient survey & chart audit were more valuable

than I thought – I learned a lot. An EMR with registry function can help with

repeated measurement.

Physician Reactions (continued)

“Ah-Ha” – I thought I was doing better• Powerful – believe data

Want to change, but don’t know how• What is “System-based practice”?• What is rapid-cycle test of change?

Emotion• Anger – “ABIM has no business being in my

business.”• Fear – “I’ll uncover performance data that will

hurt me.”• Gratitude – “Thanks for giving me a way to

know and improve practice.”

Studies Linking Certification/MOC and Quality

Better outcomes & more reliable care JAMA, 2004, Vol. 292, pp.1038-43

15% less mortality in myocardial infarction Acad. Med., 2000, Vol. 75, pp. 1193-98

Higher rates of preventive service (Mammography, hemoglobin A1c monitoring, influenza vaccination)

JAMA, 2005, Vol. 294, pp. 473-81

40% less mortality in colon resection Surgery, 2002, Vol. 132, pp. 663-70

Higher rates of prescription of aspirin and beta blocker after MI

JGIM, 2006, Vol. 21(3), pp. 238-244

Higher rates on diabetes care process measures for Medicare beneficiaries

Arch Intern Med, 2008, Vol. 168(13), pp. 1396-1403

Implementation - Planning:Committee Structure Oversight

Steering committee• Outside consultant• Project management

Board meetings/conference calls• Executive committee• Entire board

Internal Staff Committee Structure

Program Architecture• Conceptual design of CMOC• Product strategy to fulfill milestone contents

Finance• Fee structure• Incentives/behavior

Communications & Engagement• Stakeholder engagement• Web information

Early Initiatives & Evolutionary Principles• Automatic Enrollment• Multimedia to exams

Challenges for ABMS MOC

Communication about program changes Resistance to a more continuous process Many disciplines under internal medicine

• Specific tool development costly

Infrastructure changes – e.g., fees Rigorous, scientifically sound assessments

• Patient care, performance assessment

Research demonstrating validity (difficult, costly) Increasing transparency – appropriate level

Challenges (continued)

Degree of interaction with “other” stakeholders Public and government engagement Politics: Healthcare reform and meaningful use Complexity in quality arena …

Health Care Quality RelationshipsPurchasers (Government, Purchasers (Government,

Employers, Leapfrog)Employers, Leapfrog)Purchasers (Government, Purchasers (Government,

Employers, Leapfrog)Employers, Leapfrog)Insurers HC-Insurers HC-Plans CMSPlans CMSInsurers HC-Insurers HC-Plans CMSPlans CMS

Patients

QualityQuality

HospitalHospitalHospitalHospital JOINT JOINT COMMISSIONCOMMISSION

JOINT JOINT COMMISSIONCOMMISSION

NCQANCQANCQANCQA

ABMSABMSABMSABMS

FSMBFSMBFSMBFSMB

Consumer AdvocatesConsumer Advocates

Specialty Specialty SocietiesSocietiesSpecialty Specialty SocietiesSocieties

PharmaciesPharmacies LaboratoryLaboratoryLaboratoryLaboratory

AQA/NQF AQA/NQF MeasuresMeasuresAQA/NQF AQA/NQF MeasuresMeasures

Academic Academic EntitiesEntities

Academic Academic EntitiesEntities

ConsultantConsultantConsultantConsultant

Research (RWJ, Research (RWJ, RAND, AHRQ)RAND, AHRQ)

Research (RWJ, Research (RWJ, RAND, AHRQ)RAND, AHRQ)

ACGMEACGMEACGMEACGMECare Care ManagementManagement

Care Care ManagementManagement

ACCMEACCMEACCMEACCME

Physicians

QIOsQIOsQIOsQIOs

HIT VendorsHIT VendorsHIT VendorsHIT Vendors

Thank you!

Comments and Questions?