-
EducationalOutcomes Report
July 22, 2011
Grant # 8060
Accredited provider of medical and professional education
8201 W McNab Road, Tamarac, FL 33321 Phone: (954) 718-6055 Fax:
(954) 718-6013 www.primeinc.org
Copyright © 2011 PRIME Education, Inc. All Rights Reserved.
This program was sponsored by PRIME® through an independent
educational grant from Teva Neuroscience.
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 2Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Table of ContentsExecutive Summary and Program Description
.......................................................................................................
3
Expert Faculty
......................................................................................................................................................................
5
Level 1: Participant Demographics
.............................................................................................................................
6
Level 2: Participants’ Assessments of the Educational Activity
.........................................................................
6
Level 3: Knowledge Outcomes
......................................................................................................................................
11
Continuous Needs Assessment
....................................................................................................................................
13
Summary, Conclusions, and Future Directions
.......................................................................................................
15
Appendix:
Educational Needs Assessment, Activity Development, and Outcomes
Assessment at PRIME®: Ap-
plications of Established Conceptual Frameworks and Principles
of Adult Learning ............................... 16
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 3Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Executive Summary and Program DescriptionThe medical management
of patients with multiple sclerosis (MS) continues to undergo rapid
evolution. While injectable disease-modifying therapies (DMTs) for
MS have been the treatment mainstay over the last two decades, the
first oral DMT for MS was approved by the FDA in 2010. In addition,
other oral agents, as well as targeted monoclonal antibodies, are
currently in late stages of clinical development. Thus, the coming
months and years promise exciting changes as well as ongoing
challenges for neurology practices. Continuing education about the
efficacy and risks of emerging MS therapies is pivotal meeting
these challenges and for ensuring optimal outcomes for patients. To
address the current knowledge needs of neurology-based MS care
providers, PRIME® sponsored a series of educational programs that
highlighted key research findings from the 63rd annual meeting of
the American Academy of Neurology (AAN), which was held in Honolulu
in April 2011. Intended for neurologists and their treatment teams,
the programs presented the most up-to-date information on new
agents for MS and their potential role in the MS management
paradigm.
This document reports the outcomes of the PRIME®-sponsored
educational series on the 2011 AAN meeting. The body of the report
is organized by distinct levels of educational outcomes, with
focused assessments of:
• Participant demographics
• The teaching effectiveness of the faculty
• The quality of the learning experience
• Activity-influenced gains in understanding and confidence
Explanations of these outcome domains, along with their
evidence-based rationale, are presented in the appendix document
titled Educational Needs Assessment, Activity Development, and
Outcomes Assessment at PRIME®: Applications of Established
Conceptual Frameworks and Principles of Adult Learning.
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 4Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
1.5 Hours LIVE CME
There is no fee for this activity as it is sponsored by PrIME®
through an independent educational grant from Teva
Neuroscience.
Stephen Krieger, MD Assistant Professor of Neurology Mount Sinai
Medical Center Corinne Goldsmith Dickinson Center for MS Mount
Sinai Hospital New York, NY
Clyde E Markowitz, MD Director Multiple Sclerosis Center
University of Pennsylvania Associate Professor of Neurology
University of Pennsylvania School of Medicine Philadelphia, PA
Robert A Bermel, MD Associate Staff Mellen Center for Multiple
Sclerosis Cleveland Clinic Cleveland, OH
Aaron E Miller, MD Medical Director Corinne Goldsmith Dickinson
Center for Multiple Sclerosis Professor of Neurology Mount Sinai
School of Medicine New York, NY Lansing, MI
Omar Khan, MD Professor of Neurology Director, Wayne State
University Multiple Sclerosis Center Director, MR Image Analysis
Laboratory Program Director, Neuroimmunology and Multiple Sclerosis
Fellowship Wayne State University School of Medicine, Detroit
Director, National MS Society Comprehensive Care Multiple Sclerosis
Clinic Harper University Hospital, The Detroit Medical Center
Detroit, MI
Burk Jubelt, MD Professor of Neurology Division Chief, Neurology
Department of Neurology at SUNY Upstate Medical University
Syracuse, NY
Guy J Buckle, MD, MPH Director of MS Clinical Care Partners
Multiple Sclerosis Center Brigham and Women’s Hospital Assistant
Professor of Neurology Harvard Medical School Boston, MA
Claire S Riley, MD Instructor in Neurology Yale MS Center New
Haven, CT
Tom Scott, MD Director Allegheny Multiple Sclerosis Treatment
Center Professor of Neurology Director of the Medical Residency
Training Program and Medical Student Clerkship Allegheny General
Hospital Drexel University College of Medicine Pittsburgh, PA
Carlo Tornatore, MD Associate Professor Department of Neurology
Director, Multiple Sclerosis Center Georgetown University Hospital
Director, Neurology Residency Program Director, Neurology Clerkship
Georgetown University Medical Center Washington, DC
Warren L Felton III, MD Chair Division of Neuro-Ophthalmology
Virginia Commonwealth Medical Center Richmond, VA
Fred D Lublin, MD Saunders Family Professor of Neurology
Director, The Corinne Goldsmith Dickinson Center for Multiple
Sclerosis Mount Sinai School of Medicine New York, NY
Susan Gauthier, DO, MPH Assistant Professor of Neurology
Assistant Attending Neurologist NewYork-Presbyterian Hospital Feil
Family Clinical Scholar in Multiple Sclerosis Weill Cornell Medical
College
Registration web page
Program Title:
Highlights in Multiple Sclerosis Treatment from the 2011 AAN
Meeting
Accreditation:
1.5 ACCME, AANP
Print invitation (Front)
Print invitation (inside)
Learning ObjectivesNewest evidence in emerging therapies in MS
as presented at AAN ÂUtility of new therapies in MS treatment in
the absence of guidelines ÂComplications of managed care in the
absence of guidelines ÂPotential changes to neurology practice in
monitoring patients who are receiving Ânewer therapies
v
Intended Audience: Physicians, physician assistants, and nurse
practitioners
Accreditation StatementsPhysician Accreditation Statement
PRIME Education, Inc. (PRIME®) is accredited by the
Accreditation Council for Continuing Medical Education to provide
continuing medical education for physicians.
PRIME® designates this educational activity for a maximum of 1.5
AMA PRA Category 1 Credit™. Physicians should only claim credit
commensurate with the extent of their participation in the
activity.
Physician Assistant Accreditation StatementAAPA accepts AMA
Category 1 CME Credit™ for the PRA from organizations accredited by
ACCME.
Nurse Accreditation StatementPRIME Education, Inc. (PRIME®) is
accredited by the American Academy of Nurse Practitioners as an
approved provider of nurse practitioner continuing education.
Provider number: 060815. This
program is accredited for 1.5 contact hours, which includes 1.0
hour of pharmacology. Program ID# 35LV1100AAN.
This program was planned in accordance with AANP CE Standards
and Policies and AANP Commercial Support Standards.PRIME®
designates this activity for 1.5 contact hour.
CME/CE credit awards require attendance at the entire program
and completion of learner assessment tools, after which the learner
can print a certificate of completion.
Register Online at: www.primeinc.org/AAN
Registration & Dinner commence at 6:30 PMEducational
Program: 6:45 PM – 8:15 PM
Locations Invited Expert Faculty
BaltimoreCinghiale
Baltimore, MD 21202Carlo Tornatore, MD
RichmondMorton’s
Richmond, VA 23219Warren L Felton III, MD
SyracuseArad Evans Inn
Fayetteville, NY 13066Burke Jubelt, MD
BostonMorton’s
Boston, MA 02116Guy J Buckle, MD, MPH
ProvidenceCamille’s
Providence, RI 02903Syed A Rizvi, MD
HartfordMorton’s
Hartford, CT 06103Claire S Riley, MD
Worcester111 Chop House
Worcester, MA 01604James M Stankiewicz, MD
Birmingham, MIThe Townsend Hotel
Birmingham, MI 48009Omar Khan, MD
Lansing, MIMitchell’s Fish Market
Lansing, MI 48912Omar Khan, MD
ClevelandHyde Park Prime
Cleveland, OH 44115Robert A Bermel, MD
ColumbusLindey’s
Columbus, OH 43206Aaron L Boster, MD
Manhattan (NYC)Morton’s
New York, NY 10176Stephen Krieger, MD
Long IslandBlackstone
Melville, NY 11747Lauren B Krupp, MD
West OrangeHighlawn Pavilion
West Orange, NJ 07052Stuart Fox, MD
RochesterWoodcliffe Hotel
Fairport, NY 14450Susan Gauthier, DO, MPH
PittsburghMorton’s
Pittsburgh, PA 15222Tom Scott, MD
PhiladelphiaUnion Trust
Philadelphia, PA 19106Clyde E Markowitz, MD
ArlingtonMorton’s
Arlington, VA 22202Carlo Tornatore, MD
* Parking fees are the responsibility of the attendee.
Conflict of interest information is available on the program
registration page at www.primeinc.org.
PRIME® is committed to providing access to our CME programs for
individuals with disabilities as identified in Title III of the
American with Disabilities Act (ADA) for live programs. If you use
assistive technology (ie, Braille reader, screen reader, TTY) and
the format of any material on the PRIME® website’s program
registration pages, CME test/evaluation pages, or other areas
interfere with your ability to access the information, please
contact [email protected] or 954-718-6055 for assistance.
Program Dates & Locations:
April 25, 2011 West Orange, NJ
May 4, 2011 Philadelphia, PA
May 5, 2011 Cleveland, OH
May 11, 2011 New York, NY
May 12, 2011 Fayetteville, NY
May 16, 2011 Providence, RI
May 17, 2011 Boston, MA
May 23, 2011 Richmond, VA
May 23, 2011 Baltimore, MD
May 24, 2011 Rochester, NY
June 6, 2011 Pittsburgh, PA
June 7, 2011 Columbus, OH
June 8, 2011 Arlington, VA
June 9, 2011 Hartford, CT
June 13, 2011 Melville, NY
June 14, 2011 Lansing, MI
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 5Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Expert Faculty Robert Bermel, MDAssociate StaffMellen Center for
Multiple SclerosisCleveland ClinicCleveland, OH
Aaron Boster, MDAssistant Professor of NeurologyDivision of
ImmunologyOhio State University Multiple Sclerosis
CenterCo-Director, Neurology Infusion CenterDirector, Multiple
Sclerosis Center Clinical ResearchMoorehouse Multiple Sclerosis
CenterColumbus, OH
Guy Buckle, MD, MPHDirector of MS Clinical Care Partners
Multiple Sclerosis CenterBrigham and Women’s HospitalAssistant
Professor of NeurologyHarvard Medical SchoolBoston, MA
Eric Eggenberger, DO, MSEpi, FAANProfessor, Vice-Chairman,
Department of Neurology and Ophthalmology, Colleges of Osteopathic
Medicine and Human MedicineDirector, Michigan State University
Ocular Motility LaboratoryDirector, National Multiple Sclerosis
Society Clinic, Michigan State UniversityEast Lansing, MI
Warren Felton, MDChairDivision of Neuro-OphthalmologyVirginia
Commonwealth Medical CenterRichmond, VA
Andrew Goodman, MDProfessor of NeurologyDirector, Multiple
Sclerosis CenterChief, Neuroimmunology UnitDepartment of
NeurologyUniversity of Rochester Medical CenterRochester, NY
Stephen Krieger, MDAssistant Professor of NeurologyMount Sinai
Medical CenterCorinne Goldsmith Dickinson Center for MSMount Sinai
HospitalNew York, NY
Burk Jubelt, MDProfessor of Neurology, Microbiology/Immunology
and Neuroscience Division Chief, NeurologyDepartment of Neurology
at SUNY Upstate Medical UniversitySyracuse, NY
Fred Lublin, MDSaunders Family Professor of NeurologyDirector,
The Corinne Goldsmith Dickinson Center for Multiple SclerosisMount
Sinai School of MedicineNew York, NY
Clyde Markowitz, MDDirectorMultiple Sclerosis CenterUniversity
of PennsylvaniaAssociate Professor of NeurologyUniversity of
Pennsylvania School of MedicinePhiladelphia, PA
Aaron Miller, MDMedical DirectorCorinne Goldsmith Dickinson
Center for Multiple SclerosisProfessor of NeurologyMount Sinai
School of MedicineNew York, NY
Claire Riley, MDAssistant ProfessorYale School of
MedicineNeurology Multiple Sclerosis and NeuroimmunologyNew Haven,
CT
Robert Shin, MDAssociate Professor of NeurologyAssociate
Professor of Ophthalmology and Visual SciencesUniversity of
Maryland School of MedicineBaltimore, MD
Carlo Tornatore, MDAssociate ProfessorDepartment of
NeurologyDirector, Multiple Sclerosis CenterGeorgetown University
HospitalDirector, Neurology ResidencyProgram Director, Neurology
ClerkshipGeorgetown University Medical CenterWashington, DC
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 6Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Level 1: Participant Demographics 293 Registered Attendees
239 Program Participants
102 Earning CME/CE Credit
By discipline, the distribution was as follows:
• 160 Neurologists/Immunologists
• 22 Nurse Practitioners
• 14 Physician Assistants
• 43 Specialty Nurses
*The intended audience for this grant was 15 neurologists, nurse
practitioners, physician assistants and specialty nurses at each
CME activity. Therefore, the grant goals of intended audience mix
and audience size were met.
Level 2: Participants’ Assessments of the Educational
ActivityFollowing the educational activity, participants evaluated
various aspects of its effectiveness and accessibility. PRIME® uses
these outcomes to determine the extent to which its activities fill
identified educational gaps and address critical learning
objectives. In addition, participants’ assessments of educational
activities inform our future needs assessments and curriculum
development. Evaluations were completed by 102 participants.
To assess participants’ views on the value of CME, we asked the
following question on a post-activity survey:
To what extent do CME activities like this one provide greater
value to your clinical practice than promotional programs?
a. Limited additional value
b. Moderately greater value
c. Significantly greater value
As illustrated in Figure 1, the majority of participants (54%)
stated CME activities
Figure 1. Participants’ self-reported value of CME activities
vs. promotional programs to their clinical practice.
8%
38%
54%
0%
10%
20%
30%
40%
50%
60%
Limited value Moderate value Significant valuePa
rtici
pant
s Cho
osin
g Re
spon
se (%
)
Reported Immediately After Activity
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 7Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
provide significantly greater value to their clinical practice
than promotional programs.
Effectiveness in Meeting Learning Objectives
This educational activity was designed to support participants
in achieving the following learning objectives.
1. Summarize new science and evidence from the AAN 2011 meeting
pertaining to multiple sclerosis therapy
2. Debate the relative utility of new therapy in the MS
treatment paradigm in the absence of guidelines
3. Evaluate changes to practice that may be required to
effectively monitor patients receiving newer therapies
Participants assessed the educational activity’s effectiveness
in meeting the learning objectives. On a 5-point scale (with 5
indicating the highest ranking), the combined mean average was 4.6
(Figure 2).
Figure 2. Participants’ assessments of how effectively the
educational activity met its learning objectives.
4.6
4.5
4.7
1 2 3 4 5
Learning objective #3
Learning objective #2
Learning objective #1
Average Ratings
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 8Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Participants assessed the educational activity’s effectiveness
in meeting the learning objectives. On a 5-point scale (with 5
indicating the highest ranking), the combined mean average was 4.6
(Figure 2).
Faculty Ratings
Figure 3 presents participants’ ratings of the knowledge,
expertise, and presentation skills of the faculty. On a 5-point
scale (with 5 indicating the highest rating), the average rating
for these measures of faculty teaching effectiveness was 4.9.
Additional Participant Ratings
As illustrated in Figure 4, participants gave high ratings of
the educational activity’s accessibility, time allotment, rigor,
and adherence to adult learning principles.
Figure 3. Participants’ assessments of faculty teaching
effectiveness. Rating scale: 1 = lowest rating; 5 = highest
rating.
Figure 4. Participants’ assessments of the accessibility,
effectiveness, and rigor of the educational activity. Rating scale:
1 = lowest rating; 5 = highest rating.
4.9
4.8
4.9
1 2 3 4 5
Knowledge andexpertise
Ability to conveysubject matter
effectively
Ability to deliverobjective and balanced
program
Average Ratings
4.6
4.7
4.8
4.7
4.7
1 2 3 4 5
Time allotted forprogram
Principles of adultlearning
Registration process
Scientific rigor
User friendliness
Average Ratings
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 9Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Fair Balance
In a post-activity questionnaire, 100% of participants indicated
that the educational content was objective and unbiased.
Participants’ Open-Ended Comments
In an online survey delivered following the educational
activity, participants were given the opportunity to comment freely
on its strengths and weaknesses. Representative comments are
reproduced as follows.
This program provided the best overall assessment of current
therapies for multiple sclerosis of the many programs over the
years. — Physician
An informative, well-organized program. An outstanding speaker
with both extensive clinical experience and up-to-date scientific
knowledge with the rare ability to make esoteric research data both
meaningful and easy to understand. — Physician
This is one of the best CME lectures I’ve been to. The speaker
is excellent! - Physician
An excellent overview of MS treatments, and it’s exciting to
learn about “pipeline” drugs. — Nurse Practitioner
The program was very informative but needed additional time —
Physician
This program fit my needs perfectly with regard to the
timeliness of the information and how easily it was for me to fit
it into my work schedule. It provided me CME credits instead of
just being a marketing tool for a particular product. —
Physician
As usual this was an excellent program. The speaker was
exceptional, the reminders via email & phone helped to fill the
room, and many participants commented on how helpful the reminders
were. Keep up the good work. — Specialty Nurse
A great program, but the venue and food were not that great. —
Physician
A fantastic program, great venue, well-organized, and very
comfortable. — Physician
A lot of material was covered in a short period of time. I
appreciated having the slide deck ahead of time. — Nurse
Practitioner
The program was excellent. The speaker is an excellent presenter
and very knowledgeable. The room was too small and the restaurant
staff was not prepared for the amount of people who attended the
program. There should be a wider range of food choices. Servers
were very unprofessional and rude. — Specialty Nurse
I learned a great deal about the new science and evidence from
the AAN 2011 meetings from this program. — Physician
An excellent program given at an excellent time due to current
circumstances and changes in practice at the various neurology
offices. — Specialty Nurse
A nice way to have “attended” the AAN in Hawaii with respect to
MS. I would have liked updates on diagnosis, especially biomarkers.
— Physician
I loved the way the speaker presented the data. Very informative
and I learned more from this than any of the other MS CME I have
attended — Physician
The speaker is obviously very well informed regarding MS, and I
thought he gave both the pros and cons of the drug regimens. —
Nurse Practitioner
A great deal of material was summarized in the time allotted.
Perhaps either less material could have been reviewed, or a longer
period of time could have been allotted for the lecture. —
Physician
The speaker is very clear, concise, and his explanation was very
informative. Staff members were very helpful and professional. —
Physician
A very nice presentation. The speaker did an excellent job (as
always). — Physician
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 10Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
The speaker is outstanding, very well informed. I took away a
tremendous amount from this program. — Nurse Practitioner
The room was too small with cramped space at tables. I would
also like copy of slides from this presentation to review. — Nurse
Practitioner
The program was excellent. The speaker is an excellent
presenter. The environment was warm and friendly. — Specialty
Nurse
The speaker gave a wonderful presentation. — Physician
I enjoyed and learned a great deal at this educational meeting
—Physician
The room set up made it difficult to see the slides from all
seats. — Specialty Nurse
A brief clinical overview of MS prior to delving into the more
complex therapy regimens would have been beneficial. —
Physician
The speaker was well-informed, open and encouraging of
questions. An excellent opportunity. — Nurse Practitioner
The program facilitator was excellent, as was the speaker. —
Physician
A summary information sheet with name of drug, action of drug,
side effect, route of administration and indication when to pick
over the available drugs would help. — Physician
A timely and very welcome program. — Physician
A summary handout would help as there are too many new drug
names and studies to keep them all straight without a written
guide. —Physician
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 11Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Level 3: Knowledge OutcomesBefore and after the educational
activity, participants answered a series of multiple-choice
questions designed to evaluate gap-targeted knowledge outcomes.
Learning Insights, Values, and Intentions to Change
Practices
As reflected by the following pre-activity/post-activity
questions, this sub-level of knowledge outcomes addresses the
influences of educational interventions on participants’
self-reported understanding of essential topics and their
confidence in performing key clinical skills. In addition, at this
sub-level of knowledge outcomes, we assess learners’
activity-influenced intentions to change clinical and/or management
practices.
How would you describe your current understanding of new and
emerging oral therapies for MS?
a. Limited
b. Adequate
c. Good
d. Excellent
Figure 5 presents the percentages of participants reporting
different levels of understanding regarding oral MS therapies. The
greatest magnitudes of pre-activity to post-activity change were
reported for “limited” understanding, which decreased from 44% to
16% of participants and “good” understanding, which increased from
24% to 44% of participants.
Figure 5. Participants’ self-reported understanding of new and
emerging oral therapies for MS.
44%
29%
24%
3%
16%
29%
44%
11%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Limited Adequate Good ExcellentPa
rtici
pant
s Cho
osin
g Re
spon
se (%
)
Reported Prior to Activity Reported Immediately After
Activity
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 12Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Figure 6. Participants’ self-reported understanding of new
monoclonal antibodies for treating MS.
Figure 7. Participants’ self-reported confidence in discussing
the benefits and risks of emerging MS therapies with their
patients.
How would you describe your current understanding of new
monoclonal antibodies for treating MS?
a. Limited
b. Adequate
c. Good
d. Excellent
Figure 6 presents the percentages of participants reporting
different levels of understanding about monoclonal antibodies for
MS. The greatest magnitudes of pre-activity to post-activity change
were reported for “ limited” understanding, which decreased from
53% to 18% of participants and “good” understanding, which
increased from 18% to 34% of participants.
How would you describe your current level of confidence in
discussing the benefits and risks of emerging MS therapies with
your patients?
a. Limited
b. Average
c. Good
d. Excellent
Figure 7 presents the percentages of participants reporting
different levels of confidence in their ability to discuss the
benefits and risks of emerging MS therapies with their patients.
The greatest magnitudes of pre-activity to post-activity change
were reported for “limited” confidence, which decreased from 48% to
19% of participants and “good” confidence, which increased from 18%
to 38% of participants.
53%
26%
18%
3%
18%
36% 34%
12%
0%
10%
20%
30%
40%
50%
60%
Limited Adequate Good ExcellentPa
rtici
pant
s Cho
osin
g Re
spon
se (%
)
Reported Prior to Activity Reported Immediately After
Activity
48%
31%
18%
3%
19%
30%
38%
13%
0%
10%
20%
30%
40%
50%
60%
Limited Adequate Good Excellent
Part
icipa
nts C
hoos
ing
Resp
onse
(%)
Reported Prior to Activity Reported Immediately After
Activity
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 13Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Continuous Needs Assessment Our continuous needs assessment is
based on questions that participants asked the expert faculty
presenters during and after their presentations. Representative
questions are listed as follows:
Is there a higher risk of cancer in patients with MS in
general?
Would you treat a patient with CIS even with a clean MRI?
Would you treat RIS?
If a patient has a relatively quiet clinical course of MS in the
first 3-5 years, do they have a better prognosis 20 years
later?
How many people with benign MS really never have MS?
Can you comment on current CCSVI research and status in the
US?
How long can a patient take dalfampridine?
What is the primary mechanism of action of laquinimod?
What is the most important side effect of alemtuzumab?
When should patients be referred to an MS specialist?
In a newly diagnosed MS patient, what are your key
considerations in choosing a first line therapy?
What percentage of patients present with spinal cord
lesions?
Where does the JC virus “hide” in the body?
Is teriflunomide safer in men?
Do you think cladribine will be approved?
Do you think being mentally active slows
down development of brain atrophy?
Is there a strong correlation between cognitive function and
brain atrophy?
Why do some patients do not respond to dalfampridine?
Do you ever use natalizumab as a first line treatment
option?
Are there current clinical trials evaluating vitamin D in
MS?
How many attacks need to occur before you consider a treatment
to be a “failure”?
What are the key side effects of teriflunomide?
What is important about the ratio of CD4+ and CD8+ cells?
What future medicine do you feel is going to work best in MS
patients?
Where was the ocrelizumab phase II study conducted?
Where is fingolimod in your algorithm for treating patients with
RRMS?
Do you have patients that you want to go on fingolimod because
not doing well on an injectable agent?
How are you treating secondary progressive patients?
Aren’t there 2 forms of JCV – does the current test pick up
both?
Is it possible to convert from JCV negative to positive as you
age?
What would you do for your patient who has a history of
immunosuppression?
Can you use atropine in patients taking fingolimod?
Why is the thyroid particularly affected by alemtuzumab?
What is the current list price of fingolimod?
Are there any predictors of survival when you get PML?
What is your general rule regarding how long to keep a patient
on natalizumab?
After a drug holiday from natalizumab do you start patients on
interferons or glatiramer?
Is there a cheap easy way to measure neutralizing antibodies to
IFNs?
Did the researchers look at neutralizing antibodies in the
REFLEX study?
When were the blood samples taken in the MS patients on
natalizumab you showed in the study?
Are doctors pulling their patients off of natalizumab after two
years of therapy?
Do you think drug holidays are a good idea for patients on
fingolimod?
In the original trials on dalfampridine, can you describe in
more detail the responder analysis?
Do you supplement your MS patients with Vitamin D?
Have you been fooled in your diagnosis with a patient that has
presented with spinal cord lesions?
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 14Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Does dalfampridine have any significant interactions with other
drugs?
How do you encourage and convince your patients to start
treatment?
Will lymphocytes eventually die in patients taking
fingolimod?
What was the duration of the phase III trial of
teriflunomide?
Is there a direct relationship between brain atrophy and
disability?
Is laquinimod being evaluated for any other diseases?
Is there any new data on adjunct therapy?
What is the average age most MS patients start clinical
trials?
What is the youngest patient you have seen with MS?
How often should antibodies to IFN be assayed?
Could laquinimod be used as an add-on therapy?
Who is an ideal candidate for laquinimod?
Can you comment on whether you think the dosing of cladribine
could be adjusted to be safer?
Is there any truth to the hypothesis that sugar substitutes may
cause MS?
Can teriflunomide be used for primary progressive MS?
Are black holes evaluated in clinical trials of oral agents?
What type of release forms do patients have to sign before
entering a clinical trial?
Is the reduction of relapse rates better with any of the oral
agents?
Is natalizumab ever used as a first line agent?
Do you think in the future we will almost exclusively use oral
agents?
How long does it take to show a response on dalfampridine?
Do any clinical trials evaluate patients with severe disability
as measured on EDDS?
What are the main safety risks with dalfampridine?
How frequently would you test patients for JCV while they were
taking natalizumab?
Have you placed any of your newly diagnosed patients on
fingolimod?
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 15Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Summary, Conclusions, and Future DirectionsPresented immediately
in the weeks following the 2011 AAN annual meeting, this series of
educational programs highlighted the most up-to-date and important
findings from studies on approved, new, and emerging MS therapies.
The programs were attended by nearly 250 neurology-based health
professionals, including 160 neurologists/immunologists.
The main outcomes of the educational activity are summarized as
follows.
• More than half (54%) of all attendees responded that the CME
activity provided greater value to their clinical practice than did
promotional programs; 38% identified the CME program as providing
“moderate value” over promotional programs.
• Attendees open-ended comments provided revealing attitudes
about the CME activity including:
• Many reported that the dinner program was an efficient and
effective way for them to be updated on AAN without costing time
and travel to attend the Hawaii meeting
• There were complaints about the size of the venue rooms and
the lack of syllabi – two areas directly affected by the reduced
budget awarded to PRIME. While the quality of the content and
recruitment efforts remains consistently high, lesser funding is
affecting banquet services and handouts.
• There were many opt-in comments about the strength and
preparation of the faculty.
• Participants gave high ratings for the teaching effectiveness
of the faculty and the extent to which the learning objectives were
successfully achieved.
• In open-ended comments, a large number of participants
commended the topical relevance of the educational program.
• As assessed through pre-activity/post-activity survey
questions, many participants reported gains in understanding about
emerging oral therapies and monoclonal antibodies for MS. In
addition, participants reported gains in their confidence in
educating patients about the risk-benefit profiles of the emerging
medications.
As reflected by the representative questions that participants
asked the faculty presenters, participants expressed great interest
in learning more about how the new and emerging MS therapies may
eventually fit into future treatment algorithms.
60-day post-impact data for this program can be provided upon
request.
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 16Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Laurence Greene, PhD
Senior Medical Writer, PRIME Education, Inc.
Chris R Prostko, PhD
Scientific Program Director, PRIME Education, Inc.
Established in the 1970s, the field of continuing medical
education (CME) addresses the concern that the last 30 to 40 years
of physicians’ and other healthcare providers’ careers may occur
without any formal course of study.1,2 This issue is currently
compounded by several factors, including (1) ongoing advancements
in biomedical science and technology; (2) the continual development
of novel therapies for emerging diseases; (3) evolution of models
for collaborative and interprofessional medical practices; and (4)
major changes in the infrastructure of our healthcare system.
Indeed, the knowledge and skills that healthcare professionals
acquire during their formative education may be obsolete within a
matter of years or, in some cases, even months. To ensure the most
successful outcomes for their patients, healthcare professionals
must therefore engage in progressive, high-quality, and career-long
education and skill training. Thus, CME can be an important element
in continuing professional development.
In serving the vital mission of continuing education (CE) and
CME for healthcare professionals, PRIME Education, Inc. (PRIME®)
operates on established conceptual frameworks and sound principles
of adult learning. This article describes PRIME®’s theory-guided
and evidence-based processes for educational needs assessment,
activity development, and outcomes assessment. The processes are
summarized in the schematic overview in Figure 1.
Educational Needs Assessment and Activity Development Informed
by Gap AnalysisThe success of any CME/CE activity depends on an
initial comprehensive assessment of learners’ needs.3,4 A logical
and productive approach to needs assessment is gap analysis, the
systematic process of identifying differences between:
(1) The healthcare professional’s current knowledge, competence,
and performance skills; and
(2) Established standards and criteria that must be achieved to
promote the highest quality clinical performance and optimal
patient outcomes.
As depicted in steps 1 and 2 of Figure 1, gap analysis directly
informs the development of learning objectives for educational
activities. Gap analysis is also fundamental to devising the
methods and tools for outcomes assessment and for developing
effective curricular strategies, media, and content. PRIME®
operates on the principle that these two processes—the design of
outcomes methodology and the development of educational
activities—must be tightly integrated. This complementary approach,
depicted in steps 3 and 4 of Figure 1, is essential for serving
learners’ needs and ensuring successful outcomes of CME/CE
activities.
Principled Approaches to Outcomes AssessmentToday’s leading
approaches to outcomes assessment in CME/CE have been largely
shaped by conceptual frameworks developed by Donald Kirkpatrick,5
George Miller,6 and Donald Moore and colleagues.7 Among other
shared features these frameworks are based on the principle that
the highest goals of adult education are achieved when learners
successfully apply new knowledge to solve problems and master
skills in their practice settings. Thus, outcomes assessment in
CME/CE must account for the extent to which health
professionals:
(1) Acquire essential information, ideas, and procedural skills
that target identified gaps and serve an educational activity’s
learning objectives;
(2) Demonstrate competent applications of the knowledge within
the educational setting; and
(3) Skillfully transfer the newly acquired knowledge to
practical settings, effectively closing the gaps that initially
motivated the educational intervention.
PRIME®’s pyramid model of outcomes assessment, adapted largely
from the recently refined framework of Moore et al,7 is presented
in Figure 2. The base of the pyramid represents outcomes of
participant demographics (level 1) and participants’ assessments of
the quality and effectiveness of educational activities (level 2).
Level 2 assessments are implemented through post-activity
questionnaires in which participants rate the effectiveness,
scientific rigor, and objectivity of the curriculum as well as the
knowledge, expertise, and presentation skills of the faculty. In
PRIME®’s continuous assessment model, data derived from level 2
Educational Needs Assessment, Activity Development, and Outcomes
Assessment at PRIME®: Applications of Established Conceptual
Frameworks and Principles of Adult Learning
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 17Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
Figure 1. An overview of PRIME®’s approach to needs assessment,
educational activity development, and outcomes assessment.
Post-activity assessments (steps 9 and 10) are conducted
immediately following educational programs and up to 60–180 days
later.
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 18Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
evaluations are essential for guiding future gap analyses and
for informing the development of new educational activities that
enable learners to achieve higher levels of knowledge, competence,
and performance. These applications are reflected in the feedback
loop from steps 9 and 10 to step 1 in Figure 1.
At level 3 of PRIME®’s pyramid model, pre-activity and
post-activity tests are administered to assess changes in
declarative (factual) knowledge and procedural knowledge, the
latter of which is defined as an expressed understanding of the
steps involved in carrying out healthcare practices. Learners’
gains in declarative and procedural knowledge are obviously
prerequisites to improving performance skills and, ultimately, to
ensuring successful patient health and community health outcomes.
This pivotal role of core knowledge is especially pertinent for
contemporary healthcare professionals, who continually face the
challenges of grasping complex new information and techniques in
the biomedical sciences. In addition to assessing learners’
declarative and procedural knowledge acquisition, PRIME® evaluates
the extent to which CME/CE activities influence self-reported
learning insights, values, and behaviors. Outcomes are assessed,
for example,
on the influences of CME/CE interventions on participants’
attitudes about educational topics, their intentions to change
practices in ways that meet established standards, and their
subsequent self-directed learning behaviors. Positive changes in
these important subjective learning domains are very often
correlated with improved clinical performance and a deeper
engagement in lifelong education.8-10
In keeping with the recent outcomes framework developed by Moore
et al,7 PRIME® defines competence (level 4) by how successfully
learners apply knowledge within the context of an educational
activity. For healthcare professionals, gains in competence are
thus reflected by such actions as direct applications of knowledge
to diagnosing disease; selecting, administering, and adjusting
therapies; and counseling and monitoring patients to ensure
medication adherence and to prevent medication-related problems. In
live and web-based educational settings, PRIME® assesses learner
competence through such educational design strategies as
performance simulations, practice-feedback sessions involving
patient encounters, peer-to-peer virtual town-hall symposia, and
technology-driven (eg, Unique Critique®) programs in which expert
faculty provide individualized, branching feedback
Figure 2. A model for assessing outcomes of continuing education
programs for health professionals. Adapted from Kirkpatrick,5
Miller,6 and Moore et al.7
-
Educational Outcomes Report: Highlights in Multiple Sclerosis
Treatment from the 2011 AAN Meeting 19Copyright © 2011 PRIME
Education, Inc. All Rights Reserved.
Grant #8060
to learners in a case-based question-and-answer format.11 PRIME®
also applies elements of these strategies to assess outcomes at the
level of performance (level 5).
The most pressing challenge for CME/CE providers is to support
healthcare professionals in transferring newly acquired knowledge
from educational contexts to practice settings, to promote
improvement in performance. The extent to which CME/CE influences
performance can be evaluated partly through subjective measures,
including post-activity surveys. For example, in a questionnaire
administered 60-180 days after an educational activity, PRIME®
assesses participants as to how frequently they have applied
gap-targeted knowledge in their recent practice, as well as what
new actions and interventions they are regularly performing in the
clinical setting that they were not performing prior to the CME/CE
activity. Though subjective measures, these responses assist PRIME®
in tracking the participant’s journey toward performance
improvement and in identifying potential new barriers that may
thwart the journey, lending important information in the gap
analysis.
The ideal approaches to assessing performance outcomes demand
direct and objective measures. However, CME/CE providers have
traditionally faced many logistical barriers, including
patient-privacy issues, in efforts to measure the
effects of educational activities on clinical performance. A
major initiative, called performance improvement CME (PI-CME), is
currently underway to address this problem, engaging maintenance of
certification programs.12,13 Through application of this
technology, PRIME® is evaluating the extent to which performance
can be measured and level 5 learning can be achieved. The ability
to track patient health outcomes (level 6) is an intended result of
this technology through patient registry data. The success of this
technology will hinge on many factors, not the least of which is
the significant time commitment required of learners to fully
engage in the PI-CME activity. As a result, PRIME® is also
establishing business partnerships with physician member societies
and government organizations, where outcomes of patient health and
community health may be identified and measured.
The ultimate goal of CME/CE is to support healthcare
professionals in closing targeted learning gaps to improve patient
health (level 6) and community health (level 7). At present,
logistical matters usually prohibit objective assessments of
outcomes at these highest levels. The potential impact of PI-CME in
providing data to assess patient and community health outcomes
remains to be determined through future applications and associated
educational research.
References1. Mansouri M, Lockyer J. A meta-analysis of
continuing medical education effectiveness. J Contin Educ Health
Prof. 2007;27:6-15.
2. Mazmanian PE. Reform of continuing medical education in the
United States. J Contin Educ Health Prof. 2005;25:132-133.
3. Kaufman DM. Applying education theory in practice. BMJ.
2003;326:213-216.
4. Grant J. Learning needs assessment: assessing the need. BMJ.
2002;324:156-159.
5. Kirkpatrick D. Revisiting Kirkpatrick’s four-level model.
Training and Development. 1996;50:54-59.
6. Miller GE. The assessment of clinical
skills/competence/performance. Acad Med. 1990;65(9
Suppl):S63-67.
7. Moore DE, Green JS, Gallis HA. Achieving desired results and
improved outcomes: integrating planning and assessment throughout
learning activities. J Contin Educ Health Prof.
2009;29(1):1–15.
8. Stahl SM, Grady M, Santiago G, et al. Optimizing outcomes in
psychopharmacology continuing medical education (CME): measuring
learning and attitudes that may predict knowledge translation into
clinical practice. Focus. 2006;4(4):487-495.
9. Wakefield J, Herbert CP, Maclure M, et al. Commitment to
change statements can predict actual change in practice. J Contin
Educ Health Prof. 2003;23:81–93.
10. Delcourt JL. Commitment to change: a strategy for promoting
educational effectiveness. J Contin Educ Health Prof.
2000;20:156–163.
11. PRIME® Education Inc. http://www.primeinc.org.
12. American Academy of Family Physicians. Performance
Improvement in Practice.
http://www.aafp.org/online/en/home/cme/cmea/cmeapplying/perfimprovement.html.
Accessed April 5, 2010.
13. Stowell SA, Karcher RB, Carter RD, et al. Outcomes
measurement design for a performance improvement initiative in
diabetes care. CE Measure. 2009;3:76-83.