Results of the 2020 NRMP Program Director Survey
www.nrmp.org
August 2020
Questions about the contents of this publication may be directed to National Resident Matching Program, (202) 400-2233 or [email protected].
Questions about the NRMP should be directed to Donna L. Lamb, D.HSc., M.B.A., B.S.N., President and CEO, National Resident Matching Program, (202) 400-2233 or [email protected].
Suggested CitationNational Resident Matching Program, Data Release and Research Committee: Results of the
2020 NRMP Program Director Survey. National Resident Matching Program, Washington, DC. 2020.
Copyright © 2020 National Resident Matching Program. All rights reserved. Permission to use, copy and/or distribute any documentation and/or related images from this publication shall be
expressly obtained from the NRMP.
Table of Contents
Introduction ................................................................................................................................................. 1 All Specialties Charts for Individual Specialties Anesthesiology .............................................................................................................................................. 9 Child Neurology .......................................................................................................................................... 16 Dermatology ............................................................................................................................................... 23 Emergency Medicine .................................................................................................................................. 30 Family Medicine ......................................................................................................................................... 37 Internal Medicine ........................................................................................................................................ 44 Internal Medicine/Pediatrics ....................................................................................................................... 51 Interventional Radiology ............................................................................................................................. 58 Neurological Surgery .................................................................................................................................. 65 Neurology ................................................................................................................................................... 72 Obstetrics and Gynecology ......................................................................................................................... 79 Orthopaedic Surgery ................................................................................................................................... 86 Otolaryngology ........................................................................................................................................... 93 Pathology .................................................................................................................................................. 100 Pediatrics ................................................................................................................................................... 107 Physical Medicine and Rehabilitation ...................................................................................................... 114 Plastic Surgery .......................................................................................................................................... 121 Psychiatry .................................................................................................................................................. 128 Radiation Oncology .................................................................................................................................. 135 Radiology-Diagnostic ............................................................................................................................... 142 Surgery ..................................................................................................................................................... 149 Transitional Year ...................................................................................................................................... 156
Introduction
1NRMP Program Director Survey Results, 2020
In March 2020, the National Resident Matching Program (NRMP) conducted its biennial survey of the directors of all programs participating in the Main Residency Match®. The primary purpose of the survey was to shed light on the factors that program directors use to (1) select applicants to interview and (2) rank applicants for the Match.
The survey was fielded during the 18 days between the Rank Order List Deadline and Match Week so that program match outcomes would not influence respondents' answers.
The survey solicited information on: the number of applications received, screened, and
reviewed, as well as the number of applicants interviewed and ranked
the frequency with which programs interview and rank specific applicant groups
use of test scores in considering which applicants to interview, and
the factors used for both interview selection and for ranking applicants.
The overall response rate for all specialties in this report was 18.0 percent. Transitional Year programs and programs in 22 specialties with 10 or more responses are included in this report. Response rates are listed in the table below.
In the 2020 Program Director Survey, several new questions were added to learn more about the recruitment process at the program level including whether programs would offer more invitations than interview slots as well as reasons for that practice; time allowed for applicants to respond to interview invitations; and how programs scheduled interviews.
Numbers of responses are presented in most of the graphs, and some graphs use data from multiple survey questions. In those cases, different N's are listed. Numbers of applicants ranked and positions in the Match are extracted from the NRMP database. Graphs are suppressed for questions with fewer than five responses.
This report presents results by specialty on selected items from the survey. The NRMP hopes that program directors, medical school officials, and applicants find these data useful as they prepare for and participate in the Main Residency Match._______________
The NRMP's data reporting and research activities are guided by its Data Release and Research Committee. NRMP data and reports can be found at: www.nrmp.org/main-residency-match-data/.
Specialty Surveys Sent Number Responding Response Rate
Anesthesiology 191 33 17.3%
Child Neurology 77 20 26.0%
Dermatology 142 16 11.3%
Emergency Medicine 248 59 23.8%
Family Medicine 635 142 22.4%
Internal Medicine 854 69 8.1%
Internal Medicine/Pediatrics 77 10 13.0%
Interventional Radiology (Integrated) 84 19 22.6%
Neurological Surgery 112 20 17.9%
Neurology 164 30 18.3%
Obstetrics and Gynecology 288 73 25.3%
Orthopaedic Surgery 189 49 25.9%
Otolaryngology 118 30 25.4%
Pathology‐Anatomic and Clinical 136 29 21.3%
Pediatrics 228 52 22.8%
Physical Medicine and Rehabilitation 105 21 20.0%
Plastic Surgery (Integrated) 82 16 19.5%
Psychiatry 250 57 22.8%
Radiation Oncology 85 17 20.0%
Radiology‐Diagnostic 195 36 18.5%
Surgery‐General 540 70 13.0%
Transitional Year 144 22 15.3%
Vascular Surgery 59 10 16.9%
All Others 140 24 17.1%
Total 5,143 924 18.0%
All Specialties Combined(N=924)
100% 50% 0%
90%84%
78%78%76%
70%69%66%66%65%65%63%62%62%61%60%57%56%52%51%51%50%48%47%46%45%44%41%40%36%
31%28%25%
21%20%20%15%10%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
1 2 3 4 5
4.04.3
4.03.94.0
4.44.0
4.34.5
4.04.3
4.24.2
4.54.2
4.03.93.9
3.74.4
3.84.04.0
3.53.6
4.03.73.84.0
3.74.7
4.13.9
3.63.8
3.23.33.5
Figure 1All SpecialtiesPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=650)
Percent Citing Factor Average Rating
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
3NRMP Program Director Survey Results, 2020
100% 50% 0%
95%95%
89%88%
70%67%64%64%63%61%60%58%57%56%54%50%49%47%46%43%42%42%37%36%36%35%34%34%31%30%29%28%23%23%21%20%
18%18%17%16%11%11%10%8%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
All SpecialtiesPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=641)
Figure 2
1 2 3 4 5
4.84.84.84.7
4.13.94.0
4.34.7
4.44.3
4.04.44.4
3.84.04.14.0
4.44.04.0
3.94.2
3.84.4
3.94.04.04.1
3.83.83.83.9
3.74.7
3.94.0
3.83.94.03.9
3.43.4
3.7
Average RatingPercent Citing Factor
4NRMP Program Director Survey Results, 2020
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
26%
36% 38%
Test Required?
All SpecialtiesProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure 3
USMLE Step 1 Scores
USMLE Step 2 (CK) Scores
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
43%
55%
3%
Test Required?
0%
20%
40%
60%
80%
100%
Never Seldom Often
28%
57%
15%
Does your program consider applicants who fail theirLicensure test on the first attempt?
0%
20%
40%
60%
80%
100%
Never Seldom often
33%
57%
10%
Does your program consider applicants who fail theirLicensure test on the first attempt?
N=655N=651
N=651 N=629
N=171N=281N=337
USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
N=190
170
180
190
200
210
220
230
240
250
260
Scores Below Which Programs Generally Do Not Grant Interviews
170
180
190
200
210
220
230
240
250
260
Scores Above Which Programs Almost Always Grant Interviews
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol in the box represents the mean.
USMLE Step 2 CKUSMLE Step 1USMLE Step 2 CKUSMLE Step 1
5NRMP Program Director Survey Results, 2020
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6% 4%
90%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
41%45%
14%
COMLEX-USA Level 1*Test Required?
N=240%
20%
40%
60%
80%
100%
Yes No
51% 49%
USMLE Step 2 Clinical Skills (CS)Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
25%
35%39%
COMLEX-USA Level 2-CE*Test Required?
N=190%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5% 2%
93%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
39%
12%
49%
COMLEX-USA Level 2-PE*Test Required?
N=542 N=540
N=531 N=446
* DO seniors/graduates only
Figure 4 All SpecialtiesPrograms Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA
Scores* When Considering Which Applicants to Interview
N=644 N=542
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
6NRMP Program Director Survey Results, 2020
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to Oct. 12019
October 2019 November2019
December2019
January 2020or later
18%
47%
22%
10%4%
Percentage of Program's Interviews Extended During EachTime Period
All SpecialtiesProgram's Interview Activities Figure 5
0
100
200
300
400
500
600
700
800
900
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants ranked
6
789
107 86 74
Average Number of Positions, Applications Received, InterviewInvitations Sent, and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applications rejectedbased on a standardized screening
process
Percentage of applicationsreceiving an in-depth review
45%49%
Average Percentage of Applications Rejected and Reviewed
N=626 N=647N=663 N=668 N=669 N=668*N=924*
7NRMP Program Director Survey Results, 2020
* Data from the NRMP database. All responding programs included
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to Oct. 12019
October 2019 November2019
December2019
January 2020 orlater
1%
15%
32% 33%
19%
Percentage of Program's Interviews Conducted During EachTime Period
N=649 N=632
Yes, 36%
No, 64%
Program offered more interview invitations toapplicants than interview slots to fill
N=685
0% 20% 40% 60% 80% 100
I wanted to ensure we had anadequate number of alternatecandidates in the event other
applicants cancelled at the last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
68%
39%
31%
24%
Reason for offering more interview invitations toapplicants than interview slots to fill
N=245
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
1% 0% 1% 4%
15%
80%
Time given applicants to respond to interview invitationsbefore extending invitations to other applicants
0% 20% 40% 60% 80% 100
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
66%
19%
21%
How interviews were scheduled
N=676N=662
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
35% 35%
12%
23%
No communication otherthan receipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has
initiated contact withthe applicant
Other
Preferred Type of Communication About Applicants During SOAP
All SpecialtiesOther Topics Figure 6
N=638
8
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.9
4.9
4.8
4.5
4.3
4.2
3.8
3.7
3.5
3.1
Importance of Factors in Assessing Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
NRMP Program Director Survey Results, 2020
n=621
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian1% 4% 10%
26%17%
32%
67%
30%
5%
58%
26%
51%54%
51%
30%
56%94%
37%
64%
23% 29%17%
3%14%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian1% 7% 11%
26% 20%34%
67%
32%
5%
56%
25%
51%52%
48%
29%
54%94%
37%
65%
23% 28%17%
3%14%
Often Seldom Never
RankingFrequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
n=661 n=660
Anesthesiology(N=33)
NRMP Program Director Survey Results, 2020 9
Figure AN-1AnesthesiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=25)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.34.4
4.23.5
4.04.2
3.94.5
4.84.2
4.54.4
4.24.8
4.54.5
3.94.3
3.94.9
3.64.24.1
3.73.7
4.04.2
3.84.3
3.65.0
4.34.3
4.04.0
3.83.03.0
100% 50% 0%
100%68%
88%60%
92%68%
76%72%
76%80%
64%80%
52%68%
60%48%
52%56%52%
56%56%
48%56%52%
44%40%40%36%
28%36%
12%
24%12%16%
8%20%8%4%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 10
AnesthesiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=25)
Figure AN-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.94.94.8
4.14.24.34.4
4.84.44.4
4.04.2
4.63.84.0
4.64.4
4.75
3.94.2
4.43.7
4.93.9
4.44.24.4
4.14.14.3
4.03.8
5.03.0
4.73.83.7
4.25.05.0
3.53.0
100% 50% 0%
100%100%
88%88%
56%88%
80%68%
72%48%
68%92%
76%56%
36%76%
36%36%
56%52%56%52%
40%44%40%
28%20%
40%28%
36%28%
12%16%20%8%4%
12%12%12%
24%16%8%8%4%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 11
170
180
190
200
210
220
230
240
250
260
AnesthesiologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure AN-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
36%
60%
4%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
40%
56%
4%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
21%
79%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
12%
48%40%
Test Required?
N=24N=25
N=25N=25
N=11N=18 N=15 N=10
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 12
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13%
70%
17%
COMLEX-USA Level 1*Test Required?
N=24
0%
20%
40%
60%
80%
100%
Yes No
46%54%
USMLE Step 2 CSTest Required?
AnesthesiologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure AN-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5%
50%45%
COMLEX-USA Level 2-CE*Test Required?
N=190%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%11%
89%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%6%
94%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
24%
5%
71%
COMLEX-USA Level 2-PE*Test Required?
N=24N=19
N=23 N=22
N=21 N=18
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 13
Figure AN-5AnesthesiologyProgram's Interview Activities
0
200
400
600
800
1,000
1,200
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
10
1,033
168 143 125
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
48%43%
Average Percentage of Applications Rejected and Reviewed
N=26N=25N=33* N=26*N=26N=27
* Data from the NRMP database. All responding programs included
N=26
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
23%
47%
17%10%
4%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%
19%
35% 33%
12%
Percentage of Program's Interviews Conducted During EachTime Period
N=25N=25
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
76%
20%
16%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 4% 4%
23%
69%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
70%
20%
20%
20%
Reason for offering more interview invitations to applicants thaninterview slots to fill
38%Yes
62%No
Program offered more interview invitations toapplicants than interview slots to fill
N=26
N=10
N=26N=25
NRMP Program Director Survey Results, 2020 14
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
5.0
4.9
5.0
5.0
4.8
4.6
4.4
4.5
3.9
4.4
3.6
3.4
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure AN-6AnesthesiologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=24
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
50%
33%
17%13%
No communication other thanreceipt of the ERAS
application
No communication andinstead will initiate contactwith references and/or themedical school if needed
To be contacted by the applicant’s
medical school after program has initiated
contact with the applicant
Other
Preferred Type of Communication About Applicants During SOAP
N=24
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
4% 8% 8%24%
71%
16%
36% 24%
60% 60%
72%
25%
76%100%
64%72%
32% 32%
4% 4% 8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian4% 8% 8%
24%
72%
16%
40%21%
58% 60%
72%
24%
76%100%
60%75%
33% 32%
4% 4% 8%
Often Seldom Never
Rankingn=25 n=25
NRMP Program Director Survey Results, 2020 15
Child Neurology(N=20)
NRMP Program Director Survey Results, 2020 16
Figure CN-1Child NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=17)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.84.2
3.63.9
3.64.2
3.63.9
4.43.3
3.63.8
3.54.14.0
3.63.4
4.03.2
4.53.5
3.83.8
3.43.3
3.63.33.3
3.53.5
5.05.0
3.63.0
3.32.3
3.03.0
100% 50% 0%
88%82%
65%82%
65%76%76%
53%82%
71%59%
53%76%
53%59%
47%47%
59%59%
76%47%
53%24%
53%65%
41%53%53%
35%35%
24%12%
29%12%
35%24%24%24%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 17
Child NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=17)
Figure CN-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.74.64.64.5
4.03.8
3.64.1
4.44.3
3.73.8
4.44.2
3.63.73.93.8
4.52
3.44.0
3.83.0
4.53.3
4.33.7
4.03.0
3.33.8
3.63.5
4.73.7
4.52.72.8
5.03.2
3.03.0
3.5100% 50% 0%
100%100%100%
88%76%
71%53%53%
29%76%
41%59%
41%53%53%
41%41%
24%35%29%
47%35%
29%41%
24%24%18%18%
35%29%
35%24%
29%24%18%18%12%
29%24%
6%18%
6%6%
12%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 18
170
180
190
200
210
220
230
240
250
260
Child NeurologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure CN-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
24%
71%
6%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
35%
59%
6%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
41%
53%
6%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
12% 12%
76%
Test Required?
N=17N=17
N=17N=17
N=9 N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 19
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
36%
50%
14%
COMLEX-USA Level 1*Test Required?
N=17
0%
20%
40%
60%
80%
100%
Yes No
18%
82%
USMLE Step 2 CSTest Required?
Child NeurologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure CN-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
14%
29%
57%
COMLEX-USA Level 2-CE*Test Required?
N=160%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%8%
92%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
14%7%
79%
COMLEX-USA Level 2-PE*Test Required?
N=17N=16
N=14 N=14
N=14 N=12
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 20
Figure CN-5Child NeurologyProgram's Interview Activities
0
20
40
60
80
100
120
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview
invitations sent
Number ofapplicantsinterviewed
Number ofapplicants ranked
2
117
36
24 22
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
36%
67%
Average Percentage of Applications Rejected and Reviewed
N=17N=16N=20* N=18*N=18N=17
* Data from the NRMP database. All responding programs included
N=18
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to Oct. 12017
October 2017 November2017
December2017
January 2017or later
43% 40%
9% 8%0%
Percentage of Program's Interviews Extended DuringEach Time Period
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to Oct. 12017
October 2017 November2017
December2017
January 2017or later
0%
16%
38%30%
15%
Percentage of Program's Interviews Conducted DuringEach Time Period
N=16N=16
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
82%
0%
18%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0%
11%
89%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
86%
71%
86%
14%
Reason for offering more interview invitations to applicants thaninterview slots to fill
39%Yes
61%No
Program offered more interview invitations toapplicants than interview slots to fill
N=18
N=7
N=18N=17
NRMP Program Director Survey Results, 2020 21
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.7
4.8
4.8
4.5
4.6
4.3
3.4
3.7
3.6
3.1
2.8
2.9
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure CN-6Child NeurologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=17
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
53%
29%
18%12%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=17
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
6%19%
12%25%
87%
24%
53%
29%
50%76% 44%
13%
59%100%
41%
71%
31%
12%
31%18%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
6%21%
12%24%
80%
29%
50%
29%
43%76% 47%
20%
53%100%
44%
71%
36%
12%
29%18%
Often Seldom Never
Rankingn=17 n=17
NRMP Program Director Survey Results, 2020 22
Dermatology(N=16)
NRMP Program Director Survey Results, 2020 23
Figure D-1DermatologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=12)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.34.5
4.04.0
4.54.34.54.5
4.94.4
4.04.24.44.5
4.03.8
4.03.7
4.04.5
4.13.5
4.53.8
3.64.5
3.93.9
3.64.5
4.34.3
3.73.5
3.33.5
100% 50% 0%
100%83%
67%83%
92%92%92%92%
67%83%
75%75%75%75%
83%33%
75%25%
83%33%
83%17%
67%83%
58%17%
58%67%
0%58%
33%25%
33%
25%50%50%
17%0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 24
DermatologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=12)
Figure D-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.84.74.74.6
4.34.04.0
4.64.44.4
4.34.34.4
4.04.0
4.3
4.14.3
4.44.0
4.34.2
4.04.1
4.54.7
3.33.5
4.04.3
4.04.0
3.84.3
3.74.0
5.0
3.8
100% 50% 0%
92%100%100%100%
67%67%
50%67%
75%58%
67%50%
58%58%
50%50%
0%67%
25%0%
67%8%
50%42%
8%67%
0%50%
25%25%
33%50%
25%33%
17%42%
25%8%
25%8%0%0%
33%0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 25
DermatologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure D-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
50% 50%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
67%
33%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
33%
67%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
0%
27%
73%
Test Required?
N=12N=12
N=11N=12
N=7
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 26
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
43%
57%
COMLEX-USA Level 1*Test Required?
N=11
0%
20%
40%
60%
80%
100%
Yes No
36%
64%
USMLE Step 2 CSTest Required?
DermatologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure D-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
29%
71%
COMLEX-USA Level 2-CE*Test Required?
N=100%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
29%
71%
COMLEX-USA Level 2-PE*Test Required?
N=11N=10
N=7 N=7
N=7 N=6
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 27
Figure D-5DermatologyProgram's Interview Activities
0
100
200
300
400
500
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
4
488
44 41 38
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
41%52%
Average Percentage of Applications Rejected and Reviewed
N=11N=11N=16* N=12*N=12N=12
* Data from the NRMP database. All responding programs included
N=12
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
3%
20%
62%
12%3%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
3% 2%10%
21%
64%
Percentage of Program's Interviews Conducted During EachTime Period
N=11N=11
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
67%
17%
17%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0%10% 10%
80%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
25%Yes
75%No
Program offered more interview invitations toapplicants than interview slots to fill
N=12
N=10N=12
Reasons for "offering more interview invitations to applicants
than interview slots to fill" not reported because of low response
rate.
NRMP Program Director Survey Results, 2020 28
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.9
4.9
4.7
4.9
4.3
4.4
4.0
3.8
4.1
3.3
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure D-6DermatologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=12
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
8%
33%25%
50%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=12
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
8%
33%
58%42%
58%
91%
58%58%
58%
42%58%
42%
9%
42%
100%
33%
8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
8%
27%
55%42%
58%
92%
50%
58%
55%
45%58%
42%
8%
50%
100%
33%18%
Often Seldom Never
Rankingn=12 n=12
NRMP Program Director Survey Results, 2020 29
Emergency Medicine(N=59)
NRMP Program Director Survey Results, 2020 30
Figure EM-1Emergency MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=38)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.64.9
3.93.53.7
4.34.2
4.74.6
3.94.4
3.94.7
4.44.0
3.73.6
3.93.4
4.73.4
4.04.1
3.43.43.53.4
4.23.7
3.23.94.1
4.73.33.3
2.22.83.0
100% 50% 0%
95%97%
84%55%
63%76%
53%71%
76%55%
66%71%
63%45%
55%55%
66%68%
58%66%
47%66%
45%39%
50%39%
50%29%
39%26%18%
32%61%
11%11%
13%13%11%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 31
Emergency MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=38)
Figure EM-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.84.84.84.8
3.74.04.2
4.64.14.2
3.64.8
4.53.5
3.93.9
3.74.5
64.5
3.94.7
3.34.8
3.54.2
4.03.6
3.13.5
3.23.8
4.24.7
4.24.3
4.03.3
3.73.8
2.33.3
2.0100% 50% 0%
95%86%
68%81%
100%73%
81%46%
51%59%
51%43%
65%59%
38%41%
49%51%
54%62%
35%54%
49%43%
46%22%
35%24%14%22%
30%16%14%16%8%
16%54%
22%8%8%11%8%8%5%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 32
170
180
190
200
210
220
230
240
250
260
Emergency MedicineProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure EM-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
23%
72%
5%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
30%
68%
3%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
56%
44%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
28%36% 36%
Test Required?
N=39N=39
N=39N=37
N=12N=17 N=14 N=11
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 33
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
43% 40%
17%
COMLEX-USA Level 1*Test Required?
N=39
0%
20%
40%
60%
80%
100%
Yes No
33%
67%
USMLE Step 2 CSTest Required?
Emergency MedicineProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure EM-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
17%
37%46%
COMLEX-USA Level 2-CE*Test Required?
N=320%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%3%
88%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
7% 3%
90%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
26%
11%
63%
COMLEX-USA Level 2-PE*Test Required?
N=39N=32
N=35 N=35
N=35 N=30
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 34
Figure EM-5Emergency MedicineProgram's Interview Activities
0
200
400
600
800
1,000
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
11
953
182139 126
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
29%
62%
Average Percentage of Applications Rejected and Reviewed
N=39N=38N=59* N=39*N=39N=39
* Data from the NRMP database. All responding programs included
N=39
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
11%
54%
19%11%
5%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
3%12%
33% 32%
20%
Percentage of Program's Interviews Conducted During EachTime Period
N=39N=39
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
76%
7%
27%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
2% 0% 0%5%
15%
80%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
80%
35%
30%
30%
Reason for offering more interview invitations to applicants thaninterview slots to fill
49%Yes
51%No
Program offered more interview invitations toapplicants than interview slots to fill
N=41
N=20
N=41N=41
NRMP Program Director Survey Results, 2020 35
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
5.0
4.8
4.8
4.4
4.4
4.1
3.6
3.8
3.1
2.7
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure EM-6Emergency MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=38
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
22%
46%
5%
30%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=37
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
3% 3%
21%13%
41%56%
44%
74%
23%
67%
64%
59%41%
49%
100%
23%
74%
13%23%
3% 8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian5% 3%
19% 18%
46%58%
45%
64%
19%
65%59%
51%39%
45%
100%
31%
78%
16%23%
3% 3%11%
Often Seldom Never
Rankingn=39 n=39
NRMP Program Director Survey Results, 2020 36
Family Medicine(N=142)
NRMP Program Director Survey Results, 2020 37
Figure FM-1Family MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=97)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.64.1
3.94.03.9
4.53.7
4.24.1
3.74.3
4.13.9
4.54.2
4.04.0
3.63.5
4.14.1
3.93.9
3.23.7
3.93.7
3.64.0
3.34.7
4.13.8
3.63.2
2.93.43.6
100% 50% 0%
80%81%80%
84%73%
93%61%
64%76%
49%67%65%
51%62%61%
79%64%
80%32%
76%67%
78%40%38%36%
57%37%38%
74%8%
33%44%
11%
39%11%9%
25%20%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 38
Family MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=97)
Figure FM-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.84.84.7
3.93.6
3.94.3
4.64.6
4.33.8
4.34.4
4.03.7
4.14.04.0
74.0
3.63.9
3.64.04.14.0
3.74.03.93.8
3.03.8
3.44.6
3.54.04.24.2
3.94.0
3.72.3
3.8100% 50% 0%
98%95%90%91%
65%62%
76%66%
60%83%
67%63%57%
65%71%
41%72%
56%63%
71%33%
60%27%
20%61%
44%65%
31%44%
26%24%
6%15%17%21%11%4%
26%33%
23%16%14%3%
15%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 39
170
180
190
200
210
220
230
240
250
260
Family MedicineProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure FM-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
0%
55%
45%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
5%
66%
29%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
70%
29%
1%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
46%
32%
22%
Test Required?
N=97N=97
N=97N=94
N=21N=25 N=24 N=21
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 40
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
68%
31%
1%
COMLEX-USA Level 1*Test Required?
N=97
0%
20%
40%
60%
80%
100%
Yes No
69%
31%
USMLE Step 2 CSTest Required?
Family MedicineProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure FM-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
45%
33%22%
COMLEX-USA Level 2-CE*Test Required?
N=780%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
4% 0%
96%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 1%
96%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
62%
10%
29%
COMLEX-USA Level 2-PE*Test Required?
N=97N=78
N=97 N=94
N=94 N=76
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 41
Figure FM-5Family MedicineProgram's Interview Activities
0
200
400
600
800
1,000
1,200
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
8
1,105
115 86 73
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
56%
34%
Average Percentage of Applications Rejected and Reviewed
N=89N=88N=142* N=97*N=98N=94
* Data from the NRMP database. All responding programs included
N=97
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
30%40%
17%9%
3%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%
23%
38%28%
10%
Percentage of Program's Interviews Conducted During EachTime Period
N=93N=93
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
74%
12%
18%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
1% 0% 0% 1%
18%
82%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
65%
41%
27%
22%
Reason for offering more interview invitations to applicants thaninterview slots to fill
52%
Yes
48%No
Program offered more interview invitations toapplicants than interview slots to fill
N=98
N=51
N=99N=97
NRMP Program Director Survey Results, 2020 42
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.8
4.8
4.8
4.4
4.2
4.1
3.8
3.6
3.6
2.5
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure FM-6Family MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=92
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
44%35%
4%
23%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=93
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
1% 3%14%
34%
73%
36%
8%
65%
3%
69% 44%
52%
26%
52%92%
34%
97%
28%42%
15%1%
11%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian4% 3%
18%35%
73%
40%
9%
61%
2%
68% 39%
50%
26%
50%
91%
34%
98%
29%43%
15%1%
10%
Often Seldom Never
Rankingn=98 n=98
NRMP Program Director Survey Results, 2020 43
Internal Medicine(N=69)
NRMP Program Director Survey Results, 2020 44
Figure IM-1Internal MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=44)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.14.0
4.43.9
4.24.3
3.94.24.4
4.14.2
4.03.9
4.74.24.1
3.94.2
4.04.3
3.74.3
3.93.4
3.74.1
4.03.7
4.23.7
4.84.1
3.73.6
3.33.3
3.23.4
100% 50% 0%
91%75%
93%70%
89%55%
77%55%
73%80%
48%57%
66%52%
57%84%
55%80%
48%59%
39%80%
57%50%52%
61%48%43%
61%27%
48%
34%16%
32%20%23%
27%16%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 45
Internal MedicinePercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=44)
Figure IM-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.8
4.74.5
4.04.14.3
4.54.8
4.34.24.34.4
4.24.04.3
4.13.9
4.67
4.44.24.3
4.14.6
3.74.24.14.34.2
3.93.5
3.93.6
4.73.9
3.74.04.04.1
4.03.03.2
3.7100% 50% 0%
95%95%
86%82%
73%82%84%
68%77%
55%59%
80%36%
41%48%
64%73%
50%57%
70%43%
70%48%
36%48%
34%57%
41%55%
41%39%
23%39%
27%27%
16%7%
11%25%
34%2%
25%14%16%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 46
170
180
190
200
210
220
230
240
250
260
Internal MedicineProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure IM-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
18%
71%
11%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
20%
68%
11%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
32%
66%
2%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
25%
50%
25%
Test Required?
N=44N=45
N=44N=44
N=18N=27 N=25 N=17
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 47
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
28%
60%
13%
COMLEX-USA Level 1*Test Required?
N=44
0%
20%
40%
60%
80%
100%
Yes No
68%
32%
USMLE Step 2 CSTest Required?
Internal MedicineProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure IM-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
22%
54%
24%
COMLEX-USA Level 2-CE*Test Required?
N=370%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 5%
92%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6% 6%
89%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
41%
26%33%
COMLEX-USA Level 2-PE*Test Required?
N=44N=37
N=40 N=41
N=39 N=35
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 48
Figure IM-5Internal MedicineProgram's Interview Activities
0
400
800
1,200
1,600
2,000
2,400
Number ofpositions in the
Match
Number ofapplications
received
Number ofinterview invitations
sent
Number ofapplicantsinterviewed
Number ofapplicants ranked
8
2,118
199 170 141
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
59%
31%
Average Percentage of Applications Rejected and Reviewed
N=46N=44N=69* N=49*N=49N=47
* Data from the NRMP database. All responding programs included
N=49
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
18%
45%
22%
11%5%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
2%
20%
35%28%
15%
Percentage of Program's Interviews Conducted During EachTime Period
N=47N=47
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
72%
15%
20%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 2%
15%
83%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
65%
53%
59%
24%
Reason for offering more interview invitations to applicants thaninterview slots to fill
35%Yes
65%No
Program offered more interview invitations toapplicants than interview slots to fill
N=49
N=17
N=48N=46
NRMP Program Director Survey Results, 2020 49
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.8
4.9
4.8
4.7
4.4
4.4
4.2
3.9
3.7
3.1
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure IM-6Internal MedicineProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=42
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
50%
25%
11% 14%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=44
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
4% 11%2%
18% 11% 18%
60%
29%18%
58%
23%
49%
40%36%
40%
53%78%
31%
75%
33%49% 47%
18%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
7%14%
5%16% 11%
21%
60%
32%11%
55%
20%
51%
43%37%
40%
52%82%
32%
75%
33%45% 42%
16%
Often Seldom Never
Rankingn=45 n=45
NRMP Program Director Survey Results, 2020 50
Internal Medicine/Pediatrics(N=10)
NRMP Program Director Survey Results, 2020 51
Figure IMP-1Internal Medicine/PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=7)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.03.4
4.03.9
4.13.6
4.34.3
4.84.24.24.2
4.05.0
4.24.2
4.04.0
4.44.8
3.34.0
4.33.8
3.53.5
4.44.0
4.33.0
4.05.0
3.03.0
3.54.04.0
100% 50% 0%
100%71%71%
100%100%
71%86%
43%71%71%
86%86%
71%43%
71%71%
57%86%
71%57%
43%71%
57%57%
29%29%
71%57%57%
14%43%
29%
0%14%14%
29%14%14%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 52
Internal Medicine/PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=7)
Figure IMP-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.64.4
4.84.7
3.74.2
4.04.0
5.03.7
4.33.7
5.04.4
3.83.84.0
4.54.5
54.3
4.04.04.0
4.54.3
4.03.8
4.04.0
4.8
4.54.3
4.04.0
3.0
4.03.5
4.0100% 50% 0%
100%100%
86%86%
43%71%
86%57%
43%43%
86%86%
29%71%71%71%
43%57%57%
71%43%
71%71%71%
57%43%
57%57%
14%43%
57%0%
29%43%
29%14%0%0%14%0%0%
14%29%
14%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 53
Internal Medicine/PediatricsProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure IMP-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
14%
43% 43%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
43%
29% 29%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
57%
43%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14% 14%
71%
Test Required?
N=7N=7
N=7N=7
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
NRMP Program Director Survey Results, 2020 54
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
67%
33%
0%
COMLEX-USA Level 1*Test Required?
N=7
0%
20%
40%
60%
80%
100%
Yes No
43%
57%
USMLE Step 2 CSTest Required?
Internal Medicine/PediatricsProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure IMP-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
17% 17%
67%
COMLEX-USA Level 2-CE*Test Required?
N=70%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
33%
17%
50%
COMLEX-USA Level 2-PE*Test Required?
N=7N=7
N=6 N=6
N=6 N=6
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 55
Figure IMP-5Internal Medicine/PediatricsProgram's Interview Activities
0
40
80
120
160
200
240
280
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
6
242
9877 75
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
32%
74%
Average Percentage of Applications Rejected and Reviewed
N=8N=7N=10* N=8*N=8N=8
* Data from the NRMP database. All responding programs included
N=8
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
34%
53%
11%3% 0%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
25%
38%30%
7%
Percentage of Program's Interviews Conducted During EachTime Period
N=8N=8
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
71%
14%
14%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0%
13%
25%
63%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
38%Yes
63%No
Program offered more interview invitations toapplicants than interview slots to fill
N=8
N=8N=7
Reasons for "offering more interview invitations to applicants
than interview slots to fill" not reported because of low response
rate.
NRMP Program Director Survey Results, 2020 56
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.7
4.7
4.7
4.7
4.7
4.4
4.4
4.1
4.4
3.7
3.1
3.1
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure IMP-6Internal Medicine/PediatricsProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=7
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
14%
43%
0%
43%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=7
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
29%14%
43% 43%29%
86%
29%
43%
29%
43% 43% 71%
14%
71%
100%
29%
57%
14% 14%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
33%
14%
43% 43%29%
86%
29%
50%
29%
43% 43% 71%
14%
71%
100%
17%
57%
14% 14%
Often Seldom Never
Rankingn=7 n=7
NRMP Program Director Survey Results, 2020 57
Interventional Radiology(N=19)
NRMP Program Director Survey Results, 2020 58
Figure IR-1Interventional RadiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=11)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.14.3
3.73.4
4.14.4
3.94.6
4.14.0
4.34.4
3.64.3
4.03.83.8
3.03.83.8
3.13.5
3.03.2
4.03.0
4.03.63.53.43.53.4
4.04.0
3.82.8
4.04.0
100% 50% 0%
91%100%
64%73%
91%73%
82%64%
73%82%
100%64%
73%45%
64%45%45%
18%
73%45%
73%18%
27%82%
55%27%27%
64%
18%82%
36%55%
18%
9%36%36%
18%9%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 59
Interventional RadiologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=11)
Figure IR-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.84.94.9
4.84.44.5
3.84.4
4.24.44.44.4
4.84.8
3.54.7
3.64.0
4.55
4.23.5
4.34.64.5
3.53.0
4.03.5
4.45.0
3.93.6
4.04.7
4.04.3
4.04.5
4.33.5
3.03.0
100% 50% 0%
91%100%
82%73%
91%73%
55%45%
55%45%
55%64%
73%82%
55%55%
45%45%
36%18%
45%18%
27%45%
18%36%
18%18%18%
45%9%
64%45%
18%27%
9%36%
27%9%
36%0%
18%9%9%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 60
170
180
190
200
210
220
230
240
250
260
Interventional RadiologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure IR-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
20%
80%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
9%
82%
9%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
27%
73%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
18%27%
55%
Test Required?
N=11N=10
N=11N=11
N=8 N=7
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 61
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
38%
13%
50%
COMLEX-USA Level 1*Test Required?
N=11
0%
20%
40%
60%
80%
100%
Yes No
36%
64%
USMLE Step 2 CSTest Required?
Interventional RadiologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure IR-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
13%
88%
COMLEX-USA Level 2-CE*Test Required?
N=110%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%0%
91%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
13%
88%
COMLEX-USA Level 2-PE*Test Required?
N=11N=11
N=8 N=8
N=8 N=8
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 62
Figure IR-5Interventional RadiologyProgram's Interview Activities
0
20
40
60
80
100
120
140
160
180
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
2
168
36 30 28
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
34%
72%
Average Percentage of Applications Rejected and Reviewed
N=11N=11N=19* N=11*N=11N=11
* Data from the NRMP database. All responding programs included
N=11
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
14%
70%
5%11%
0%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%10%
35% 32%
20%
Percentage of Program's Interviews Conducted During EachTime Period
N=11N=11
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
45%
45%
9%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0%
18%27%
55%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
27%Yes
73%No
Program offered more interview invitations toapplicants than interview slots to fill
N=11
N=11N=11
Reasons for "offering more interview invitations to applicants
than interview slots to fill" not reported because of low response
rate.
NRMP Program Director Survey Results, 2020 63
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.7
4.8
4.8
4.4
4.8
4.6
4.3
3.7
4.0
3.7
3.3
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure IR-6Interventional RadiologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=11
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
10%
60%
20%
10%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=10
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
9%18% 18%
45%
27%18%
64%55%
82%73%
55%
73%
73%100%
36% 36%
9% 9%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
9%18% 18%
45%
27%18%
64% 45%
82%73%
55%
73%
73%100%
36%45%
9% 9%
Often Seldom Never
Rankingn=11 n=11
NRMP Program Director Survey Results, 2020 64
Neurological Surgery(N=20)
NRMP Program Director Survey Results, 2020 65
Figure NS-1Neurological SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=13)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.24.8
4.04.4
3.84.9
4.34.0
5.04.04.2
4.64.64.64.7
3.84.7
4.03.3
4.04.0
3.83.73.7
4.44.0
3.74.0
4.35.0
3.83.7
5.04.7
3.53.0
100% 50% 0%
92%92%
54%62%62%
69%46%
69%23%
46%69%
38%38%
54%54%
31%54%
8%54%
0%38%
8%38%
23%54%
38%23%23%
8%69%
23%38%
46%8%
46%15%
8%0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 66
Neurological SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=13)
Figure NS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.94.94.8
4.34.04.24.3
4.94.9
4.53.9
4.24.8
4.54.2
3.84.44.3
04.4
4.04.5
3.4
4.04.04.2
4.54.04.0
4.44.04.0
5.04.3
3.63.5
4.03.5
4.03.0
3.5
100% 50% 0%
92%85%85%85%
92%69%
46%77%
62%69%
46%54%
69%62%
46%38%
31%62%
23%8%
38%8%
31%38%
0%38%
8%46%
31%23%15%
54%15%
23%15%
46%38%
46%8%
31%46%
8%15%
0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 67
170
180
190
200
210
220
230
240
250
260
Neurological SurgeryProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure NS-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
54%46%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
70%
30%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
8%
92%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
15%
31%
54%
Test Required?
N=13N=13
N=13N=10
N=12 N=7
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 68
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
33%
67%
0%
COMLEX-USA Level 1*Test Required?
N=13
0%
20%
40%
60%
80%
100%
Yes No
38%
62%
USMLE Step 2 CSTest Required?
Neurological SurgeryProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure NS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
67%
33%
COMLEX-USA Level 2-CE*Test Required?
N=110%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9% 9%
82%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
33%
67%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
67%
33%
COMLEX-USA Level 2-PE*Test Required?
N=13N=11
N=3 N=3
N=3 N=3
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 69
Figure NS-5Neurological SurgeryProgram's Interview Activities
0
40
80
120
160
200
240
280
320
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
2
283
46 34 28
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
55%45%
Average Percentage of Applications Rejected and Reviewed
N=13N=13N=20* N=13*N=13N=13
* Data from the NRMP database. All responding programs included
N=13
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
33%43%
11% 12%
0%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
6%11%
33% 33%
17%
Percentage of Program's Interviews Conducted During EachTime Period
N=13N=13
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
54%
31%
15%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
8%0% 0%
15% 15%
62%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
50%
33%
50%
17%
Reason for offering more interview invitations to applicants thaninterview slots to fill
43%Yes
57%No
Program offered more interview invitations toapplicants than interview slots to fill
N=14
N=6
N=13N=13
NRMP Program Director Survey Results, 2020 70
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
5.0
5.0
5.0
5.0
4.8
4.3
4.5
4.3
3.8
3.8
4.0
4.1
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure NS-6Neurological SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=12
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
33%
50%
17%8%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=12
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
62%69%
15%
69%
15%8%
54%
23%23%
77%92%
23%
54%
92%
46%
15%8% 8% 8% 8%
31%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
9%
62% 67%
15%
69%
17%8%
45%
31% 25%
77%92%
23%
50%
92%
45%
8% 8% 8% 8% 8%
33%
Often Seldom Never
Rankingn=13 n=13
NRMP Program Director Survey Results, 2020 71
Neurology(N=30)
NRMP Program Director Survey Results, 2020 72
Figure N-1NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=25)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.94.2
3.83.9
4.14.5
3.94.34.3
4.04.4
3.93.9
4.24.14.2
3.73.63.8
4.43.83.7
4.13.5
4.04.4
3.53.6
4.03.2
4.43.73.8
3.43.6
3.23.2
3.7100% 50% 0%
96%92%
84%84%
92%64%64%
76%68%
80%64%64%
68%76%
64%68%
36%44%
52%44%48%
44%44%
52%44%
48%52%
44%40%44%
32%24%
36%28%
20%20%
20%12%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 73
NeurologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=25)
Figure N-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
5.04.9
4.74.5
4.23.8
3.64.3
4.74.5
3.94.14.24.3
3.74.04.1
3.74.5
83.93.94.0
4.44.5
3.63.9
3.64.5
3.84.0
3.84.3
3.64.4
3.43.6
3.33.8
3.43.7
3.33.0
3.5100% 50% 0%
100%100%100%
88%68%
72%64%
68%64%
68%56%
64%52%
64%60%
48%60%
44%60%
36%40%
28%32%32%
40%32%
36%24%
32%36%
20%24%20%20%20%20%
24%24%20%20%16%16%8%8%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 74
170
180
190
200
210
220
230
240
250
260
NeurologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure N-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
16%
84%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
17%
83%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
52%44%
4%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
28%32%
40%
Test Required?
N=25N=25
N=25N=24
N=6N=11 N=10 N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 75
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
45%41%
14%
COMLEX-USA Level 1*Test Required?
N=24
0%
20%
40%
60%
80%
100%
Yes No
50% 50%
USMLE Step 2 CSTest Required?
NeurologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure N-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%36% 36%
COMLEX-USA Level 2-CE*Test Required?
N=230%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9%0%
91%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%5%
85%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
38%
19%
43%
COMLEX-USA Level 2-PE*Test Required?
N=24N=23
N=22 N=22
N=21 N=20
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 76
Figure N-5NeurologyProgram's Interview Activities
0
100
200
300
400
500
600
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
6
519
92 71 59
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
40%
58%
Average Percentage of Applications Rejected and Reviewed
N=22N=21N=30* N=24*N=24N=23
* Data from the NRMP database. All responding programs included
N=24
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
23%
44%
20%
9%3%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%
20%
34% 33%
12%
Percentage of Program's Interviews Conducted During EachTime Period
N=23N=23
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
67%
33%
8%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0%
12%
88%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
85%
31%
23%
8%
Reason for offering more interview invitations to applicants thaninterview slots to fill
52%Yes
48%No
Program offered more interview invitations toapplicants than interview slots to fill
N=25
N=13
N=25N=24
NRMP Program Director Survey Results, 2020 77
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
5.0
4.9
4.9
4.9
4.9
4.3
4.4
3.9
4.2
3.2
3.4
3.2
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure N-6NeurologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=24
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
25%
42%
21%17%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=24
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
4%16%
4% 8%
54%
12%4%
36%
24%
52%60%
64%
42%
64%96%
60%76%
32% 36%28%
4%
24%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian4%
16%4% 8%
54%
12%4%
56%
24%
56%
60%64%
42%
68%96%
40%
76%
28%36%
28%
4%20%
Often Seldom Never
Rankingn=25 n=25
NRMP Program Director Survey Results, 2020 78
Obstetrics and Gynecology(N=73)
NRMP Program Director Survey Results, 2020 79
Figure OG-1Obstetrics and GynecologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=56)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.14.0
4.33.94.1
4.34.14.3
4.74.14.24.34.3
4.74.3
4.14.14.1
3.84.6
3.94.24.1
3.63.7
4.03.93.94.1
3.54.8
4.23.43.53.5
3.33.53.7
100% 50% 0%
95%88%82%77%71%
55%70%
61%75%70%
63%64%
68%64%64%64%
57%52%52%52%
41%41%39%
43%50%
41%63%
39%
36%36%32%
21%13%
27%23%
14%7%5%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 80
Obstetrics and GynecologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=56)
Figure OG-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.84.94.8
3.94.24.24.3
4.74.34.3
3.94.34.3
3.64.0
4.34.14.2
74.0
4.34.1
3.94.4
3.94.4
4.14.0
3.84.0
3.43.7
3.94.7
3.73.3
4.34.0
4.33.8
4.72.8
4.7100% 50% 0%
93%93%93%93%
55%59%
63%66%66%
52%55%
46%55%50%
41%48%43%43%38%36%32%29%
43%29%29%25%25%
32%18%
29%36%
25%21%21%18%21%11%11%14%5%
14%5%
11%5%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 81
170
180
190
200
210
220
230
240
250
260
Obstetrics and GynecologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure OG-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
33%
61%
5%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
50% 50%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
33%
63%
4%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
28%
53%
19%
Test Required?
N=57N=57
N=57N=56
N=27N=34 N=25 N=21
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 82
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%
54%
19%
COMLEX-USA Level 1*Test Required?
N=55
0%
20%
40%
60%
80%
100%
Yes No
67%
33%
USMLE Step 2 CSTest Required?
Obstetrics and GynecologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure OG-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
25%
48%
27%
COMLEX-USA Level 2-CE*Test Required?
N=440%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5% 7%
89%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6% 9%
85%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
46%
13%
41%
COMLEX-USA Level 2-PE*Test Required?
N=55N=44
N=48 N=48
N=46 N=34
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 83
Figure OG-5Obstetrics and GynecologyProgram's Interview Activities
0
100
200
300
400
500
600
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
5
589
87 76 66
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
46% 46%
Average Percentage of Applications Rejected and Reviewed
N=54N=51N=73* N=56*N=56N=55
* Data from the NRMP database. All responding programs included
N=56
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
10%
70%
13%7%
1%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
17%
47%
31%
5%
Percentage of Program's Interviews Conducted During EachTime Period
N=57N=57
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
53%
28%
19%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
2% 3% 0% 3%9%
83%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
25%
0%
0%
75%
Reason for offering more interview invitations to applicants thaninterview slots to fill
7%Yes
93%No
Program offered more interview invitations toapplicants than interview slots to fill
N=58
N=4
N=58N=57
NRMP Program Director Survey Results, 2020 84
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
5.0
4.9
4.9
4.9
4.9
4.8
4.4
4.2
4.1
3.7
3.9
3.1
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure OG-6Obstetrics and GynecologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=55
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
32%27%
9%
36%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=56
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
2%16%
39%28%
47%
66%
40%
7%
63% 32%
34% 56%
44%
30%
54%
93%
35%
53%
27%16%
9% 4% 5%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian2%
9%18%
41%30%
49%
68%
41%
5%
58%30%
34% 54%
44%
28%
54%
93%
33%
52%
25%16%
7% 4% 5%
Often Seldom Never
Rankingn=57 n=57
NRMP Program Director Survey Results, 2020 85
Orthopaedic Surgery(N=49)
NRMP Program Director Survey Results, 2020 86
Figure OS-1Orthopaedic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=36)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.14.5
3.83.6
3.94.4
4.04.5
4.84.0
4.44.6
4.14.7
4.53.84.0
4.53.8
4.64.1
4.43.9
3.53.73.93.83.7
4.34.1
4.93.83.83.83.8
3.32.0
100% 50% 0%
97%86%
75%67%64%
50%72%
81%31%
75%72%69%
58%78%
58%47%
58%22%
69%14%
50%14%
42%47%
39%19%
36%42%
8%50%
25%17%
33%11%11%22%
6%
0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 87
Orthopaedic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=36)
Figure OS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.84.84.84.7
4.23.94.14.0
4.84.44.5
3.64.54.4
3.54.14.0
3.94.8
2.03.73.84.0
3.84.03.9
3.53.8
4.53.1
3.43.94.0
3.94.8
4.04.2
3.5
5.0
4.55.0
100% 50% 0%
89%95%89%
92%68%65%
41%54%54%
24%70%
46%65%
49%30%
49%19%
43%22%11%
41%16%22%
49%5%
24%5%
27%5%
19%27%27%
14%22%14%5%
16%0%0%3%5%0%5%3%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 88
170
180
190
200
210
220
230
240
250
260
Orthopaedic SurgeryProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure OS-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
82%
18%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
71%
29%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
16%
81%
3%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
19%27%
54%
Test Required?
N=37N=34
N=37N=31
N=7N=29 N=23 N=6
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 89
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%
60%
25%
COMLEX-USA Level 1*Test Required?
N=36
0%
20%
40%
60%
80%
100%
Yes No
36%
64%
USMLE Step 2 CSTest Required?
Orthopaedic SurgeryProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure OS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%
30%
55%
COMLEX-USA Level 2-CE*Test Required?
N=320%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 0%
97%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
11%
89%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20% 20%
60%
COMLEX-USA Level 2-PE*Test Required?
N=36N=32
N=20 N=20
N=20 N=18
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 90
Figure OS-5Orthopaedic SurgeryProgram's Interview Activities
0
100
200
300
400
500
600
700
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
4
603
71 59 51
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
50% 46%
Average Percentage of Applications Rejected and Reviewed
N=38N=37N=49* N=39*N=39N=38
* Data from the NRMP database. All responding programs included
N=38
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
19%
60%
16%
5%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1% 0%6%
38%
55%
Percentage of Program's Interviews Conducted During EachTime Period
N=38N=38
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
54%
21%
31%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0%11% 8%
82%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
50%
50%
25%
25%
Reason for offering more interview invitations to applicants thaninterview slots to fill
21%Yes
79%No
Program offered more interview invitations toapplicants than interview slots to fill
N=39
N=8
N=38N=39
NRMP Program Director Survey Results, 2020 91
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.9
4.9
4.6
4.6
4.4
4.1
3.6
3.5
3.6
3.4
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure OS-6Orthopaedic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=36
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
38%32%
8%
24%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=37
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
3%13%
31%
54% 49%
74%90%
46%
10%
67%
49%
46%49%
23%8%
54%
87%
21% 21%
3% 3% 3%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian3%
13%
33%
53% 51%
77%90%
50%
11%
67%
46%
47% 46%
21%10%
50%
87%
21% 21%
3% 3%
Often Seldom Never
Rankingn=39 n=39
NRMP Program Director Survey Results, 2020 92
Otolaryngology(N=30)
NRMP Program Director Survey Results, 2020 93
Figure O-1OtolaryngologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=25)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.04.6
3.53.7
4.14.44.4
4.24.8
4.04.5
4.04.54.5
4.13.63.7
3.43.5
4.53.8
2.74.3
3.53.73.9
3.33.8
3.34.1
4.43.0
3.7
4.43.3
3.03.0
100% 50% 0%
100%96%
64%72%
76%56%
84%84%
48%68%64%64%
68%68%
52%36%
64%20%
76%16%
56%12%
64%60%
52%32%
40%48%
16%72%
20%12%
40%0%
32%32%
4%4%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 94
OtolaryngologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=25)
Figure O-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.84.84.8
4.74.5
3.93.3
4.44.54.7
4.43.9
4.44.6
3.64.0
2.73.7
4.3
4.03.0
4.53.8
3.53.9
03.8
4.33.7
3.34.5
3.73.8
4.34.2
3.83.7
3.0
3.5
100% 50% 0%
96%88%88%84%80%
36%24%
64%80%
64%60%
52%56%
64%44%
36%12%
40%24%
4%48%
8%44%
36%8%
48%4%
36%12%
36%16%
52%28%20%24%
48%40%
0%0%4%
12%0%
16%0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 95
170
180
190
200
210
220
230
240
250
260
OtolaryngologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure O-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
67%
33%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
65%
30%
4%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
32%
60%
8%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
21%13%
67%
Test Required?
N=25N=24
N=24N=23
N=15 N=12
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 96
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
27%
45%
27%
COMLEX-USA Level 1*Test Required?
N=23
0%
20%
40%
60%
80%
100%
Yes No
35%
65%
USMLE Step 2 CSTest Required?
OtolaryngologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure O-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%
0%
82%
COMLEX-USA Level 2-CE*Test Required?
N=200%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
5%0%
95%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%0%
90%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
9% 9%
82%
COMLEX-USA Level 2-PE*Test Required?
N=23N=20
N=11 N=11
N=11 N=10
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 97
Figure O-5OtolaryngologyProgram's Interview Activities
0
50
100
150
200
250
300
350
400
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
3
364
57 45 41
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
20%
81%
Average Percentage of Applications Rejected and Reviewed
N=26N=24N=30* N=26*N=26N=26
* Data from the NRMP database. All responding programs included
N=26
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
5%
37%45%
11%2%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0% 1%
15%
49%
36%
Percentage of Program's Interviews Conducted During EachTime Period
N=25N=25
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
64%
20%
16%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0%
12% 8%
24%
56%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
29%
57%
14%
29%
Reason for offering more interview invitations to applicants thaninterview slots to fill
28%Yes
72%No
Program offered more interview invitations toapplicants than interview slots to fill
N=25
N=7
N=25N=25
NRMP Program Director Survey Results, 2020 98
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.8
5.0
4.9
4.8
4.8
4.3
4.4
4.2
3.8
3.6
3.9
3.8
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure O-6OtolaryngologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=24
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
28%20%
24%32%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=25
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
4%
40%56%
20%
40%
72%
16%
64%
40%
40%
76%
56%
28%
72%96%
36%20%
4% 4% 4%12%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian4% 4%
44%60%
32%44%
75%
30%
52%
36%
36%
64%52%
25%
57%96%
44%
20%4% 4% 4%
13%
Often Seldom Never
Rankingn=25 n=25
NRMP Program Director Survey Results, 2020 99
Pathology(N=29)
NRMP Program Director Survey Results, 2020 100
Figure PA-1PathologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=24)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.14.6
4.23.73.9
4.84.1
4.34.5
3.94.03.93.8
4.54.3
4.14.03.9
3.24.6
3.33.83.8
3.33.4
3.83.5
3.83.8
3.55.0
3.74.2
4.53.73.7
4.03.0
100% 50% 0%
92%100%
83%100%
63%100%
63%67%
50%63%63%
50%58%
63%58%58%
46%63%
29%46%42%
54%46%
33%33%
71%21%
38%
33%46%
58%13%
42%13%
29%29%
29%17%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 101
PathologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=24)
Figure PA-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.74.84.8
4.74.5
3.73.8
4.14.4
4.74.0
3.84.64.6
3.63.8
4.03.8
4.45
3.83.5
3.83.8
4.43.33.53.73.7
3.53.73.6
3.53.6
4.83.7
4.54.0
4.33.0
4.04.0
3.53.5
100% 50% 0%
88%92%
83%96%
67%63%
54%67%
71%75%
50%54%50%
42%58%
50%54%
33%42%42%38%
42%38%
21%42%
25%33%29%
46%25%13%
33%21%21%
33%25%
33%17%13%
4%8%
17%17%
8%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 102
170
180
190
200
210
220
230
240
250
260
PathologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PA-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
21%
54%
25%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
26%
48%
26%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
46% 46%
8%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
29%21%
50%
Test Required?
N=24N=24
N=24N=23
N=7 N=6
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 103
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
43% 43%
14%
COMLEX-USA Level 1*Test Required?
N=24
0%
20%
40%
60%
80%
100%
Yes No
33%
67%
USMLE Step 2 CSTest Required?
PathologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PA-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
24% 24%
52%
COMLEX-USA Level 2-CE*Test Required?
N=240%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
4% 0%
96%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%0%
90%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
24%
10%
67%
COMLEX-USA Level 2-PE*Test Required?
N=24N=24
N=21 N=21
N=21 N=20
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 104
Figure PA-5PathologyProgram's Interview Activities
0
50
100
150
200
250
300
350
400
450
500
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
4
456
70 53 45
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
40%47%
Average Percentage of Applications Rejected and Reviewed
N=21N=21N=29* N=22*N=22N=22
* Data from the NRMP database. All responding programs included
N=22
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
22%
45%
21%
9%3%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
16%
34% 34%
15%
Percentage of Program's Interviews Conducted During EachTime Period
N=23N=23
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
54%
21%
25%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0% 4%
96%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
63%
25%
25%
50%
Reason for offering more interview invitations to applicants thaninterview slots to fill
33%Yes
67%No
Program offered more interview invitations toapplicants than interview slots to fill
N=24
N=8
N=23N=24
NRMP Program Director Survey Results, 2020 105
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.9
4.9
4.7
4.3
4.3
4.5
3.7
3.7
3.1
3.2
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure PA-6PathologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=21
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
18%
59%
9%14%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=22
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
9% 9%
41%
13%8%
30% 13%
57%
30% 26%
50%
52%
92%
70%78%
35%
70% 74%
9%
35%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian5% 5%
43%
14%9%
36%18%
62%
32%23%
48%
45%
91%
64%77%
33%
68%77%
10%
41%
Often Seldom Never
Rankingn=24 n=23
NRMP Program Director Survey Results, 2020 106
Pediatrics(N=52)
NRMP Program Director Survey Results, 2020 107
Figure P-1PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=38)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.83.94.0
3.34.4
4.14.14.44.5
4.04.4
4.04.0
4.33.94.0
3.74.0
3.74.5
3.74.0
3.63.53.73.93.83.94.0
3.34.8
4.33.3
3.63.9
2.72.83.0
100% 50% 0%
92%68%
84%71%
82%47%
76%74%
82%68%68%
61%66%61%63%
68%53%
87%58%
74%58%
76%47%45%
58%71%
50%39%
58%29%32%29%
18%29%
21%18%13%11%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 108
PediatricsPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=38)
Figure P-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.84.84.7
3.83.94.14.1
4.64.04.14.24.34.4
3.84.24.2
4.04.6
74.14.0
4.43.9
4.64.14.1
3.74.0
3.63.8
3.64.1
3.84.4
4.03.0
3.93.83.94.0
3.72.8
3.8100% 50% 0%
100%97%
100%95%
78%78%
86%62%
70%70%70%73%76%
65%54%
68%73%
57%73%76%
54%81%
41%43%
68%51%
70%38%
54%41%
51%35%32%27%
35%19%14%19%
35%24%19%19%16%11%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 109
170
180
190
200
210
220
230
240
250
260
PediatricsProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure P-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
3%
72%
26%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
5%
87%
8%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
51%46%
3%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
18%
36%
46%
Test Required?
N=39N=39
N=39N=39
N=12N=18 N=17 N=11
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 110
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
37%
58%
5%
COMLEX-USA Level 1*Test Required?
N=39
0%
20%
40%
60%
80%
100%
Yes No
51% 49%
USMLE Step 2 CSTest Required?
PediatricsProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure P-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
16%
39%45%
COMLEX-USA Level 2-CE*Test Required?
N=310%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 0%
97%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
3% 0%
97%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
41%
8%
51%
COMLEX-USA Level 2-PE*Test Required?
N=39N=31
N=38 N=38
N=37 N=29
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 111
Figure P-5PediatricsProgram's Interview Activities
0
200
400
600
800
1,000
1,200
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
11
1,113
230170 153
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
49% 45%
Average Percentage of Applications Rejected and Reviewed
N=40N=36N=52* N=40*N=39N=40
* Data from the NRMP database. All responding programs included
N=39
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
31%40%
17%9%
3%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
19%
38%32%
11%
Percentage of Program's Interviews Conducted During EachTime Period
N=38N=38
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
72%
10%
26%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
3% 0% 0%5%
24%
68%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
78%
39%
28%
22%
Reason for offering more interview invitations to applicants thaninterview slots to fill
43%Yes
58%No
Program offered more interview invitations toapplicants than interview slots to fill
N=40
N=18
N=38N=39
NRMP Program Director Survey Results, 2020 112
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.8
4.9
4.9
4.3
4.2
4.1
4.1
3.6
3.1
3.1
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure P-6PediatricsProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=35
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
32%41%
16% 14%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=37
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
15%8% 13%
66%
8%
53%
23%
50%
43%
55%
32%
57%
100%
48%
78%
35%50%
32%
3%
35%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
15%8%
15%
66%
8%
49%
23%
48%
41%
48%
26%
53%
100%
51%
78%
38%51%
38%
8%
40%
Often Seldom Never
Rankingn=40 n=40
NRMP Program Director Survey Results, 2020 113
Physical Medicine and Rehabilitation(N=21)
NRMP Program Director Survey Results, 2020 114
Figure PMR-1Physical Medicine and RehabilitationPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=16)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.63.9
3.54.2
3.74.3
3.84.24.2
3.43.94.14.2
4.63.83.9
3.73.7
3.44.2
3.73.73.6
3.33.4
3.03.33.5
3.93.4
5.03.73.9
3.03.2
2.74.0
3.7100% 50% 0%
94%94%
63%81%
88%88%
69%88%
63%75%
69%63%
69%69%
56%63%
69%94%
50%
56%63%63%
69%56%
50%25%
44%38%
50%44%
31%19%
50%6%
38%19%6%
19%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 115
Physical Medicine and RehabilitationPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=16)
Figure PMR-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.74.84.74.8
3.83.53.6
3.94.6
4.34.3
3.64.14.3
3.93.6
4.03.6
4.21.1
3.63.6
4.23.4
4.23.7
4.04.0
3.53.84.0
3.73.6
3.05.0
3.43.6
3.5
2.03.5
5.03.0
5.0100% 50% 0%
100%100%
88%81%
69%63%
31%81%
63%69%69%
44%63%
50%44%44%
31%31%31%
38%31%
56%38%
31%31%
44%31%
44%25%
31%25%
38%31%
13%19%
31%50%
25%0%6%
13%6%6%6%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 116
170
180
190
200
210
220
230
240
250
260
Physical Medicine and RehabilitationProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PMR-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
13%
75%
13%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
25%
75%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
35%
53%
12%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
29%
12%
59%
Test Required?
N=17N=16
N=17N=16
N=8 N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 117
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
35%
53%
12%
COMLEX-USA Level 1*Test Required?
N=17
0%
20%
40%
60%
80%
100%
Yes No
47%53%
USMLE Step 2 CSTest Required?
Physical Medicine and RehabilitationProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PMR-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
29%
12%
59%
COMLEX-USA Level 2-CE*Test Required?
N=150%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
47%
0%
53%
COMLEX-USA Level 2-PE*Test Required?
N=17N=15
N=17 N=17
N=17 N=15
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 118
Figure PMR-5Physical Medicine and RehabilitationProgram's Interview Activities
0
50
100
150
200
250
300
350
400
450
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
5
436
83 71 58
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
31%
64%
Average Percentage of Applications Rejected and Reviewed
N=19N=19N=21* N=19*N=19N=19
* Data from the NRMP database. All responding programs included
N=19
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
14%
69%
10% 6%1%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
2%10%
34%
45%
8%
Percentage of Program's Interviews Conducted During EachTime Period
N=17N=17
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
71%
18%
24%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0%5% 5%
26%
63%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
75%
50%
50%
25%
Reason for offering more interview invitations to applicants thaninterview slots to fill
21%Yes
79%No
Program offered more interview invitations toapplicants than interview slots to fill
N=19
N=4
N=19N=17
NRMP Program Director Survey Results, 2020 119
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.9
4.8
4.8
4.8
4.2
4.0
4.0
3.8
3.2
3.1
3.4
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure PMR-6Physical Medicine and RehabilitationProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=16
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
38%
13% 13%
44%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=16
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
13% 12%
41%
65%
19%6%
76%63%
82%
53%
35%
63%94%
24%
100%
25%
6% 6%19%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
6%18% 12%
40%
63%
18%6%
71%59%
82%
60%
38%
65%94%
24%
100%
24%
6%18%
Often Seldom Never
Rankingn=17 n=17
NRMP Program Director Survey Results, 2020 120
Plastic Surgery(N=16)
NRMP Program Director Survey Results, 2020 121
Figure PS-1Plastic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=12)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.44.9
4.03.0
3.84.8
4.04.85.0
3.84.6
3.84.64.7
4.03.8
3.65.0
3.75.0
3.6
3.54.0
3.33.0
3.64.3
4.45.0
3.54.3
3.04.3
3.54.0
100% 50% 0%
92%92%
67%67%67%
33%67%
42%58%
67%67%
50%67%
50%50%
42%42%
8%92%
25%42%
0%58%
33%58%
8%50%
33%0%
75%17%17%
58%8%
50%25%
8%0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 122
Plastic SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=12)
Figure PS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.95.0
4.84.84.9
3.93.6
4.24.84.8
3.93.6
4.64.4
3.03.3
4.03.7
4.7
4.5
4.84.04.04.0
3.5
3.63.5
4.34.04.0
5.04.34.4
3.0
3.33.5
100% 50% 0%
100%82%
100%91%
82%91%
64%45%45%
55%64%
45%73%
45%55%55%
36%55%
27%0%
36%0%
36%55%
9%45%
0%36%
0%45%
36%55%
36%27%
9%55%
45%0%9%0%0%
27%18%
0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 123
170
180
190
200
210
220
230
240
250
260
Plastic SurgeryProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PS-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
82%
18%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
80%
20%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
17%
83%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
8%
50%42%
Test Required?
N=12N=11
N=12N=10
N=6N=9 N=8 N=5
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 124
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
29%
43%
29%
COMLEX-USA Level 1*Test Required?
N=12
0%
20%
40%
60%
80%
100%
Yes No
50% 50%
USMLE Step 2 CSTest Required?
Plastic SurgeryProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
29%
71%
COMLEX-USA Level 2-CE*Test Required?
N=90%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0%
20%
80%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
17% 17%
67%
COMLEX-USA Level 2-PE*Test Required?
N=12N=9
N=7 N=7
N=6 N=5
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 125
Figure PS-5Plastic SurgeryProgram's Interview Activities
0
40
80
120
160
200
240
280
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
2
249
45 35 32
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
21%
76%
Average Percentage of Applications Rejected and Reviewed
N=12N=12N=16* N=12*N=12N=12
* Data from the NRMP database. All responding programs included
N=12
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0% 0%
62%
27%
11%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0% 0%
11%
44% 44%
Percentage of Program's Interviews Conducted During EachTime Period
N=11N=11
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
67%
17%
25%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0%8% 8%
83%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
25%Yes
75%No
Program offered more interview invitations toapplicants than interview slots to fill
N=12
N=12N=12
Reasons for "offering more interview invitations to applicants
than interview slots to fill" not reported because of low response
rate.
NRMP Program Director Survey Results, 2020 126
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.8
4.7
4.9
4.6
4.9
4.2
4.1
3.6
4.2
3.5
3.9
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure PS-6Plastic SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=10
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
42%
25%
0%
33%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=12
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
33%50%
42% 42%
73%
33%
67%
58%42%
33%50%
27%
58%
100%
33%
8% 8%25%
8% 8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
33%50%
42% 42%
73%
33%
67%
58%42%
33%50%
27%
67%
100%
33%
8% 8%25%
8%
Often Seldom Never
Rankingn=12 n=12
NRMP Program Director Survey Results, 2020 127
Psychiatry(N=57)
NRMP Program Director Survey Results, 2020 128
Figure PSY-1PsychiatryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=37)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.54.1
3.84.3
4.04.6
4.04.34.3
3.84.5
4.34.34.54.6
3.84.3
3.53.3
4.34.1
3.64.0
3.73.7
4.23.7
4.23.63.7
5.04.1
3.83.9
4.33.8
3.03.0
100% 50% 0%
62%73%76%
86%78%81%
62%65%
70%41%
65%51%
76%57%
70%73%
57%51%
32%54%
46%57%
46%41%
32%57%
41%49%
54%16%
41%24%16%
30%19%16%14%
3%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 129
PsychiatryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=37)
Figure PSY-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.94.94.94.9
4.03.43.6
4.44.9
4.54.5
4.24.54.6
4.23.83.8
4.14.3
73.9
3.34.3
3.34.14.2
3.84.14.14.2
3.54.1
4.03.5
4.84.1
4.03.3
3.84.5
3.92.4
4.33.5
100% 50% 0%
100%97%95%97%
59%49%
57%78%
59%62%
43%68%
54%54%
68%35%
59%43%43%
51%46%41%
49%22%
35%35%
49%32%
46%19%
30%22%24%22%
30%22%
5%27%22%16%8%
14%8%
11%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 130
170
180
190
200
210
220
230
240
250
260
PsychiatryProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure PSY-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
11%
45% 45%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
13%
55%
32%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
78%
19%
3%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
50%
31%
19%
Test Required?
N=37N=38
N=36N=38
N=5N=7 N=6
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
NRMP Program Director Survey Results, 2020 131
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
74%
24%
3%
COMLEX-USA Level 1*Test Required?
N=35
0%
20%
40%
60%
80%
100%
Yes No
63%
37%
USMLE Step 2 CSTest Required?
PsychiatryProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure PSY-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
57%
26%17%
COMLEX-USA Level 2-CE*Test Required?
N=270%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
7%0%
93%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13%
0%
87%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
53%
9%
38%
COMLEX-USA Level 2-PE*Test Required?
N=35N=27
N=34 N=35
N=34 N=23
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 132
Figure PSY-5PsychiatryProgram's Interview Activities
0
200
400
600
800
1,000
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
7
909
86 74 58
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
49%
36%
Average Percentage of Applications Rejected and Reviewed
N=38N=35N=57* N=39*N=39N=39
* Data from the NRMP database. All responding programs included
N=39
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
21%
48%
19%9%
4%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
2%
23%33%
26%16%
Percentage of Program's Interviews Conducted During EachTime Period
N=39N=39
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
55%
26%
26%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0%
13%
88%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
60%
47%
53%
20%
Reason for offering more interview invitations to applicants thaninterview slots to fill
38%Yes
63%No
Program offered more interview invitations toapplicants than interview slots to fill
N=40
N=15
N=40N=38
NRMP Program Director Survey Results, 2020 133
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
5.0
4.9
4.9
4.9
4.9
4.3
4.4
4.1
3.9
3.4
3.4
2.8
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure PSY-6PsychiatryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=37
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
38%30%
16%22%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=37
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
3% 5%
24%11%
24%
55%
29%
8%
61%
16%
47%
55%
61%
37%
63%92%
37%
79%
29% 34%
16%8% 8%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
8% 3%
27%16%
24%
58%
29%
8%
58%
14%
49%55%
63%
32%
63%92%
34%
84%
24% 29%13% 11% 8%
Often Seldom Never
Rankingn=38 n=38
NRMP Program Director Survey Results, 2020 134
Radiation Oncology(N=17)
NRMP Program Director Survey Results, 2020 135
Figure RO-1Radiation OncologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=15)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
3.74.3
3.84.3
3.54.54.44.4
4.74.0
4.54.34.44.5
4.13.3
3.6
3.45.0
3.5
4.33.53.7
4.03.2
3.8
4.75.0
4.03.8
2.04.5
3.04.0
100% 50% 0%
93%93%
53%87%
93%80%
93%80%
27%73%
67%73%
80%67%
73%40%
67%0%
60%7%
40%0%
60%60%
73%13%
33%40%
0%73%
13%7%
40%
7%47%
20%
7%0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 136
Radiation OncologyPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=15)
Figure RO-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.74.9
4.84.7
4.23.94.0
4.44.6
4.44.3
3.34.54.6
4.03.94.0
3.74.5
3.6
4.34.0
5.03.8
5.03.0
3.83.0
4.43.8
3.05.0
4.14.0
4.0
3.0
100% 50% 0%
79%93%
86%93%
71%50%
36%79%
57%57%
71%43%
57%57%
50%50%
29%50%
14%0%
50%0%
29%14%
7%29%
0%21%
7%29%
7%57%
29%29%
7%57%
29%0%0%
14%0%0%
14%0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 137
Radiation OncologyProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure RO-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
40%
60%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
50% 50%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
67%
27%
7%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
8% 8%
85%
Test Required?
N=15N=15
N=13N=14
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
NRMP Program Director Survey Results, 2020 138
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
25% 25%
50%
COMLEX-USA Level 1*Test Required?
N=13
0%
20%
40%
60%
80%
100%
Yes No
8%
92%
USMLE Step 2 CSTest Required?
Radiation OncologyProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure RO-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13% 13%
75%
COMLEX-USA Level 2-CE*Test Required?
N=120%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
8%0%
92%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13%
0%
88%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13% 13%
75%
COMLEX-USA Level 2-PE*Test Required?
N=13N=12
N=8 N=8
N=8 N=8
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 139
Figure RO-5Radiation OncologyProgram's Interview Activities
0
20
40
60
80
100
120
140
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
3
123
4335 31
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
19%
83%
Average Percentage of Applications Rejected and Reviewed
N=13N=13N=17* N=14*N=14N=14
* Data from the NRMP database. All responding programs included
N=14
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
60%
32%
4% 3%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0% 2%12%
48%38%
Percentage of Program's Interviews Conducted During EachTime Period
N=15N=15
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
60%
40%
7%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0%7%
93%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
20%Yes
80%No
Program offered more interview invitations toapplicants than interview slots to fill
N=15
N=15N=15
Reasons for "offering more interview invitations to applicants
than interview slots to fill" not reported because of low response
rate.
NRMP Program Director Survey Results, 2020 140
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.8
4.9
4.9
4.6
4.8
4.3
4.1
4.1
3.5
2.8
3.3
3.7
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure RO-6Radiation OncologyProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=14
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
36%
14% 14%
36%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=14
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
20% 27%
60%
33%47%
80%
27%
60%
67%
40%
67%53%
20%
67%
100%
20%7% 7%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
21% 27%
64%
29%
64%80%
27%
57%
67%
36%
71%
36%20%
67%
100%
21%7% 7%
Often Seldom Never
Rankingn=15 n=15
NRMP Program Director Survey Results, 2020 141
Radiology(N=36)
NRMP Program Director Survey Results, 2020 142
Figure RD-1Radiology-DiagnosticPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=27)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.34.14.0
3.54.2
4.04.34.4
4.84.34.3
3.94.24.5
4.13.5
4.04.3
3.64.9
3.84.44.4
3.63.6
4.43.4
4.13.8
3.74.6
4.03.8
3.73.0
3.53.5
4.5100% 50% 0%
100%81%78%
89%93%
52%70%
63%81%
89%59%
70%48%
74%59%
44%52%44%
59%
63%48%
30%63%
52%41%
41%56%
41%
19%33%
44%33%
19%11%
7%33%
7%7%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 143
Radiology-DiagnosticPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=27)
Figure RD-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.74.84.8
4.64.14.2
4.14.3
4.74.14.34.3
4.54.3
3.64.3
3.74.1
4.85
4.44.04.14.0
4.73.8
3.64.34.24.1
3.83.5
3.94.1
4.94.0
3.84.5
3.84.3
3.92.7
4.23.0
100% 50% 0%
93%93%93%
81%74%
81%70%
59%74%
52%63%
85%59%
67%63%
78%48%
41%59%
30%63%
44%37%
59%44%44%
19%59%
37%41%41%41%
30%33%
44%15%19%
33%15%
30%7%
11%22%
7%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 144
170
180
190
200
210
220
230
240
250
260
Radiology-DiagnosticProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure RD-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
41%
56%
4%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
37%
59%
4%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
30%
70%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
26% 22%
52%
Test Required?
N=27N=27
N=27N=27
N=6N=18 N=16 N=6
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 145
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
32% 32%36%
COMLEX-USA Level 1*Test Required?
N=27
0%
20%
40%
60%
80%
100%
Yes No
37%
63%
USMLE Step 2 CSTest Required?
Radiology-DiagnosticProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure RD-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
23% 23%
55%
COMLEX-USA Level 2-CE*Test Required?
N=210%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6% 6%
89%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
32%
5%
64%
COMLEX-USA Level 2-PE*Test Required?
N=27N=21
N=22 N=22
N=22 N=18
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 146
Figure RD-5Radiology-DiagnosticProgram's Interview Activities
0
100
200
300
400
500
600
700
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
6
615
115 96 87
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
37%
55%
Average Percentage of Applications Rejected and Reviewed
N=26N=26N=36* N=26*N=27N=27
* Data from the NRMP database. All responding programs included
N=27
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
27%
52%
9% 8%4%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%
22%
33% 31%
14%
Percentage of Program's Interviews Conducted During EachTime Period
N=27N=27
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
74%
15%
19%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
4% 0% 4% 0%
21%
71%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
86%
64%
36%
29%
Reason for offering more interview invitations to applicants thaninterview slots to fill
50%Yes
50%No
Program offered more interview invitations toapplicants than interview slots to fill
N=28
N=14
N=28N=27
NRMP Program Director Survey Results, 2020 147
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
4.7
4.9
4.8
4.6
4.8
4.2
4.4
3.4
3.7
3.0
3.1
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure RD-6Radiology-DiagnosticProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=26
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
37%33%
22%26%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=27
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
4%19%
4%15%
56%
15%
67%
26%
62%
74%67%
33%
63%100%
30%
74%
19% 22% 19%11%
22%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian4%
19%4%
19%
56%
15%
63%
28%
58%
70%
59%
30%
63%100%
33%
72%
23% 26% 22%15%
22%
Often Seldom Never
Rankingn=27 n=27
NRMP Program Director Survey Results, 2020 148
Surgery(N=70)
NRMP Program Director Survey Results, 2020 149
Figure GS-1SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=42)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.34.24.3
4.03.9
4.54.24.1
4.94.34.24.4
4.34.54.6
4.34.0
4.24.2
4.93.9
4.64.3
3.63.6
4.14.04.24.4
3.84.7
4.23.5
4.04.0
3.94.34.3
100% 50% 0%
90%88%86%81%
60%67%67%
60%45%
62%67%69%71%69%
60%45%
69%45%
52%
33%43%
38%50%52%48%43%
40%40%
19%38%
29%
29%10%12%14%19%
10%7%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 150
SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=42)
Figure GS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.84.94.84.8
4.04.24.3
4.64.84.7
4.53.8
4.44.5
4.04.24.24.2
4.75
3.84.24.1
4.04.9
3.94.3
3.84.2
3.43.83.9
4.34.0
4.84.2
4.54.54.7
4.03.94.0
4.54.5
100% 50% 0%
95%95%
83%88%
71%78%
71%73%
59%56%
66%27%
51%51%
59%44%
32%59%
37%27%
44%32%
39%41%
20%34%
15%34%
24%29%32%34%
22%32%
12%22%10%20%7%
15%10%10%7%5%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 151
170
180
190
200
210
220
230
240
250
260
SurgeryProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure GS-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
44%
56%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
54%46%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
17%
83%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
12%
71%
17%
Test Required?
N=41N=41
N=41N=39
N=27N=31 N=29 N=25
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 152
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
28%
50%
22%
COMLEX-USA Level 1*Test Required?
N=41
0%
20%
40%
60%
80%
100%
Yes No
68%
32%
USMLE Step 2 CSTest Required?
SurgeryProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure GS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
13%
50%
38%
COMLEX-USA Level 2-CE*Test Required?
N=350%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
6% 6%
89%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
10%3%
86%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
41%
13%
47%
COMLEX-USA Level 2-PE*Test Required?
N=41N=35
N=32 N=32
N=32 N=29
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 153
Figure GS-5SurgeryProgram's Interview Activities
0
100
200
300
400
500
600
700
800
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
5
739
75 61 50
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
56%
37%
Average Percentage of Applications Rejected and Reviewed
N=44N=44N=70* N=44*N=44N=46
* Data from the NRMP database. All responding programs included
N=45
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
9%
53%
19%13%
6%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%9%
34% 37%
19%
Percentage of Program's Interviews Conducted During EachTime Period
N=42N=42
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
65%
23%
16%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 2% 0%7% 4%
87%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
86%
14%
14%
21%
Reason for offering more interview invitations to applicants thaninterview slots to fill
30%Yes
70%No
Program offered more interview invitations toapplicants than interview slots to fill
N=46
N=14
N=46N=43
NRMP Program Director Survey Results, 2020 154
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.9
5.0
4.9
5.0
4.8
4.7
4.4
4.2
4.1
4.0
3.7
3.5
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure GS-6SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=40
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
29%
52%
17% 14%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=42
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
5% 7% 12%28%
15%
39%
70%
40%
2%
46% 31%
42%
49%
44%
30%
35%93%
46%57%
30% 37%
17%25%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian5% 10% 12%
30%23%
45%
73%
45%
5%
45%32%
40%45%
39%
25%
33%90%
45%56%
30% 32%16%
3%
22%
Often Seldom Never
Rankingn=41 n=41
NRMP Program Director Survey Results, 2020 155
Transitional Year(N=22)
NRMP Program Director Survey Results, 2020 156
Figure TY-1Transitional YearPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=13)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.53.5
4.54.0
3.95.0
3.63.8
4.74.44.54.54.74.8
4.34.4
4.24.5
4.04.7
3.54.6
4.33.2
3.84.04.04.0
4.53.8
4.55.0
3.3
4.02.0
100% 50% 0%
92%62%
85%85%
62%23%
46%31%
85%46%
31%54%
23%31%
54%77%
54%77%77%
85%38%
77%23%
38%38%
46%54%
15%62%
46%15%
31%0%
23%0%8%8%0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 157
Transitional YearPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=13)
Figure TY-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.74.7
4.44.1
3.64.54.64.64.6
5.04.3
3.83.5
5.04.04.24.4
3.94.5
33.8
4.6
4.14.6
4.04.64.5
4.33.3
4.34.04.0
3.33.0
5.0
03.5
5.03.0
0
100% 50% 0%
85%92%
85%85%
54%46%
62%54%
62%15%
46%62%
15%8%
77%38%
54%54%
46%54%
46%38%
0%62%
38%23%
38%31%
46%23%23%
31%8%
23%8%8%0%8%
15%8%8%8%0%0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 158
170
180
190
200
210
220
230
240
250
260
Transitional YearProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure TY-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
36%
64%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
43%
57%
0%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
21%
79%
0%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
14%
64%
21%
Test Required?
N=14N=14
N=14N=14
N=9N=11 N=10 N=8
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
170
180
190
200
210
220
230
240
250
260
USMLE Step 2 CKUSMLE Step 1 USMLE Step 1 USMLE Step 2 CK
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
NRMP Program Director Survey Results, 2020 159
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%
85%
0%
COMLEX-USA Level 1*Test Required?
N=14
0%
20%
40%
60%
80%
100%
Yes No
57%
43%
USMLE Step 2 CSTest Required?
Transitional YearProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure TY-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
15%
62%
23%
COMLEX-USA Level 2-CE*Test Required?
N=110%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
18%
0%
82%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
20%
0%
80%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
46%
15%
38%
COMLEX-USA Level 2-PE*Test Required?
N=14N=11
N=13 N=13
N=13 N=10
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 160
Figure TY-5Transitional YearProgram's Interview Activities
0
100
200
300
400
500
600
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
8
515
9469 60
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
57%
38%
Average Percentage of Applications Rejected and Reviewed
N=14N=14N=22* N=14*N=14N=14
* Data from the NRMP database. All responding programs included
N=14
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
21%
38%
23%
11%7%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
1%
16%
36%27%
20%
Percentage of Program's Interviews Conducted During EachTime Period
N=13N=13
0% 20% 40% 60% 80% 100%
In the order in which applicantsresponded to interview offers
Applicants submitted a rank order ofinterview date preferences by a
specified deadline (e.g., within 72hours), with no preference given to the
order of responses
Other
71%
0%
29%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0%
27%
73%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
0% 20% 40% 60% 80% 100%
I wanted to ensure we had an adequatenumber of alternate candidates in theevent other applicants cancelled at the
last minute
I wanted to ensure all interview spotsfilled in a timely manner
I worried I would not fill all my interviewslots
Other
44%
44%
11%
11%
Reason for offering more interview invitations to applicants thaninterview slots to fill
56%
Yes
43.75%
No
Program offered more interview invitationsto applicants than interview slots to fill
N=16
N=9
N=15N=14
NRMP Program Director Survey Results, 2020 161
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
4.8
4.8
4.8
4.8
4.6
3.9
4.2
4.3
4.0
3.1
4.1
3.3
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure TY-6Transitional YearProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=10
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
50%
29%
7%14%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=14
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
14%
36%
7%
36%
79%
57%
57%
14%
36%
57%
57%
21%
36%
100%
29%
86%
29%36%
7% 7%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
14%
38%
18%29%
79%
57%
57%
23%
38%
55%
64%
21%
36%
100%
29%
77%
23% 27%
7% 7%
Often Seldom Never
Rankingn=14 n=14
NRMP Program Director Survey Results, 2020 162
Vascular Surgery(N=10)
NRMP Program Director Survey Results, 2020 163
Figure VS-1Vascular SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Selecting Applicants to Interview(N=6)
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.35.0
4.44.5
4.05.0
3.54.54.7
4.04.8
4.04.3
4.85.0
4.53.0
4.34.04.0
3.54.54.7
3.34.0
5.04.0
3.54.0
3.75.0
3.54.7
3.0
4.0
100% 50% 0%
100%100%
83%67%67%
100%50%
67%50%50%
83%50%50%
67%83%
67%33%
50%50%
17%33%33%
50%50%
67%17%17%
67%50%50%
17%33%
50%17%0%0%
33%0%
USMLE Step 1 score
Letters of recommendation in the specialty
USMLE Step 2 CK score
Personal Statement
Medical Student Performance Evaluation (MSPE/Dean's Letter)
Perceived commitment to specialty
Grades in required clerkships
Audition elective/rotation within your department
Any failed attempt in USMLE
Class ranking/quartile
Personal prior knowledge of the applicant
Leadership qualities
Grades in clerkship in desired specialty
Evidence of professionalism and ethics
Perceived interest in program
Passing USMLE Step 2 CS
Other life experience
COMLEX‐USA Level 1 score*
Alpha Omega Alpha (AOA) membership
Any failed attempt in COMLEX‐USA*
Volunteer/extracurricular experiences
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*
Consistency of grades
Awards or special honors in clinical clerkships
Graduate of highly‐regarded medical (MD/DO) school
Evidence of continuous medical education without gaps
Gold Humanism Honor Society (GHHS) membership
Awards or special honors in clerkship in desired specialty
Passing COMLEX‐USA Level 2 Performance Evaluation*
Demonstrated involvement and interest in research
Applicant was flagged with Match violation by the NRMP
Visa status**
Away rotation in your specialty at another institution
Fluency in language spoken by your patient population
Interest in academic career
Awards or special honors in basic sciences
USMLE Step 3 score
COMLEX‐USA Level 3 score*
NRMP Program Director Survey Results, 2020 164
Vascular SurgeryPercentage of Programs Citing Each Factor And Mean Importance Rating¹ for Each Factor in Ranking Applicants(N=6)
Figure VS-2
Average RatingPercent Citing Factor
¹ Ratings on a scale from 1 (not at all important) to 5 (very important).* DO seniors/graduates only.** International Medical Graduates only.
1 2 3 4 5
4.75.0
4.84.84.7
4.54.2
5.05.0
4.84.5
4.04.44.6
4.04.0
4.74.0
4.55
4.54.0
4.54.04.0
4.54.5
4.04.5
4.34.0
5.04.5
5.04.0
5.04.04.04.0
100% 50% 0%
100%100%100%
67%100%
67%83%83%
50%100%
33%33%
83%83%
50%50%50%
17%33%33%
0%33%33%33%
17%17%
33%33%33%33%
0%50%
33%17%
33%17%
33%33%
17%17%17%
0%0%0%
Interpersonal skillsInteractions with faculty during interview and visit
Interactions with housestaff during interview and visitFeedback from current residents
Letters of recommendation in the specialtyUSMLE Step 1 scoreUSMLE Step 2 score
Perceived interest in programEvidence of professionalism and ethics
Perceived commitment to specialtyLeadership qualities
Medical Student Performance Evaluation (MSPE/Dean's Letter)Audition elective/rotation within your department
Personal prior knowledge of the applicantPersonal Statement
Class ranking/quartilePassing USMLE Step 2 CS
Other life experienceAny failed attempt in USMLE
COMLEX‐USA Level 2 Clinical Evaluation (CE) score*Grades in required clerkshipsCOMLEX‐USA Level 1 score*
Grades in clerkship in desired specialtyAlpha Omega Alpha (AOA) membership
Any Failed COMLEX attempt*Volunteer/extracurricular experiences
Passing COMLEX‐USA Level 2 Performance Evaluation*Consistency of grades
Evidence of continuous medical education without gapsGraduate of highly regarded medical (MD/DO) schoolGold Humanism Honor Society (GHHS) membershipDemonstrated involvement and interest in research
Awards or special honors in clerkship in desired specialtyAwards or special honors in clinical clerkships
Applicant was flagged with Match violation by the NRMPInterest in academic career
Away rotation in your specialty at another institutionOther post‐interview contact
Fluency in language spoken by your patient populationVisa status**
Second interview/visitUSMLE Step 3 score
Awards or special honors in basic sciencesCOMLEX Level 3 score*
NRMP Program Director Survey Results, 2020 165
Vascular SurgeryProgram Usage of USMLE Step 1 and Step 2 Clinical Knowledge (CK) Scores When Considering Which Applicants to Interview
Figure VS-3
USMLE Step 2 (CK) Scores
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom Often
43% 43%
14%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Never Seldom often
43% 43%
14%
Does your program consider applicants who failtheir Licensure test on the first attempt?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
29%
57%
14%
Test Required?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Yes, pass only Yes, target score No
43% 43%
14%
Test Required?
N=7N=7
N=7N=7
Average USMLE Step 1 and Step 2 CK Scores Programs Consider When Granting Interviews
USMLE Step 1 Scores
The boxes in the boxplots above represent the interquartile range (IQR), which is the range between the 25th and 75th percentiles, and the line in the box is the median. The x-shaped symbol is the mean.
Scores Above Which Programs Almost Always Grant Interviews
Scores Below Which Programs Generally Do Not Grant Interviews
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
USMLE Step 2 CK not reported
because of low response rate
USMLE Step 1 not reported because of low response
rate
NRMP Program Director Survey Results, 2020 166
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
33% 33% 33%
COMLEX-USA Level 1*Test Required?
N=7
0%
20%
40%
60%
80%
100%
Yes No
71%
29%
USMLE Step 2 CSTest Required?
Vascular SurgeryProgram Usage of USMLE Step 2 Clinical Skill (CS), Step 3, and COMLEX-USA Scores* When Considering Which Applicants to Interview
Figure VS-4
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
50%
17%
33%
COMLEX-USA Level 2-CE*Test Required?
N=50%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
USMLE Step 3Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
0% 0%
100%
COMLEX-USA Level 3*Test Required?
0%
20%
40%
60%
80%
100%
Yes, pass only Yes, target score No
50%
0%
50%
COMLEX-USA Level 2-PE*Test Required?
N=7N=5
N=6 N=6
N=6 N=4
* DO seniors/graduates only
USMLE Step 2 CS and Step 3 Scores
COMLEX-USA Level 1 and Level 2-CE Scores
COMLEX-USA Level 2-PE and Level 3 Scores
NRMP Program Director Survey Results, 2020 167
Figure VS-5Vascular SurgeryProgram's Interview Activities
0
20
40
60
80
100
120
140
160
# positions inthe Match
# applicationsreceived
# interviewinvitations
sent
# applicantsinterviewed
# applicantsranked
1
151
4333 27
Average Number of Positions, Applications Received, Interview Invitations Sent,and Applicants Interviewed and Ranked
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Percentage of applicationsrejected based on a standardized
screening process
Percentage of applicationsreceiving an in-depth review
37%
65%
Average Percentage of Applications Rejected and Reviewed
N=7N=7N=10* N=7*N=7N=7
* Data from the NRMP database. All responding programs included
N=7
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0%
85%
15%
0% 0%
Percentage of Program's Interviews Extended During Each TimePeriod
0%10%20%30%40%50%60%70%80%90%
100%
Prior to Oct.1 2019
October2019
November2019
December2019
January2020 or later
0% 0%
24%
40%36%
Percentage of Program's Interviews Conducted During EachTime Period
N=7N=7
0% 20% 40% 60% 80% 100%
In the order in which applicants respondedto interview offers
Applicants submitted a rank order ofinterview date preferences by a specifieddeadline (e.g., within 72 hours), with no
preference given to the order of responses
Other
29%
43%
29%
How interviews were scheduled
0%
10%20%
30%
40%50%
60%70%
80%
90%100%
Less than1 hour
1 -3 hours 3 - 12hours
12 - 24hours
24 - 48hours
more than48 hours
0% 0% 0% 0%
14%
86%
Time given applicants to respond to interview invitations beforeextending invitations to other applicants
43%Yes
57%No
Program offered more interview invitations toapplicants than interview slots to fill
N=7
N=7N=7
Reasons for "offering more interview invitations to applicants
than interview slots to fill" not reported because of low response
rate.
NRMP Program Director Survey Results, 2020 168
0 1 2 3 4 5
Professionalism
Quality of patient care
Clinical competency
Ethics
Communication skills
Passing board certification examination
Personality
Academic performance during residency
Ability to teach medical students
Performance on in-training examination
Patient Satisfaction Scores
Research and publications
5.0
5.0
5.0
5.0
4.8
4.8
4.3
4.5
3.7
4.0
4.2
3.8
Importance of Factors in Determining Residents' Success in Residency ProgramAverage rating on a scale of 5 (5=very important)
Figure VS-6Vascular SurgeryProgram's Interview Activities (Continued) and Program Assessment of Residents' Success
n=5
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
29% 29% 29%
14%
No communication other thanreceipt of the ERAS
application
No communication and instead will initiate contact with references and/or the medical school if needed
To be contacted bythe applicant’s
medical school afterprogram has initiated
contact with theapplicant
Other
Preferred Type of Communication About Applicants During SOAP
N=7
Frequency of Programs Interviewing and Ranking Candidates in the Main Residency Match
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MD SeniorMD Grad
DO SeniorDO Grad
US IMGNon-US IMG
PathwayCanadian
14% 14%
57%
29%
71%
14%
29%43%
14%
71%
57%
29%
71%100%
57%43%
29% 29%14% 14%
Often Seldom Never
Interview
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
MDSenior
MD Grad DOSenior
DO Grad US IMG Non-USIMG
Pathway Canadian
14% 14%
57%
14%
43%
71%
29%
29%43%
14%
43%
29%
14%
43%100%
57%43%
29%43%
29%14%
29%
Often Seldom Never
Rankingn=7 n=7
NRMP Program Director Survey Results, 2020 169