USMLE Step 1 Pass/Fail Scoring: A Paradigm Shift for Neurosurgical Resident Selection

0
7952

In early 2020, it was announced that USMLE Step 1 would move from a three-digit numerical score to a pass/fail score (P/F) as early as January 2022.1 This fundamental change in an important educational milestone for medical students has broad potential implications for future neurosurgery resident selection.

The Step 1 numerical score is currently a critical component of the neurosurgery residency interview selection process.2 The largest reason cited by the USMLE for changing the Step 1 score reporting policy is to reduce the overemphasis of USMLE Step 1 performance in resident selection, which was described as an unintended consequence of secondary score use.1 Step 1 was originally designed to evaluate basic science knowledge for medical licensure eligibility,1,3 not to be a predictor of success in residency in the way that the MCAT functions as a predictor of first-year medical student success.4

o

While a standardized evaluation tool such as USMLE Step 1 is helpful in resident selection,  there is limited data on the predictive nature of Step 1 scores with success as a neurosurgery resident. High Step 1 scores were correlated with higher neurosurgery board performance in a single institutional study; however, STEP 1 score did not predict ABNS certification status across a larger analysis.5,6 Additionally, a review of pre-residency factors across other surgical subspecialties found scores to be poorly correlated with resident performance evaluations.7

The Downside of Numerical Scores

In recent years, students have increasingly focused efforts on maximizing USMLE numeric score performance at the expense of engagement in medical school preclinical curriculum, participation in class, interaction with faculty, and involvement in research, volunteering, and social opportunities.

A parallel curriculum consisting of board review material has emerged separate from medical school classes. The proliferation of online medical review companies such as Pathoma, Sketchy Medical, Firecracker, Kaplan, USMLERx, Amboss, and Osmosis has led to an anxiety-inducing number of avenues to engage in this parallel curriculum at significant additional cost. With P/F preclinical grades, some students forgo their school-specific curriculum entirely. This has led to negative effects on student well-being, some struggling to manage learning across two different incentive structures.8

The Risks of Pass/Fail

While the USMLE score may have limitations, it has represented the only common objective metric by which to evaluate all applicants across medical schools. Strong performance on Step 1 may be evidence of the work ethic and intelligence required to succeed as a neurosurgical resident and may have allowed students from less well-known medical schools to match into top training programs. The Step 1 score has served as an important variable in a competitive field where applicants outnumber available positions.

How Will Students Stand Out in the Future?

While removing the numerical score aims to reduce pressure on students, it is likely that the emphasis will simply be shifted to other metrics. The field of neurosurgery has always valued research, with applicants to the field having the highest number of publications, abstracts, and presentations compared to applicants in any other specialty.9 Some authors have described the temporal trend toward increasingly robust research and publication expectations for entry into the field as a research “arms race”.10 Of all 2018 NRMP match specialties, neurosurgery had the second highest percentage of US MD graduates from a top 40 NIH funded institution at 44%.9 A further emphasis on research may benefit students coming from institutions with robust research environments. Nonetheless, this change may incentivize more students to gain exposure to research, think critically as scientists, and propel future advances in the field.

Read Dr. Mulligan’s Editorial Response in So What Now?

Although losing an important metric will present a challenge to residency program directors, it will also present a new opportunity to broadly consider what makes a great resident. To be a successful neurosurgeon,  the ability to assimilate didactic knowledge and test well is important, but there are also many important qualities beyond test-taking such as work ethic, resiliency, efficiency, honesty, collaboration, and empathy. The de-emphasis on Step 1 will allow medical students to explore other opportunities to be competitive for residency making them more well-rounded applicant and potentially increasing diversity within the field.11 As programs adjust evaluation metrics to accommodate this paradigm shift in score reporting, we are optimistic neurosurgery will continue to attract a uniquely talented group of residents who, like generations before them, will train to provide the highest quality care for patients and rise to become leaders in the field.

References

[expand title=”View All”]

1. https://www.usmle.org/incus/#decision

2. https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.pdf

3. Prober CG, Kolars JC, First LR, Melnick DE. A Plea to Reassess the Role of United States Medical Licensing Examination Step 1 Scores in Residency Selection. Academic Medicine. 2016;91(1). https://journals.lww.com/academicmedicine/Fulltext/2016/01000/A_Plea_to_Reassess_the_Role_of_United_States.11.aspx.

4. Busche K, Elks ML, Hanson JT, et al. The Validity of Scores From the New MCAT Exam in Predicting Student Performance: Results From a Multisite Study. Academic Medicine. 2020;95(3). https://journals.lww.com/academicmedicine/Fulltext/2020/03000/The_Validity_of_Scores_From_the_New_MCAT_Exam_in.29.aspx.

5. Nagasawa DT, Beckett JS, Lagman C, et al. United States Medical Licensing Examination Step 1 Scores Directly Correlate with American Board of Neurological Surgery Scores: A Single-Institution Experience. World Neurosurg. 2017;98:427-431.

6. Gelinne A, Zuckerman S, Benzil D, Grady S, Callas P, Durham S. United States Medical Licensing Exam Step I Score as a Predictor of Neurosurgical Career Beyond Residency. Neurosurgery. 2019;84(5):1028-1034.

7. Zuckerman SL, Kelly PD, Dewan MC, et al. Predicting Resident Performance from Preresidency Factors: A Systematic Review and Applicability to Neurosurgical Training. World Neurosurg. 2018;110:475-484.e10.

8. Moynahan KF. The Current Use of United States Medical Licensing Examination Step 1 Scores: Holistic Admissions and Student Well-Being Are in the Balance. Academic Medicine. 2018;93(7). https://journals.lww.com/academicmedicine/Fulltext/2018/07000/The_Current_Use_of_United_States_Medical_Licensing.9.aspx.

9. https://www.nrmp.org/wp-content/uploads/2018/06/Charting-Outcomes-in-the-Match-2018-Seniors.pdf

10. Wadhwa H, Shah SS, Shan J, et al. The neurosurgery applicant’s “arms race”: analysis of medical student publication in the Neurosurgery Residency Match. J Neurosurg. 2019;:1-9.

11. Rubright JD, Jodoin M, Barone MA. Examining Demographics, Prior Academic Performance, and United States Medical Licensing Examination Scores. Academic Medicine. 2019;94(3). https://journals.lww.com/academicmedicine/Fulltext/2019/03000/Examining_Demographics,_Prior_Academic.28.aspx.[/expand]

Print Friendly, PDF & Email
o