To Increase Diversity, Medicine Should Rethink USMLE Scores, Psychiatrists Say
Abstract
To deliberately recruit a diverse medical workforce, ending the overreliance on USMLE scores in residency programs is an important first step.
Physicians must make decisions with life-and-death consequences every day. With such high stakes, some psychiatrists are asking: Why are the courses of physicians’ careers often decided by the results of standardized tests?
Medical students’ scores on the three-step U.S. Medical Licensing Examination (USMLE) are often viewed as an important measure of their abilities as future physicians. Yet a growing number of voices are pointing to the tests as impediments to diversity and inclusion within medicine, rather than fair representations of whether medical students will make good physicians.
“We have to think about how we can change our existing practices, because we know we have the technology, and we know we have the innovation,” says Ayana Jordan, M.D., Ph.D.
During an APA town hall in June titled “Member Town Hall–APA Addresses Structural Racism, Part One,” Ayana Jordan, M.D., Ph.D., pointed to one way that the field of medicine can deliberately recruit a more diverse workforce that is reflective of the population: End the overreliance on USMLE scores. Jordan is an assistant professor of psychiatry and director of the Social Justice and Health Equity Curriculum at Yale University School of Medicine.
According to a 2016 survey of residency program directors by the Association of American Medical Colleges, 75% of the 1,453 respondents reported using filters or minimum thresholds when selecting applicants for interviews, including the USMLE Step 1 score.
“Having a diverse workforce means that you have physicians who speak the same language as the patient or who have the same racial and ethnic background as the patient, and we know the outcomes are better for the patient,” said Michelle Durham, M.D., M.P.H., an assistant professor of psychiatry and psychiatry residency training director at Boston University School of Medicine. “That’s critical, and we shouldn’t have to make any further arguments. If people are going to be healthier and do better, that should be reason enough to take steps to diversify the workforce.”
Jordan, Danielle Hairston, M.D., and Durham will soon publish an article about this issue in which they outline how the USMLE scores perpetuate a lack of diversity in medicine and provide recommendations to create equity in the residency selection process. Hairston is the president of the APA Caucus of Black Psychiatrists and the psychiatry residency training director at Howard University College of Medicine.
“We have to look at more than standardized test scores that we know don’t necessarily correlate with clinical performance,” says Danielle Hairston, M.D. “Medical schools are at the risk of becoming some sort of USMLE Step assembly lines.”
“Despite spending so much money on health care, the U.S. continues to fall short in providing care for everyone,” Jordan said. “We have to think about how we can change our existing practices, because we know we have the technology, and we know we have the innovation. What we continue to fall short on is making sure we are engaging a diverse workforce to be able to bring access to a diverse patient population.”
Inequity Within Standardized Tests
Putting so much emphasis on standardized tests gives an advantage to people of more privileged backgrounds who can afford testing materials and tutoring programs, Durham argued.
“We already know that the ZIP code where you come from can determine your future from birth,” she said. “When families are living at or below the federal poverty level, they don’t have access to the same educational resources as a family with more money. Unfortunately, this typically affects more people of color than the white population.”
“Medicine is so much more than an exam score,” Hairston said in an email. “Where is the value for students who are dedicated to underserved populations, changing policies, collaborative care, and listening? If residency programs continue to value and screen students based on just this exam metric, things will remain the same, dictated by exams with proven bias.”
“I have no doubt that there is inequity in access to test preparation resources among medical students due to family income levels, parental education levels, etc.,” said Howard Liu, M.D., M.B.A., chair of the University of Nebraska Medical Center Department of Psychiatry, president of the Association of Directors of Medical Student Education in Psychiatry, and a member of APA’s Council on Communications. “For a first-generation student, the cost of rent can be stressful, let alone paying for high-priced test preparation materials and question banks. In addition, underrepresented students have faced years of systemic racism in education by the time they matriculate into medical school, which is another risk factor for lower performance on standardized exams.”
The American Association of Directors of Psychiatric Residency Training (AADPRT) Assessment Committee shares the concern that the USMLE’s approach to assessing readiness for state licensure has several limitations, said the committee’s chair, John Q. Young, M.D., M.P.P., Ph.D., in a statement. He added that the exams do not address competencies such as teamwork, collaboration, and integrity, which are “critical to the safe and effective practices of medicine.”
“Regarding diversity—the evidence is clear,” Young continued in the statement. “Health outcomes are better when the physician workforce mirrors the patient population that it serves. This is true in psychiatry. Increasing diversity in psychiatry should be a public health imperative.”
In addition to Young’s comments, AADPRT President Melissa Arbuckle, M.D., Ph.D., noted in an email that recruiting underrepresented students to psychiatry requires “a sustained commitment to anti-racism, to understanding and addressing disparities, and to supporting and nurturing underrepresented students.”
Step 1 Test Becomes Pass/Fail
The Federation of State Medical Boards and the National Board of Medical Examiners decided to make Step 1 of the USMLE a pass/fail score starting no earlier than January 1, 2022.
The change, Jordan and Durham said, is a step in the right direction. But they worry that emphasis will simply be shifted to the Step 2 test score, thus not solving the problem but moving it down the line.
Moving to a pass/fail Step 1 score creates an opportunity to do what Adrienne Adams, M.D., M.S., refers to as holistic interviewing. Adams is the chair of AADPRT’s diversity and inclusion committee, an associate professor of psychiatry at Rush Medical College, and the director of the child and adolescent fellowship program at Rush University.
“I think it’s important to look at the whole person, not just a three-digit score,” she said. “How have they done with classes? What are some of the community activities or volunteer work they’ve done outside of school? Do they have research experience? What shows their resilience—did they have to work full time to support themselves in addition to attending school?”
“If we’re truly concerned about addressing health care disparities, it is imperative that we adequately train a diverse set of physicians that can meet our patients’ complex medical as well as social needs,” Jordan said. ■
