No Sugar Coating Allowed In Board Feedback Program
Brutal honesty and candid assessment—that's the best advice for training directors and faculty providing feedback to psychiatrists preparing for the oral board exam of the American Board of Psychiatry and Neurology (ABPN).
And it's especially important for psychiatrists who may have already failed the exam multiple times, said Nyapati Rao, M.D., past chair of the APA Council on Medical Education and a professor and director of residency training at SUNY Downstate Medical Center in Brooklyn, N.Y.
At the recent meeting of the American Association of Directors of Psychiatric Residency Training (AADPRT) in San Diego, Rao offered an overview of lessons learned from APA's Diagnostic Evaluation Project to Assist ABPN Recidivists. This project was designed to help candidates who have failed the oral exam at least three times.
Through the project candidates have the opportunity to take an oral test similar to that offered in the board certification exam—including a live patient interview and oral presentation—in front of“ examiners” who have served as examiners for the real thing.
Data from the ABPN have indicated that the failure rate on the Part II oral exam rate is high—as much as 50 percent. And it appears that candidates who fail the exam more than once are at high risk of failing it multiple times (Psychiatric News, September 2, 2005).
“My interviewers for the live patient were brutal, which is where I needed the `hold no punches feedback.' ”
APA's project has held two diagnostic sessions with candidates, one in 2004 and 2005. Rao told Psychiatric News that of the 24 candidates who have taken the diagnostic exam, 15 have gone on to pass the ABPN examination.
Candidates taking the APA diagnostic program are told whether they pass or fail and—most importantly—if they fail, they are told why. The APA program does not include remediation strategies, but candidates are encouraged to use the information that examiners give them to prepare for the real certification examination.
The advice that candidates taking the APA test are given is unvarnished.
“Candidates who fail the board exam tell us that they often go to a colleague and ask for a critique of their performance on a practice exam,” Rao told Psychiatric News. “The people they seek out don't want to criticize a former or current colleague, and unless they have been board examiners themselves, they may have only limited experience with the actual conditions of the exam. So they will sit in on a patient interview and give the candidate feedback that in the end is not very helpful.”
In some cases, candidates have reported taking commercially offered mock board exams, sometimes costing thousands of dollars, after which the candidates are typically told that they will have no problem passing the exam, Rao said.
Not so with APA's practice exam. Faculty examiners for the APA diagnostic exam are actual ABPN board examiners, and they pull no punches in their assessment of candidates' performance. Here is testimony from candidates who took the diagnostic exam and later passed the ABPN exam:
“Despite doing mock boards with people I knew, I think no matter how poor you may perform, people who know you are not going to tell you what they really think, and they want to reassure you and decrease your anxiety. Also, people can reconcile how good a day-to-day clinician you may be, and they find it difficult to break the news that you are not playing the role of ABPN psychiatry candidate well. My interviewers for the live patient were brutal, which is where I needed the `hold no punches feedback.'”
“The examiners were painfully blunt and direct with their feedback to me.... Despite how hurt I was by the feedback, I left that evening finally knowing what I had been doing that was getting in the way of passing the exam.”
In a survey of 36 candidates who took the diagnostic exam, APA has sought to derive correlations between performance on the live patient interview and video portion of the diagnostic, and background characteristics of the candidates.
At the AADPRT meeting in San Diego, Rao told training directors that the strongest predictor of poor performance is the length of time since graduation from residency. Somewhat less predictably, the length of time spent in an outpatient rotation also correlates with poor performance—specifically, the more time spent in outpatient rotation, the more likely it is that the candidate will perform poorly on the exam. Rao explained that the correlation appears to reflect the less-intensive supervision that typically occurs in outpatient rotations.
“In the inpatient setting, supervisors are liable to be sitting with the trainee and providing close supervision, and the resident is part of a group support system,” he speculated. “But in the outpatient setting, the trainee only once a week presents a case to a supervisor who may have little or nothing to do with the clinic.”
No diagnostic session is scheduled for 2006, according to Deborah Hales, M.D., director of APA's Division of Education and Career Development.
“We will be taking a hiatus in 2006, since the Part II exam is changing this May, and we want this program to reflect the actual exam as closely as possible. We will review the new exam procedures and incorporate them into our program for 2007.” ▪
