A system for assessing residents' competency in psychotherapy that includes direct supervision of psychotherapy sessions by faculty appears to be promising, according to psychiatry faculty at the University of Wisconsin School of Medicine and Public Health.

At a poster session at last month's meeting of the American Association of Directors of Psychiatric Residency Training in San Juan, Puerto Rico, psychiatry training director Art Walaszek, M.D., presented preliminary results of a comprehensive system for assessing psychotherapy competence.

That system includes direct faculty observation of resident performance, measurement of patient outcomes using rating scales and satisfaction surveys, and a "capstone" experience that tests the ability of residents to develop and present a comprehensive psychotherapy formulation and outcome.

In an interview with Psychiatric News, Walaszek said faculty observation involves a "drop-in" visit by supervising faculty typically in the middle of a 50-minute session and usually for about 10 minutes.

"The resident can use that as a time to consult with the faculty supervisor right there in the session, with the patient included as a partner," Walaszek said. "The supervisor can directly observe and provide feedback on the resident's psychotherapy skills."

He said the practice of drop-in supervision has been in place at the University of Wisconsin for at least five years— though the formal assessment system was not instituted until 2005—and that he believes patients, residents, and faculty are now acculturated to the idea. "We train people early on that this is part of what we do here," he said.

Walaszek said that a survey of patients and residents conducted by colleague Pauline Thome, Ph.D., indicated that the practice of drop-in supervision is not only tolerated but also appreciated.

"Patients were the most open to it and the least likely to see it as detrimental" among those surveyed, he told Psychiatric News." My speculation is that patients appreciate the fact that there is a team involved in their care and that the team is available."

Walaszek acknowledged the practice is controversial in some quarters, but said he believes it may come to be more widely practiced as training programs struggle with how to assess psychotherapy competence.

After a period of orientation of faculty and residents, the Psychotherapy Competency Assessment Plan was instituted in July 2005 for PGY-2 residents. The second iteration began in July 2006 for PGY-2 and PGY-3 residents.

As part of building a portfolio of psychotherapy cases, residents were asked to administer rating scales to patients at the start and end of therapy and at several interim points. The rating scales include at least one measure of symptoms (depending on each patient's diagnosis) and a global measure (typically the Global Assessment of Functioning). At the end of therapy, patients complete a patient satisfaction survey, developed internally.

Finally, residents are required to develop and present a comprehensive" capstone" formulation of one psychotherapy case and a description of the course of therapy. A faculty member evaluates each presentation using a locally designed assessment tool.

Outcome data from the patient rating scales were not available at the time of the poster session presentation, but Walaszek said the quality of first-year "capstone" case formulations surpassed faculty expectations and that residents seemed to find the sessions enjoyable and informative as well.

"We are seeing the residents more rapidly getting an understanding of what we expect from them, and that in turn is resulting in more rapid mastery and confidence on the part of residents," Walaszek said. "I am looking forward to expanding this further and being able to say with some reliability that residents have seen a certain number of patients and can perform the necessary skills.

"Residents are also using patient outcome measures in a way that puts us in line with what the rest of medicine is doing and with what is going to be expected from psychiatrists in the future," he continued. "I think it helps us know that we are delivering services that help people in a measurable way." ▪