Digital Mental Health Apps Need More Regulatory Oversight

iStock/Thanmano

Eight years ago, researchers at the University of New South Wales in Australia published a review of dozens of apps available for people with bipolar disorder in Google Play and iOS app stores. Their analysis found that by and large the information in these apps varied from incomplete to “critically wrong,” with one app going so far as to encourage the use of liquor before bed to assist with sleep during a manic episode.

“There are lots and lots of mental health apps out there, and it is concerning that many offer bad, or even dangerous, advice,” said David Mohr, Ph.D., director of the Center for Behavioral Intervention Technologies at Northwestern University.

Mohr has long been an advocate for greater integration of digital tools in psychiatry but cautioned that the results of the review article show that there are risks to using apps, video games, or other online products—just as there are risks to medication and medical procedures.

What Are the Risks?

Even thoughtfully designed, evidence-based apps are not without risks, Mohr continued. “In such cases, the risk is typically not derived from the product itself, but how it was implemented,” he said. For example, some clinics offer patients mobile self-care apps while they are on waitlists for psychotherapy.

“The clinicians may think they are doing patients a favor by offering this ‘bridge care,’ but many patients see it as a doctor fobbing them off with a second-rate therapy.” Mohr said such perceptions may lead to general annoyance or aggravation with the health care process and potentially cause problems down the road.

A study published in February 2022 by researchers at Kaiser Permanente found that using apps to supplement regular care can also be potentially harmful.

The study tested the effects of two low-intensity add-ons to usual care for individuals who frequently reported suicidal ideation: a text-based care management intervention and an online skills training program. The care management intervention involved regular outreach to patients about their suicidality and referred the patients for outpatient treatment as needed. The online program offered patients coaching in dialectical behavior therapy (DBT) skills, including mindfulness and paced breathing.

The researchers found that people who received the care management intervention had similar rates of self-harm as those who received usual care alone. Patients assigned to the online skills training experienced higher rates of self-harm compared with those assigned to usual care alone.

“Both intervention groups showed the same pattern: lowest risk in those actively refusing intervention services, intermediate risk in those either not responding to the intervention invitation or remaining engaged in the intervention for more than three months, and highest risk in those engaging in the intervention for less than three months,” the Kaiser researchers wrote.

They noted that such a finding suggests that “some patients may have been dissatisfied with these limited interventions and become more discouraged.”

John Torous, M.D., M.B.I., director of the Digital Psychiatry Division at Beth Israel Deaconess Medical Center, noted, “Context is huge when it comes to deploying apps. That’s one reason why adhering to digital therapeutics is even harder than medications.”

FDA Still Figuring Out How to Regulate Medical Software

To begin to understand the risks and benefits of mental health apps, a national postmarket surveillance program—akin to the FDA Adverse Event Reporting System (FAERS)—is needed, Torous and Mohr agreed. Through FAERS, health care professionals, patients, and manufacturers can submit reports about adverse events, medication errors, and product quality complaints used for drugs or other biological agents.

Creating a system of digital product oversight will be a tall order, Mohr acknowledged. In addition to the sheer number of mental health–related apps on the market (more than 20,000 by some estimates), software is rapidly evolving such that usability or risk data from an app may be irrelevant in a few months. Additionally, “as software developers continue to make their products more customizable, individual users will have vastly different experiences with the same app,” he said.

“The FDA knows it needs to be involved in digital health oversight,” Torous told Psychiatric News. “They just haven’t figured out how to do it yet.”

In 2017 the FDA launched the Software Precertification (“Pre-Cert”) Pilot Program to explore new approaches to regulatory oversight of medical device software. Under Pre-Cert, the FDA examined the possibility of “approving” trusted digital companies rather than individual programs, thus enabling the companies to rapidly develop and update digital therapeutics.

In September 2022, the FDA ended their Pre-Cert pilot program and reported that the model was not practical under current regulatory guidelines.

Mohr suggested that the FDA’s Sentinel product safety system may provide a way forward for oversight of digital mental health products. This program is like FAERS but rather than having patients or clinicians manually submit reports of adverse medical events, Sentinel automatically compiles safety information on drugs and medical devices from linked health insurance and electronic health record data. Mohr thinks Sentinel (or a related system) could be adapted to monitor app-related safety.

As he and colleagues wrote this June in a commentary published in Psychiatric Services, available electronic health records and insurance data could be combined with usage data from the app itself, providing a comprehensive picture of how much an app is being used and the associated risks and benefits.

Other Issues

“I think the technical details in setting up a digital health database are not that daunting,” Mohr said. “The key issue is developing standardized data related to engagement and outcomes. Some apps expect you to log in every day, while others may require once a week or only when you’re in distress.” Mohr said that health centers, insurers, and the FDA would need some way to reliably compare disparate products.

A centralized surveillance system is valuable, but it needs to be backed with compliance measures that have some teeth, Torous added. Voluntary reporting or suggested safety guidelines aren’t going to change the current practices of commercial app developers, who routinely short shrift considerations such as user privacy, he said.

Torous said that he was encouraged by recent enforcement action taken by the Federal Trade Commission against the mental health app company Better Health for sharing private data. He believes such action suggests that the government is taking digital mental health enforcement seriously.

“Don’t expect any big changes overnight; this will take time,” he said.

In the meantime, psychiatrists can certainly practice some small-scale postmarket surveillance with patients by discussing their app use and if they’ve had any issues, even banal symptoms such as eye strain and/or headaches.

“It’s common to ask patients about dietary supplements they might be taking; think of apps and games as digital supplements,” Torous said.

He suggested reviewing the APA App Evaluation Framework or evaluating apps with the mindapps.org database as practical steps both psychiatrists and patients can take today to make more informed decisions. ■