Smart Glasses May Help Children With Autism Recognize Emotions
Abstract
Glasses are connected to an app with facial recognition technology that uses emojis to depict the emotion of the person with whom the child is interacting.
As the number of children diagnosed with autism spectrum disorder (ASD) continues to rise, the bottleneck for getting these children the behavioral therapy that teaches them important social and communication skills is only getting worse.
A team at Stanford Medical School might have developed one potential solution to more quickly connect children with ASD to social-skills training—a wearable technology platform that combines facial recognition software with smart glasses to help the children identify emotions and facial expressions.
Children with ASD who used the smart glasses while receiving conventional behavioral therapy experienced greater gains in social skills compared with those who received only behavioral therapy, according to a report published March 25 in JAMA Pediatrics.
“My lab has been interested in digital solutions for autism for years. When Google Glass hit the market, it seemed like a perfect tool to address common struggles facing these children, such as making eye contact and recognizing faces,” said Dennis Wall, Ph.D., an associate professor of pediatrics, psychiatry, and biomedical data sciences and the lead investigator for this study.
Helping children with ASD recognize emotions is a central part of behavioral therapy, typically using flash cards of various facial expressions. As Wall told Psychiatric News, static images capture only a fragment of the ways in which someone can smile or frown and sometimes lead to unwanted associations (such as a child associating facial hair with a certain emotion if a bearded person is the model on a card). Smart glasses, such as Google Glass, can capture emotions in real time to better guide children about what they are seeing, he said. Best of all, this training can be done outside of office visits.
Early tests of the technology proved problematic as the glasses used by Wall’s team had limited processing power and overheated after longer periods of use. Wall’s lab found a solution by moving most of the functionality of the glasses to a smartphone-based app. The glasses are modified with no lenses, just a small prism next to the camera that can display emojis within the child’s peripheral vision. The emojis depict one of eight emotions—happy, sad, angry, scared, surprised, disgust, “meh,” and neutral—recognized by the facial recognition software.
To make the glasses more engaging, Wall’s team added two social games into the app: a “guess the emotion” game in which children try to correctly identify their caregiver’s facial expression and a “capture the smile” activity in which children are given an emotion and try to elicit that facial expression in their caregiver (like telling a joke to make a happy face).
After some promising pilot studies, Wall and his team conducted a trial with 71 children with ASD aged 6 to 12 years; 41 of the children were given the wearable intervention (dubbed Superpower Glass by a focus group of children), and 30 were in the control group. Both groups received in-person behavioral therapy at home for six weeks. The children using Superpower Glass were instructed to use the device at home for 20 minutes, four times a week: three times with family members and once with their behavioral therapist.
After six weeks, the children using Superpower Glass showed significant improvements in their social behavior (a 4.5-point higher score on the ASD assessment known as the Vineland Adaptive Behaviors Scale) compared with children in the control group.
Wall said he also received positive feedback from parents on the progress their children had made while using Superpower Glass.
“This justifies further work with these devices in larger and younger groups of children,” Wall said. He added that, ideally, future studies would involve children who are waiting to begin behavioral therapy to see how well they improve with the glasses alone.
If the children can use these glasses while waiting for therapy, they might be better prepared for therapy and perhaps need therapy for a shorter time. This would reduce costs for parents and free up time for therapists to see more children, which would start to alleviate the wait-list bottleneck.
Wall also hopes these positive findings will catalyze interest and energy in the use of augme\nted-reality tools, such as smart glasses for ASD. Despite how well such tools complement the deficits seen in children with ASD, augmented reality remains a niche field in autism research, he said.
Wall’s study was supported by the National Institutes of Health, the Lucile Packard Foundation for Children’s Health, the Hartwell Foundation, the Wallace H. Coulter Foundation, Stanford University, the Walter V. and Idun Berry Postdoctoral Fellowship Program, the Bill and Melinda Gates Foundation, Peter Sullivan, and Bobby Dekeyser. The glasses were provided by Google Inc.
Wall is the founder of Cognoa Inc., a company specializing in digital diagnostics and personalized therapies. ■
“Effect of Wearable Digital Intervention for Improving Socialization in Children With Autism Spectrum Disorder: A Randomized Clinical Trial” can be accessed here.
