Esaïe Prickett wearing Google Glass. He and his family tested the device in a clinical trial. Photo / Cayce Clifford, The New York Times
Privacy concerns caused the computerised eyewear to fail with the general public. But researchers believe it could help autistic children learn to recognise emotion and make eye contact.
When Esaïe Prickett sat down in the living room with his mother, father and four older brothers, he was the only one wearing Google Glass.
As Esaïe, who was 10 at the time and is 12 now, gazed through the computerised glasses, his family made faces — happy, sad, surprised, angry, bored — and he tried to identify each emotion. In an instant, the glasses told him whether he was right or wrong, flashing tiny digital icons that only he could see.
Esaïe was 6 when he and his family learned he had autism. The technology he was using while sitting in the living room was meant to help him learn how to recognise emotions and make eye contact with those around him. The glasses would verify his choices only if he looked directly at a face.
He and his family tested the technology for several weeks as part of a clinical trial run by researchers at Stanford University in and around the San Francisco Bay Area. Recently detailed in The Journal of the American Medical Association, Pediatrics, the trial fits into a growing effort to build new technologies for children on the autism spectrum, including interactive robots and computerised eyewear.
The Stanford study's results show that the methods have promise and indicate that they could help children like Esaïe understand emotions and engage in more direct ways with those around them. They could also measure changes in behaviour, something that has historically been difficult to do.
Experts believe that other new technologies may help in similar ways. Talking digital assistants like Amazon's Alexa, for example, could help children who misuse their pronouns. But even as these ideas spread, researchers warn that they will require rigorous testing before their effects are completely understood.
Catalin Voss started building software for Google Glass in 2013, not long after Google unveiled the computerised eyewear amid much hullabaloo from the national media. An 18-year-old Stanford freshman at the time, Voss began building an application that could automatically recognise images. Then he thought of his cousin, who had autism.
Growing up, Voss' cousin practised recognising facial expressions while looking into a bathroom mirror. Google Glass, Voss thought, might improve on this common exercise. Drawing on the latest advances in computer vision, his software could automatically read facial expressions and keep close track of when someone recognised an emotion and when they did not.
"I was trying to build software that could recognise faces," Voss said. "And I knew that there were people who struggled with that."
At the time, the brief moment Google Glass spent in the national spotlight was already coming to an end. Google stopped selling the device to consumers amid concerns that its built-in camera would compromise personal privacy.
But Google Glass lived on as something to be used by researchers and businesses, and Voss, now a doctoral student, spent the next several years developing his application with Dennis Wall, a Stanford professor who specialises in autism research, and others at the university.
Their clinical trial, conducted over two years with 71 children, is one of the first of its kind. It spanned everything from severe forms of autism, including children with speech impairments and tactile sensitivities, to much milder forms. Children who used the software in their homes showed a significant gain on the Vineland Adaptive Behavior Scales, a standard tool for tracking the behavior of those on the autism spectrum, Voss said.
The gain was in line with improvements by children who received therapy in dedicated clinics through more traditional methods. The hope is that Voss' application and similar methods can help more children in more places, without regular visits to clinics.
"It is a way for families to, on some level, provide their own therapy," Voss said.
Jeffrey Prickett, Esaïe's father, said he had been drawn to the study because he had known it would appeal to his son, who enjoys using iPad apps and watching movies.
"He does fine interacting with people," Prickett said. "But he does better interacting with technology."
Prickett found it hard to judge whether the Google device helped his son recognize emotions, but he saw a marked improvement in Esaïe's ability to make eye contact.
Heather Crowhurst, who lives near Sacramento, California, said she had experienced something similar with her 8-year-old son, Thomas, who also participated in the trial. But Thomas was not entirely captivated with the digital therapy. "It was kind of boring," he said.
The concern with such studies is that they rely on the observations of parents who are helping their children use the technology, said Catherine Lord, a clinical psychologist at the University of California, Los Angeles, who specialises in the diagnosis and treatment of autism. The parents are aware of the technological intervention, so their observations may not be reliable.
Still, the Stanford team considers its study a first step toward wider use of this and other technologies in autism. It has licensed the technology to Cognoa, a Silicon Valley startup founded by Wall. The company hopes to commercialise the method once it receives approval from the Food and Drug Administration, which oversees the use of medical devices in the United States. That may still be years away.
Other companies are taking a different approach. Brain Power, a startup in Massachusetts that has built similar software for Google Glass, is selling its technology to local schools. The company considers it a teaching tool, not a medical device.
Patrick Daly, assistant superintendent of the school district in North Reading, Massachusetts, is testing Brain Power's technology after watching its effect on his 9-year-old son, who is on the spectrum. The district intends to test the technology over the next few years.
Previously, the district tried to teach similar skills through iPad computer tablets. Daly sees Google Glass as a big improvement.
"It can actually maintain eye contact," he said. "They are not looking down while they try to learn an emotion."
Robokind, a startup in Dallas, applies the same philosophy to different hardware. The company spent the past several years designing a robot that attempts to teach many of the same skills as technologies built for digital eyewear. Called Milo, the doll-like, 2-foot-tall robot mimics basic emotions and tries to make eye contact with students. It also asks questions and tries to engage students in simple conversations.
Robokind has sold hundreds of the robots to schools for testing. Each one costs US$12,000, plus more than US$3,500 for additional software.
In some ways, such a device is a poor substitute for real human interaction. But the strength of this and other technologies is that they can repeat tasks time and again, without getting tired or bored or angry. They can also measure behavior in precise ways, said Pam Feliciano, scientific director of the nonprofit Simons Foundation Powering Autism Research.
For these reasons, Feliciano also sees promise in Amazon's Alexa. Her 14-year-old son is on the spectrum and struggles with his pronouns. He sometimes calls himself "you," not "I."
Her task is to correct him each time he makes a mistake. But she's human and gets tired. She does not always remember. A device like Alexa could help, she said, provided that researchers can show it is reliable and effective.
"The technologies are there," she said. "It is just a matter of the right technologists working with the right clinicians."
Written by: Cade Metz
Photographs by: Cayce Clifford
© 2019 THE NEW YORK TIMES
Economists expect the recovery continued during the first quarter of the year.