Social robots for autism education
By Dr Alyssa M. Alcorn
Developing a robot-assisted intervention programme for emotion teaching, powered by artificial intelligence
Autistic children frequently have difficulties in understanding and using social and emotional cues, and researchers agree that targeting these developmental “building blocks” can have broad, long-term benefits.
Existing research has shown promise in using robot-assisted interventions for teaching social and academic skills to autistic children, including emotion recognition. Most of this work has focused on older children in mainstream settings, and has not addressed young children or wider range of cognitive and language abilities. Given claims that robot-assisted interventions could present lower, less complex social demands than human-led interventions, it is particularly important to investigate their feasibility for children whose social, daily life, or language skills may present barriers to participation in “traditional” interventions.
As part of a European-wide consortium, researchers from the UCL Institute of Education (IOE) have contributed expertise in autism and psychology to designing this new programme and are leading its evaluation with autistic children and special schools.
The first stage of the DE-ENIGMA project (2016-2017) has tested the feasibility of an emotion-recognition training programme (based on “Teaching Children with Autism to Mind-Read”; Howlin, Baron-Cohen & Hadwin, 1999) with a large sample of autistic children in the UK and Serbia, as part of the larger goal of developing the potential of intelligent robot-assisted interventions.
66 school-aged autistic children from 3 London special schools have participated so far, with an additional 66 children participating in Belgrade. Half were randomly assigned to participate in an emotion recognition teaching programme led by an adult, and half assigned to the same programme assisted by Zeno the robot. The majority of our participants had intellectual disabilities and language issues in addition to autism.
The IOE’s DE-ENIGMA team is currently preparing the results of these initial studies, comparing the performance of children in the robot-led and adult-led conditions, as well as making cross-cultural comparisons between Serbia and the UK. Classroom observations and interviews conducted with educators in London special schools provide additional, qualitative context for this experimental work, and have helped to identify teaching and interaction strategies that can be adapted for Zeno the robot’s future activities.
Initial results at this stage suggest that many aspects of the robot-assisted programme were highly successful in both the UK and Serbia, particularly in term of fostering children’s interest and engagement in the emotion activities.
Zeno was frequently requested when we returned to schools to collect the observational data!
The results of this study will be an important contribution to the ongoing research discussion on whether robots can be useful tools for teaching social skills to children on the autism spectrum. The large and diverse sample will also facilitate analyses of which children may particularly benefit from work with robots, or with humans.
- Research stories
Here's a selection of interactions children have had with Zeno.
Focusing with Zeno: Young autistic children can “tell us” they are really interested in something when they sit, listen, and focus. When one teacher watched her student “Simon” work with Zeno, she told us how amazed she was that he focused for much longer than he’d ever done in class. Finishing 10 minutes of emotion practice sounds short to an adult, but can be a real achievement for many young children on the spectrum. If a child is willing to work with a robot for longer than they would work with a person, that is a really valuable opportunity for learning.
Who’s this? It’s Zeno! Zeno really captured “Timothy’s” imagination. After participating in a week of emotion-recognition teaching, his teaching assistant reported that Timothy began talking about emotions and faces in class. He also took his new information home: Timothy’s mother told us how her son talked about the robot and identified his picture in the school newsletter, telling her “It’s Zeno!”
Coping with sadness. During his participation in tasks with Zeno, “Marko” was fascinated with sadness and he kept insisting for Zeno to cry and show sad face. This raised some concerns, and we talked to his parents who told us that “Marko”s grandpa is terminally ill. Shortly after the completion of “Marko”s participation, we found out that his grandpa had passed away. According to his parents, “Marko” was drawing sad faces on paper and talking about emotions at home. This was his way of coping with the loss.
My buddy! “David” was fascinated with Zeno! He visited Zeno for 2 days, and over that period he was very successful with tasks Zeno gave him. On day 1, he tried to stuff his rubber grasshopper toy to Zeno's hand because he perceived Zeno as a peer, and wanted to share his toy with him. He actually managed to ‘give’ Zeno the toy and was fascinated with Zeno raising the toy up in the air! On day 2 he came to the test site, calling Zeno ''My buddy!'' He repeated that on the entire way from home to the test site, showing his excitement.
Note: Names have been changed to maintain confidentiality.
Results and insights from first round of DE-ENIGMA robot studies, teacher interviews, and classroom observations have encouraged us to re-shape the emotion teaching programme away from “one size fits all” direct instruction, and toward smaller, flexible, more game-like “modules”.
The IOE team is collaborating closely with the University of Twente on these new teaching activities, which will be enabled by multi-modal artificial intelligence developed by the DE-ENIGMA technical teams. Over several rounds of development, Zeno will be equipped to process audio, video, and gesture input in real time, and autonomously plan interactions with a child user.
Zeno will perceive more about what a child says and what expressions he or she shows, and will do more to offer personalised emotion teaching.
By 2019, DE-ENIGMA will have advanced, state-of-the art functionality for analysing speech, faces, and motion in real time—all under messy, real-world conditions. In the final version of the DE-ENIGMA system, Zeno the robot will lead a novel, intelligent emotion-teaching programme shaped by autism research and teacher input, which will be evaluated in a randomised control trial.
A parallel output for this project will be the DE-ENIGMA database, a multi-modal online database of audio, video, and gesture data from child participants, with many behaviours already labelled by autism experts. So far, it includes 121 British and Serbian children, representing 152 hours of interaction and ~13 terabytes of data. It will be the largest existing dataset of its kind—available free to academic researchers worldwide from 2018. It represents a rich resource for research questions about autism, including as child-robot or child-adult interactions, emotion recognition, social and communicative behaviours, and cross-cultural comparisons. It also provides the required scale of data needed for furthering machine learning, computer vision, audio processing and other technical techniques that aim to represent and adapt to autistic behaviours.