Research Stream: Social Technologies
Author: Emily McConway, Undergraduate Intern in Psychology, Maynooth University and Mac MacLachlan, Professor of Psychology & Social Inclusion, and Co-Director of the ALL Institute, Maynooth University
Early assessment and intervention are vital in facilitating positive developmental and behavioural outcomes for children with neurodevelopmental conditions. Early intervention has a positive long-term effect on both autistic children and their caregivers. The current process of assessing the needs of children with possible autistic traits focuses on the use of behavioural clinical diagnostic instruments such as Autism Diagnostic Observation Schedule (ADOS) and Autism Diagnostic Interview-Revised (ADI-R). Both instruments require direct clinician-to-child observation and can take hours to administer and score. In many countries, long waiting lists, coupled with social, economic and geographic barriers hinders timely assessment of neurodiverse children. The ALL Institute is interested in pragmatic ways to streamline access to services, including assessing a person’s needs for services and supports.
Creative and innovative solutions are necessary to ensure children and young people have access to the appropriate help at the times when it is most needed, especially during developmentally critical periods. Recently, we have argued that technology-enhanced digital approaches to assessment may shorten waiting lists, helping clinicians to identify needs and make diagnostic decisions, where these are called for. One of the helpful learnings from the Covid-19 pandemic for the National Clinical Programme for People with Disability (NCPPD, in Ireland) was that neurodiversity assessments could be undertaken remotely by clinicians through digital platforms.
Another innovative practice used during the Covid-19 pandemic was remote analysis of home videos, rather than observation of a child or young person in a clinic setting. Again, while much of the research has focused on diagnosis as opposed to the more useful determination of a person’s needs, this literature is still instructive. Dahiya et al’s (2020) systematic review outlined the potential for remote telehealth assessments in Autism diagnosis (that is “Autism Spectrum Disorder” according to DSM 5 and ICD 11 diagnostic criteria). Their review showed favourable results when both video and mobile applications where integrated into the assessment of neurodiversity.
Typically, remote assessments depend on the judgment of expert clinicians, but what about when there are insufficient numbers of these, or they are not available at all, in resource poor settings? In a more recent study conducted by Fathalla (2021) and associates at the Arab Academy for Science and Technology in Egypt, a machine learning approach was adopted for the remote determination of autism. These researchers asked “minimally trained individuals” – that is, non-clinicians who were blind (unaware) of the diagnosis given – to analyse and rate home videos. The raters were students or working professionals.
Each rater received a brief induction on the features they would be rating and were tasked with answering 30 questions on each video. In each question they were asked to indicate the presence and extent of a certain aspect of a child’s behaviour in the video. The features which the raters were tasked to evaluate, included expressive language, eye contact, emotion expression, communicative engagement, joint attention or pointing, developmental delay, social participation, sensory aversion, imitation of actions and sensory seeking. These behaviours (features) were selected from both the ADOS and ADI-R according to established criteria, and used to train a machine learning algorithms.
Feature selection and state-of-the-art classification methods allowed the researchers to train the algorithm to provide a diagnosis from home videos with a True Positive Rate (identifying true cases) of 94%. The success of this study hints at a scalable, rapid and possibly more accessible assessment of neurodiversity, at least from a diagnostic perspective. Diagnosis can, of course, be informative for a person’s understanding of their own experience, and their sense of identity, and in some countries to provision of services that are provided on a medical model of diagnostic access. But the potential of such an approach to reach beyond diagnosis, to identifying the supports and services that a child or young person requires, when they require it, is also evident.
One interesting feature of the previous study was that those who rated the presence of behavioural features on home videos were not clinically-trained experts, but rather, lay people. The lack of clinically-trained experts often constitutes a bottleneck to assessment and therefore access to the services and supports they need. So how can we select those non-experts who would be best at the task of identifying key behavioural traits?
Washington et al., (2021) used a process to identify and certify what they call a “trustworthy workforce” of raters for video feature extraction. Applicants were asked to evaluate a series of videos according to a set of feature identification criteria, and 102 out of 1107 applicants were then chosen, through a filtration process based on the accuracy of their ratings. The chosen applicants then rated previously unseen home videos and these ratings were used to power a machine learning algorithm. The result was a sensitivity level (correctly identifying true cases) of 96% and specificity level (correctly identifying non-cases) of 80%. Of course maintaining the privacy of these children is of paramount importance, given that unknown strangers are rating their home videos. These results are therefore even more impressive because these home videos were rated while maintaining the child’s privacy – faces were obscured with a red box, while voices were pitch shifted – preserving all of the original content of speech but obscuring potentially identifying vocal features.
One of the challenges of current applications of machine learning to people with neurodiverse conditions is that they treat such conditions as a binary – you have it or you don’t! – and not as a continuum, or spectrum, which is now well recognised in autism, and increasingly also in other neurodiverse conditions. So, an approach that identifies the extent of occurrence of different types of behaviours would be much more appropriate. It could also allow for the co-occurrence of many neurodiverse conditions, again by focusing on the difficulties rather than diagnostic labels to direct service needs. We must move away from a diagnostic “what have you got” assessment, to a much more useful “what do you need to help you” assessment, by focusing on the behaviours or difficulties that children and young people experience.
As well as being pragmatic with individual’s assessments, we also need to be pragmatic as a society. We can do this by carefully harnessing the possibilities of artificial intelligence towards assessments that are ‘sufficient to provide the services needed, in a fair way, to the range of people needing them, within the resources available’. Just as psychometrics was once a new and controversial technology so too now is artificial intelligence. How best and most responsibly to combine them is our urgent, moral, practical and scientific challenge, which we must collectively address.