Development of a deep-learning algorithm for autonomy evaluation in children with autism from RGB-D videos

Autism spectrum disorder (ASD) is a chronic childhood-onset neurodevelopmental condition with effects on adaptive functions throughout life [1]. The worldwide population prevalence for autism is 1%, increasing in the last decades. The underlying reasons for this increase are not fully understood [2]. The growing interest in ASD is due to the severe invalidation of affected subjects but also of the great conditioning of the relatives’ lives which are the real caregivers for theautistic. The cost of autism, over the lifespan, is about double for an affected subject w.r.t. a person without this kind of disability [3]. The economic burden of ASD in US is of hundreds billion dollars projected to nearly double by 2025. People more able to communicate, care for themselves and participate in the workforce at greater levels, will need less financial support in their lives [4]. The average age for ASD diagnosis in the United States (US) stands at 5.7 years and the 27% of the subjects with autism remains undiagnosed at 8 years. ASD should be diagnosed in the early years of life because the affected subject, with a personalized therapy, shows encouraging improvements. This outcome has led the study of ASD in several fields, from genomics to biochemistry and physiology, passing through food science and psychology. Nowadays, the ASD diagnosis is based on assessing answers given by the subject’s relatives during interviews focused on 3 developmental domains: communication and social interactions, restricted interests and behaviors, and stereotypical behaviors. Other approaches try to extract stereotypical motor movements patterns ASD-related from accelerometric signals, or by video observation to analyze eye-gaze movements, the level of engagement and the emotional state of the children which can be associated to ASD syndrome. In this context, the application of novel learning algorithms has gained a role in analyzing and infer over one or more types of information acquired by traditional methods to reduce the time requested for the autism assessment. These algorithms can extract more informative content from available datasets. Examples are: alternating decision tree (ADTree) [5, 6] and support vector machine (SVM) [7] in evaluating diagnostic interview (ADI); Naïve bayes and random forests [8] to determine ASD traits like developmental delay, less physical activity; neural networks, SVM and random forest to identify ASD patients using brain imaging; SVM [9] to analyze eye movements; convolutional neural networks(CNN) used to recognize stereotypical motor movements from accelerometric signals [10]. The applied behavioural analysis (ABA) is a science dealing to shape an individual’s behavior [11], and is a standard in ASD therapy. In this work, a deep learning(DL) approach is proposed to evaluate the autonomy of autistic children in performing daily life activities, namely the hands-washing action. To the scope, a dataset of videos was acquired during the sessions of the ABA therapy at the facilities of ”Il faro”. The selected frames, have been annotated as belonging to class no-aid or to class aid, then used as input to the algorithms. The goal is to provide to the professionalities involved in the therapy, a tool that easily, rapidly and correctly gives them a picture of the subject’s condition, reducing the assessment time and customizing the therapy. The proposed archictectures are 2 CNN, VGG16 and ResNet50. Both the nets were used in a from scratch version and in a pretrained version to achieve better results thanks to the knowledge acquired during the training on the ImageNet dataset [12]. Encouraging results were obtained even if the dataset was limited to 9700 frames. The fine-tuned ResNet50 was the best model with an accuracy of 0.83. To conclude, this work has shown that the use of DL methods with a simple acquisition setup, makes the evaluation faster and objective, allowing the therapy personalization.