by Eleonora Zedda (University of Pisa and ISTI-CNR ), Marco Manca and Fabio Paternò (ISTI-CNR)
Serious games in humanoid robots have an interesting potential to help older adults with cognitive stimulation in non-pharmacological treatments. Additionally, exploiting the multimodal possibilities of the humanoid robots in such a way to provide them with a personality can be suitable to improve users' engagement, and thus their potential participation in cognitive training programs.
The number of people aged 65 or older is projected to grow by nearly 1.5 billion by 2050. Ageing has a considerable impact on the health of older adults in terms of cognitive and physical impairments, which influence the abilities to complete and perform basic activities of daily living, such as cooking, shopping, managing the home, bathing, and dressing. Additionally, given the high healthcare expenditure at older ages, and effect on family caregivers, new technologies to assist older adults with cognitive impairments are urgently needed.
In recent years, humanoid robots have increased their similarity to human behaviour thanks to their ability to perform gestures and facial expressions, interact vocally and move. These capabilities are essential to creating social and emotional interaction with the users to increase their acceptability and engagement, which may increase the possibility of reaching the goal of the assistance in less time and with better results. Furthermore, serious games can provide positive results in stimulating older adults and helping them improve their cognitive abilities with respect to traditional training.
This study aims to design and implement a serious game for a humanoid robot able to show different personalities through different multimodal interactions to provide a more engaging interaction and obtain better results in terms of cognitive improvements.
At the HIIS Laboratory of CNR-ISTI [L1] we design serious games that stimulate multiple cognitive domains for reducing the risk of cognitive decline and build more powerful cognitive resources [1]. In this study, we chose to design a cooking game requiring users to recognise the recipe ingredients’ chronological sequence, the typology of the ingredients, and their weight. It aims to stimulate working memory, visual memory, and verbal memory. During the game, the robot shows and vocally synthesises the ingredients for the selected recipe. Then, it starts the quizzes, during which the user should use visual memory and verbal memory to recognise the right ingredients and select them over other options. In this game, the types of interaction supported are vocal, graphical and touching the robot sensors. The robot that we use in our project is Pepper Robot developed by Softbank's Robotics (a 1.2-m-tall wheeled humanoid robot, with 17 joints for expressive body language). During the user’s interaction, the robot can show two different types of personality traits: a calm and introverted personality, and an extroverted and active personality. We based the implementation of these personalities according to the Big Five-Factor Model [2]. In summary, the extrovert personality is shown through a more joyful and active interaction while the introvert with a more neutral and calm interaction. Both personalities are designed to modulate different parameters related to the verbal and non-verbal channels. In particular, for the verbal channel, we modulate the robot’s intonation, pitch, volume rate and speech rate while for the non-verbal channel we modulate the robot’s gestures, movements speed, motor trajectories and motor orientation.
The sessions are structured into three main phases: introduction, play and results. In each phase, the robot shows the personality chosen modulating the parameters identified for that personality. For example, when the robot asks the question regarding the chronological sequence of the ingredients, at the same time it displays and vocally renders the question and generates some animations. After the user answers the question, reinforcing feedback is provided by Pepper in the form of vocal feedback. Nonverbal feedback is combined with verbal and visual feedback, for example in the extrovert personality, Pepper raises the right arm and moves its elbow up and down, and simultaneously it nods its head twice (see Figure 1). During all the sessions, the robot provides different animations and vocal and visual feedback for various aims: to stimulate the user to focus on the game, create more engagement and stimulate attention in the user, and create a more natural interaction and communication.
Figure 1:Animation for positive answer in extravert personality.
Figure 2: Animation for positive answer in introvert personality.
We performed a first validation of the game design with seven experts in psychology, who played the game and provided positive feedback and found the game as a helpful tool for cognitive stimulation. Indeed, the combination of the game and the robot stimulate multiple cognitive domains and are useful in the cognitive context because they improve engagement and attention. For what concerns the evaluation of the robot personalities, the two personalities were found well distinguished and representative of both conditions, and the users were satisfied with the interaction with the robot performing both. In future work, we plan to assess the engagement and usefulness of cognitive assistance provided by a humanoid robot performing two personalities with older adults in the serious game.
Link:
[L1] https://giove.isti.cnr.it/lab/home
References:
[1] M. Manca, et al.: “ The impact of serious games with humanoid robots on mild cognitive impairment older adults”, International Journal of Human-Computer Studies 145 (2021): 102509.
[2] O. P. John, et al.. “The Big Five Trait Taxonomy: History, measurement, and theoretical perspectives.” Handbook of personality: Theory and research, 2nd ed., pp. 102–138 (1999).
Please contact:
Eleonora Zedda, CNR-ISTI and University of Pisa
