by Benedetta Catricalà, Marco Manca, Fabio Paternò, Carmen Santoro and Eleonora Zedda (ISTI-CNR)
We present an approach to novel digital cognitive training through serious games able to adapt to personally relevant material from the older adult’s life. The games are based on memories associated with the older adult’s biography, thus making interactions personalised, relevant, and more engaging. The serious games are accessed through humanoid robots, which can make the training exercise more engaging because of their human-like behaviour.
Currently, cognitive stimulation exercises conducted with mild cognitive impairment (MCI)-affected older adults are typically administered by care professionals through exercises, which require a high degree of involvement by professional therapists and are often perceived as tedious by older adults.
One way to overcome such issues is to offer older adults opportunities to engage in mentally stimulating activities. Though researchers have found that digital games may provide benefits for some cognitive domains, the game industry seems to have ignored this important part of the population as a special category of users. Tahmassebi [1] found that older adults complain about the complexity of both hardware and software, which can be problematic due to their age-related cognitive and physical limitations. So far there have been mixed results related to game-based interventions [2]. Unfortunately, despite the potential benefits to cognition described above, older adult populations are generally not targeted audiences when it comes to the design of digital games. However, we have found [3] that emerging humanoid robots may open up new possibilities in more effectively engaging MCI older adults during repetitive cognitive training. While the use of multiple modalities such as speech and visual interfaces has already been explored for older adults, social robots have recently shown to have the potential to support care and independent living because they are now capable of exhibiting natural-appearing social characteristics. Thus, they can help formal and informal caregivers to monitor and support older adults, in particular those with MCI, while also providing a mixture of empathy, motivation, encouragement and companionship.
For this purpose, we have designed a novel solution of serious games, based on users’ memories and played through the social humanoid robot, which aims to help older adults to maintain their cognitive functional level, and prolong independent living. The serious games, installed on the humanoid robot, motivate older adults by engaging them in playful situations that draw on their personal memories with which they can interact. Since the games are based on elements associated with the biography of the older adults, they can make interactions more relevant and more likely to keep the older adults engaged while enhancing their well-being as well.
Design
For the motivations discussed in the previous section, in the context of the SERENI (SERious gamEs with humanoid robots in cogNItive training) CNR Project, we have designed a platform (see Figure 1) to deliver serious games able to exploit personally relevant material from an older adult’s life through a humanoid robot. It aims to stimulate cognitive functions through play sessions, which should last about 20 minutes. The exercises should be useful for making the participants think and reason to provide the correct answers. The platform can be a solution for day-care centres where older adults with mild cognitive impairments can go to perform relevant exercises.
Figure 1: The SERENI Platform.
The platform is based on various components. The first one is the Remind App, which is a responsive multimodal web application to collect memories from the older adults and their relatives, which can be entered by both graphical and vocal interaction.
Six categories of memories have been identified:
- Music-related, concerning songs or singers of their youth;
- Game-related, activities they liked to play;
- Event-related, specific events that they still remember;
- Locations, which are associated with particular memories of travels;
- Food-related, with the possibility to indicate the corresponding recipes by listing the steps that should be performed;
- Affects, memories related to family members and loved people.
The entered memories are stored in a database from which they can be automatically retrieved and exploited to provide contents for the games to play with the humanoid robot. We have used a Pepper robot for proposing the games. It is a 1.2-metre-tall robot with the ability to use various interaction modalities (speech, touch, gestures, graphics through a screen on the chest).
An initial set of four types of games have been identified:
- Memory completion, where a memory presented by the robot has a missing detail, which should be indicated by the user;
- Activities ordering, which can be applied to food recipes, in which a list of relevant tasks should be put in the right order by the user;
- Associate memories, where 3–4 memories are briefly described along with a set of details, and the user has to connect each memory with the corresponding detail (for example associate song titles with the corresponding singers);
- Memory-related question, where the user has to answer specific questions about an event drawn from the memories (for example: what happened in 1945?).
The platform is also able to store some data regarding the user performance, such as when and for how long the user played with a given game, the number of errors made in a session, and the type of games that have been played. Such information can be delivered to older adults and their caregivers both through the robot and the web app.
Conclusions
We have presented an approach to memory-based games delivered through a Pepper humanoid robot. We have identified a first set of memory and game types, which can be extended in the following. A first prototype of the platform for collecting the memories, and exploiting them in a set of games for cognitive stimulation has been delivered, and is under test with older adults with mild cognitive impairments in a Train the Brain programme to assess their user experience with such games.
Links:
[L1] https://hiis.isti.cnr.it/lab/home
[L2] https://hiis.isti.cnr.it/sereni/index.html
References:
[1] S. Tahmassebi, “Digital Game Design for Elderly People”, in Digitala Vetenskapliga Arkivet, 2018.
[2] K. A. Blocker, T. J. Wright, W. R. Boot, “Gaming preferences of aging generations”, in Gerontechnology: International Journal on the Fundamental Aspects of Technology to Serve the Ageing Society, 12(3), 174, 2014.
[3] M. Manca, et al., “The impact of serious games with humanoid robots on mild cognitive impairment older adults”, in International Journal of Human-Computer Studies, 2021.
Please contact:
Fabio Paternò, CNR-ISTI, Italy
