by Arzu Guneysu Ozgur, Barbara Bruno, Thibault Asselborn and Pierre Dillenbourg, (EPFL)
A large body of research suggests that robots could indeed be useful for supporting children in learning handwriting. However, few studies have investigated the role and use of tangible robots in teaching handwriting to children with attention and/or visuo-motor coordination difficulties. Over the course of multiple iterations, globally involving 17 typically developed children and 12 children with attention and visio-motor coordination issues within one school and two different therapy centres, we have designed a robotic activity to teach the grapheme (shape) and the ductus (the way to draw) of cursive letters.
Research shows that having richer sensory information enhances visual perception and visual-motor coordination [1]. To let children experience a range of sensory information during letter learning, teachers use techniques such as drawing letters in sand-filled boxes or touching the letters carved in a piece of wood. The same principle is adopted by therapy centres for the development of language skills, such as the “draw on your back” game. During the game, a child and a therapist take turns in drawing a letter on the other’s back with their finger and guessing what is drawn.
Similarly, robots can be used as tangible tools enhancing sensory information, as we demonstrated in a previous study that used the Cellulo robots to help pre-schoolers learn the grapheme and the ductus of letters [1]. Cellulo is a palm-sized, haptic-enabled robot developed at EPFL [L1] [2] that can move and be moved on “maps”, i.e., printed sheets of paper covered with a dotted-pattern that enables accurate localisation (Figure 1 shows children interacting with their robots on letter maps). Cellulo robots are versatile, easy to set up and well-suited for classroom activities. They are controlled by an application running on a computer or a tablet and feature various interaction modalities; many such robots can also be simultaneously used as a “swarm” in an activity [3].
Our goal, inspired by interactions with teachers and therapists, was to design a modular, highly engaging, highly adaptable robot-assisted activity to help children with attention and/or visuo-motor coordination issues in learning the grapheme and ductus of cursive letters.
Through several iterations within a school and a number of therapy centres, we designed an activity composed of three sub-activities targeting different aspects of letter handwriting learning. The setup envisions multiple kids sitting at a table, each with a Cellulo and a map displaying the grapheme of the letter, as shown in Figure 1. The sub-activities are:
1. “Watch the Robot”: Cellulo autonomously moves along the letter’s grapheme on the map, following the ductus. The child only watches the robot and the letter’s phoneme is played at the beginning and end of the robot’s writing, to strengthen the link with the corresponding grapheme and ductus.
2. “Feel the Robot”: We ask the child to put their hand on Cellulo while it moves along the letter’s grapheme to passively feel the motion of the robot. Experiments with children with visio-motor coordination difficulties highlighted the importance of this activity: the child has to adjust the force applied on top of the robot, since applying excessive force prevents the robot from moving.
3. “Drive the Robot”: The child actively drives Cellulo along the letter’s grapheme, trying to repeat the ductus seen previously. Although the child has full control over the robot’s movements, Cellulo provides haptic feedback, trying to lead the child to the right path if it is moved away from it. Additionally, Cellulo’s LEDs are blue when the correct path is followed, and red otherwise. This visual + haptic feedback helps the child recognise and avoid errors.
Figure 1: Robot-assisted writing activity with Cellulo: Children interact with the haptic-enabled Cellulo robots on maps displaying the grapheme of a cursive letter, where the start and end of the letter are indicated with a starting cue and a trophy respectively. After three different sub-activities where the robots provide a range of sensory information, a final team activity in the form of a guessing game is played.
The design includes a final team activity in which three to four children, alternatively playing as one “writer” and many “guessers”, sit side-by-side at the table, with a barrier in between preventing the guessers from seeing the writer. Using the map as a reference, the “writer” is asked to write a letter using Cellulo as done in “Drive the Robot”. The movements of the “writer” robot are replicated in real-time by the “guesser” robots, autonomously moving on a blank map placed in front of each “guesser”. The goal of the game is for the “guessers” to successfully guess the letter drawn by the “writer”. During one iteration, therapists suggested to adapt this game to be more challenging for the “writer”: to this end, we tested the case in which the “writer” is also given a blank map and has to remember both the proper grapheme and ductus for the letter that they are instructed to write. The game thus becomes a team effort in which wrong guesses from the “guessers” are fed back to the “writer” and the goal is for the team to make the “guessers” correctly guess the letter with the least number of attempts. This adaptation also allows us to track the progress of the child in handwriting, by analysing the “writer” robot’s trajectory.
The map also proved to be a crucial adaptation element: different institutions use slightly different graphemes for the same letter, as well as different cues for the initial strokes. For this reason, we designed multiple variants of the maps (e.g., featuring car-racing elements to introduce a gaming dimension or skate ramps and sea waves to relate to one way in which specific strokes are taught).
Finally, therapists highlighted the importance of knowledge transfer, i.e., ensuring that children who learn the appropriate ductus and grapheme with the robot are capable of writing equally well with pen and paper. Thus, in one iteration we interspersed pen-and-paper writing tasks between the sub-activities.
Encouraged by the positive results of the iterations, we are working to test the activity with more children and institutions. In sharing the lessons we learned during the design iterations we hope to inspire and support colleagues working in the field... and get useful feedback!
Link:
[L1] https://www.epfl.ch/labs/chili/index-html/research/cellulo/
References:
[1] T. Asselborn, A. Guneysu, et al. “Bringing letters to life: handwriting with haptic-enabled tangible robots”, in Proc. of the 17th ACM Conference on Interaction Design and Children (IDC '18). ACM, New York, NY, USA, 219-230, 2018.
[2] Ö.Özgür et al.: “Cellulo: Versatile handheld robots for education”, in 12th ACM/IEEE International Conference on Human-Robot Interaction, pp. 119-127. IEEE, 2017.
[3] A. Özgür et al.: "Declarative Physicomimetics for Tangible Swarm Application Development." Proceedings of the 11th International Conference on Swarm Intelligence, ANTS 2018. Vol. 11172. No. CONF. 2018.
Please contact:
Arzu Guneysu Ozgur
EPFL, CHILI (Computer-Human Interaction in Learning and Instruction Laboratory), Switzerland.
