Augmenting the Rubber Hand Illusion

Augmenting the Rubber Hand Illusion

by Filip Škola, Szymon Fiałek and Fotis Liarokapis

Augmented reality (AR) is a technology that merges real and virtual information in real-time performance. AR introduces new opportunities in a number of application domains, one of the least explored to date being perception and psychology. Researchers from the HCI Lab developed a novel AR experiment in order to test the effects of the well known rubber hand illusion.

The traditional rubber hand illusion is an old psychological experiment where participants are under the illusion that a rubber hand is part of their own body. During the experiment, the rubber hand is positioned in front of the participant while their real hand is kept hidden from their view. Synchronous touches are then applied to both their real hand and the rubber hand and within minutes participants get the illusion that the rubber hand is part of their body [1]. This experiment has been previously exploited in immersive virtual reality (VR) environments [2]. The purpose of this study is to investigate whether AR can be used as a medium to provide a similar level of ownership.

Compared to the aforementioned experiment where a plastic rubber hand was used, a virtual 3D representation was chosen to create the same illusion this time in an immersive AR environment. The 3D rubber hand was made out of photogrammetric techniques and belongs to one of the lectures of Masaryk University. In particular, 15 high definition images were taken and then they were processed using medical imaging software to produce the 3D mesh. Textures were also taken from the high definition images and the resulted 3D model consists of 46611 vertices and 93218 triangles offering a very realistic model of a human hand.

For the visualization, a state-of-the-art head-mounted display (HMD) was used for presenting the augmented rubber hand illusion to the participants. The device used was the lightweight Vuzix Wrap 1200DXAR. The whole scene was then ported into the ARToolkit software tool. The AR application displays an animation of the hand being stroked by a virtual brush. Moreover, an EEG 32-sensor device called Enobio-32 was used and the electrodes were located at the extended 10-20 system. The raw EEG has usually been described in terms of frequency bands: gamma (greater than 30 Hz) beta (13-30 Hz), alpha (8-12 Hz), theta (4-8 Hz), and delta (less than 4 Hz). The setup of the electrodes takes approximately 30 minutes and the AR experience approximately three minutes.

Experiments were performed on 21 healthy volunteers (six females and 15 males), aged 20-35 years old. Participants were asked to complete two different questionnaires, one measuring their cognitive workload (based on the standard NASA TLX questionnaire) and another one regarding their experience with the rubber hand illusion. In addition, EEG signals of the individuals were recorded and stored for further processing. Figure 1 illustrates a participant experiencing the AR rubber hand illusion.

Figure 1: A participant experiencing the AR rubber hand illusion.

In the analysis of the questionnaire data, four questions were found to be correlated with the recorded EEG data. These were (a) I felt as if the rubber hand was controlling my will; (b) I felt as if the rubber hand was controlling my movements; (c) I could sense the movement from somewhere between my real hand and the rubber hand; (d) It seems as if the rubber hand has a will of its own. The questionnaire results are presented in Table 1.

Results indicate that agency control, understood as the subjective awareness that one is initiating, executing, and controlling one’s own volitional actions in the world, correlated positively with Theta frequency (r = 0.54, n = 21, p < 0.02). Theta frequency is believed to be associated with the intention of movement [3]. Agency control also correlated positively with Alpha frequency
( r= 0.55, n = 21, p < 0.01 ). Brain activity resulting in increased alpha frequency is believed to be connected with higher state of relaxation. Frustration measured by NASA TLX correlates positively with Delta frequency (r = 0.44, n = 21, p = 0.04).

Concluding, this prototype experiment demonstrated that the AR medium seems to be providing a similar level of ownership as reported in the original rubber hand illusion experiment. In the future, a comparative analysis between the original rubber hand and the AR representation will be performed with more participants to measure more accurately the level of ownership.

Links:
Medical Imaging software: http://www.canfieldsci.com/imaging-systems/mirror/
Vuzix Wrap 1200DXAR: http://www.vuzix.com/augmented-reality/products_wrap1200dxar/
ARToolkit: http://www.artoolkit.org/
Enobio-32: http://www.neuroelectrics.com/products/enobio/enobio-32/
HCI Lab: http://decibel.fi.muni.cz/wiki/index.php/
Personal Web Page: http://www.fi.muni.cz/~liarokap/

References:
[1] Kalckert, A., Ehrsson, H.H. The moving rubber hand illusion revisited: Comparing movements and visuotactile stimulation to induce illusory ownership, Conscious Cogn. 26:117-32, 2014.
[2] Ye, Y., Steed, A. Is the Rubber Hand Illusion Induced by Immersive Virtual Reality?, Proceedings of IEEE Virtual Reality 2010, 95-102, 2010.
[3] Whishaw, I.Q., Vanderwolf, C.H. Hippocampal EEG and behavior: changes in amplitude and frequency of RSA (theta rhythm) associated with spontaneous and learned movement patterns in rats and cats, Behav Biol 8 (4): 461-84, 1973.

Please contact:
Fotis Liarokapis, Masaryk University, Faculty of Informatics, HCI Lab, Czech Republic
Tel: +420 549493948
E-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.