by Björn Löfvendahl, SICS Swedish ICT
For a global industrial company like ABB, sending maintenance specialists around the world costs a lot of money and is neither time efficient nor environmentally friendly. Augmented reality allows the specialist to stay in the office and guide a local employee remotely.
ABB FACTS runs large power installations in many parts of the world, often located far from cities and airports. Usually the sites have their own service personnel, but sometimes when unexpected problems occur it is necessary to send in maintenance specialists, incurring significant expenses for both ABB and its customers. Furthermore, flying in specialists is neither time efficient nor environmentally friendly. This raises the question as to whether the problems could be solved remotely without having to fly in external personnel.
The project INCODE [L1] (Information and Competence on Demand) investigates how modern technology such as augmented reality (AR) and virtual reality (VR) can be used to facilitate the transfer of knowledge within a company. INCODE is run by SICS Swedish ICT Västerås together with Interactive Institute Swedish ICT Piteå and several companies, as part of the Process Industrial IT and Automation programme (PiiA), a Swedish strategic innovation programme. The project will run from summer 2015 to spring 2017.
AR is about superimposing digital objects like 3D models or text onto the real world around you, as opposed to VR where you are totally immersed in a virtual world. By using AR equipment developed by XMReality, one of the project partners, the project aims at solving ABB’s problem with dispersed service. With this equipment it is possible to guide another person, a “follower”, remotely using both voice and hands (see Figure 1). The guide sees what the follower sees and can use her hands to show how certain task should be performed. The follower wears AR goggles and can both hear the guide’s voice and see her hands superimposed in front of him. Using AR for remote guidance has been done before [1], but what is interesting about this equipment is the potential to use gestures in a natural way.
Figure 1: The AR equipment. A screen together with a camera pointing downwards for the guide (left) and a pair of HUD goggles with a front facing camera for the follower (right).
In a pilot study prior to INCODE the AR equipment was evaluated. Tests were performed in an ABB laboratory where two different ABB technicians were given a maintenance task to complete. One user performed the task guided with the AR equipment and another user was only guided using a mobile phone. Although this was only a small test there were some interesting observations. The technician with the AR equipment perceived a much higher sense of safety compared to the other, as the guide could see what the technician using AR was doing and correct him if he did something wrong. A third test with a user, equipped with AR gear but with no connection to or knowledge of ABB’s equipment, was also performed. This user had no problems performing the same task as the more experienced technicians. In fact, some parts of the test went even faster compared to the other since after a while this user started acting like a robot, doing what the guide told her to do without questioning.
INCODE continues the work started in the pilot and plans to do tests in real industrial environments. There are several conditions that have to be studied further, including:
- What happens if the connection between the guide and the follower is broken?
- Is it possible to integrate XMReality’s and a customer’s systems and what advantages would this have?
- What routines must be implemented to make sure that the follower will not expose himself to danger just because the guide tells him to perform a certain task?
Being able to quickly perform remote service has many advantages over having to travel a long distance and perform the service in place. For ABB’s customer it would mean that the installation would be up and running much faster, which would save them a lot of money. ABB would be able to coordinate maintenance tasks much more easily and the specialists would not have to waste time on travelling. Reduced travelling would also mean reduced travel costs and a more environmentally friendly way of working.
There are still several questions and issues to solve, but we believe that AR has the potential to revolutionize how industrial maintenance will be performed in the future.
Link:
[L1] https://www.sics.se/projects/incode
Reference:
[1] J. Wang et al.: “An Augmented Reality Based System for Remote Collaborative Maintenance Instruction of Complex Products”, Proc. IEEE CASE 2014, Taipei, Taiwan, 2014.
Please contact:
Björn Löfvendahl, SICS Swedish ICT
