by Charles Woodward, Mika Hakkarainen and Timo Kuula
Augmented Reality (AR) can help mobile workers to obtain timely and accurate information related to maintenance targets. We implemented a mobile AR system based on the BIM representation of a building, combined with FMS and other data sources. Technical challenges included indoors locationing and 3D tracking of the mobile device. User evaluations were conducted in two phases, first providing valuable input for system design, and finally showing very positive results on technology acceptance.
Building maintenance workers have a number of locations to visit and they are constantly on the move, thus their tools should support mobile work. Also, the workers frequently meet with new locations and unknown equipment, requiring tools to provide them with up-to-date information at the site. Mobile Augmented Reality can be used to visualize alerts and operating instructions directly on the target, thus helping mobile workers with improved situational awareness, also reducing the workers’ need to shift their attention from the work target to external devices or manuals.
VTT was among the first in the world to implement a mobile AR system for building maintenance workers. The work was conducted in the national project DigiSpaces 2011-2013, with support from Tekes (the Finnish Funding Agency for Innovation) and involving industrial partners Granlund (building services), Pöyry (engineering), Skanska (construction), Tekla (BIM design), Solibri (BIM verification), and Nokia (mobile devices and location based services).
The implementation was based on participatory design (PD) and user centric (UCD) principles, involving several users and experts from a group of industrial partners. The users in the group interviews provided a large number of good quality ideas and generally saw that a mobile tablet application would be easily utilized in their work. Among the most important findings in the first round of user tests, it was concluded that in addition to Building Information Models (BIM), other data sources such as Facility Management System (FMS) should also be integrated into the system. Other features that were proposed by the users included Virtual Reality (VR) view for remote inspections.
These and several other ideas from the user studies were taken into account in the final system implementation. The system allows the user to view the BIM with maintenance targets and alarms in a real time video view on a Windows tablet. The BIM is partly transparent when visualized in the right position and orientation, giving an illusion that the BIM is merged with the real building, enabling the user to “see behind walls” etc. The user may freeze the view, investigate the problem in more detail, and select parts for additional information. The user is able to add feedback on the part, e.g. mark a maintenance procedure being done or pending for further action.
Locationing and orientation tracking with the system is based on hybrid markerless tracking. The final solution provides very good accuracy, speed and robustness, combining elements of VTT’s point cloud based computer vision tools (ALVAR Tracker), orientation sensors (IMU) and indoors locationing (Nokia’s HAIP). Furthermore, an autonomous mapping robot was implemented to automatically acquire and update the point cloud data for tracking.
Figure 1: Mobile user browsing the air conditioning problem in AR view (left). BIM with malfunctioning devices highlighted (right).
Figure 2: Mobile user finding warnings and alarms in a room (left). Alerts highlighted in mobile AR view (right).
With the system implementation ready, we implemented a second round of user tests with a real world pilot case. Figures 1 and 2 show two use case scenarios; a video of the pilot is provided in YouTube (see Links). The results indicate very good user acceptance and strong potential for utilizing BIM data and mobile AR solutions in building maintenance work. On a scale of 1 to 7 (1 = completely disagree, 7 = completely agree), some of the average user ratings were:
- The tablet system would be useful maintenance work – 6,50
- AR visualization of the targets would help maintenance work – 6,00
- BIM contents of the building on a mobile device would help in maintenance work – 6,38
- Use of the current version of the tablet system was easy – 4,75
- It is worthwhile to develop the system further into a completed product – 6,63.
In conclusion, we hope that BIM practices will be developed to better serve the whole building life cycle, besides their current use for planning and construction. Much of the current BIM information is actually not required after the construction phase, and it should be made easy to extract the relevant components and level-of-detail for having BIMs serve as a living document and data repository for building life cycle management.
Links:
VTT Augmented Reality / 3D Tracking website: http://www.vtt.fi/multimedia
http://youtu.be/uYFtYbqvoq0
References:
[1] T. Kuula et al.: “User requirements for mobile AR and BIM utilization in building life cycle management”, Proc. CONVR2012, Taipei, Taiwan, 1-2 November, 2012, pp. 203-211.
[2] C. Woodward et al.: “Implementation and evaluation of a mobile augmented reality system for building maintenance”, Proc. CONVR2014, Sharjah, UAE, 16-18 November, 2014, pp. 306-315.
Please contact:
Charles Woodward
VTT, Finland
E-mail:
