by Mathieu Razafimahazo, Nabil Layaïda, Pierre Genevès and Thibaud Michel
The TyReX research team at the Inria Grenoble - Rhône-Alpes Research Centre, France is working on the design of a high-accuracy localization system. This key technology allows for a smooth integration of vision-based techniques in augmented reality applications [1].
Since 2007, the widespread use of mobile devices (e.g., phones, tablets) and emergence of a new generation of devices which are equipped with more precise sensors and cameras has placed augmented and virtual-reality technologies within reach. Such applications are very resource-demanding and critically interdependent on the quality of user-location information and environment tracking (e.g., buildings, monuments). Several mobile services based on augmented-reality technologies have already been developed. These can aid navigation, offer context-aware access to a wealth of geo-tagged content related to cultural heritage, provide information on city events and assist with urban planning issues. However, these services still lack enhanced and robust tracking techniques that are sufficiently effective enough to make the applications reliable and widely deployed.
Today, one of the main challenges in mobile augmented reality applications design is understanding how our perception of reality can be profitably augmented and how this augmentation can be made to fit seamlessly with the user’s interaction with the world. The European VENTURI project which began in 2011 is aiming to develop the first generation of ubiquitous augmented reality (AR) tools that meet real user needs and fit within the context they operate in. To illustrate the potential impacts such technologies might have in our everyday lives, a demonstration will be conducted in front of a large audience in late 2014. This demonstration will be a cultural-heritage visit of Grenoble (France) presented on smartphones (Figure 1). The application features an augmented tour of Grenoble’s antique neighbourhood as far as the 19th century fort (476m elevation). Along the route, the application behaves like an audio AR guide, delivering walking directions and historical information. It also includes a set of immersive virtual history galleries. Overall, it exposes the user to Grenoble’s past and present by combining 3D-reconstructed monuments and old geo-positioned pictures. For example, the city’s old roman fortifications are overlaid on a live camera view using visual-recognition techniques. Finally, to further enhance user immersion, the application provides different 3D soundscapes synchronized with the visual rendering.
This demonstration addresses a wide spectrum of the technological challenges a project faces including audio-visual scene analysis to understand the user’s context, the collection, creation, fusion and delivery of AR content, 3D audio rendering, mobile human-machine interactions and finally the provision of a high-accuracy localization system. To this end, the VENTURI project consortium brings together different ERCIM members such as Inria, Fraunhofer, FBK, mobile device manufacturers SONY and ST Microeletronic and software companies like Metaio GmbH and EDIAM Sistemas. The common goal of all these partners is to design a hardware and software platform dedicated to such applications.
Of these technologies, the high-accuracy localization is central to allowing for smooth integration with vision-based techniques. The TyReX research team, located in the Inria Grenoble - Rhône-Alpes Research Centre, France has been working since 2010 on the design of such a localization system, commonly known as Pedestrian Dead-Reckoning (PDR). This system is based on the traditional micro-electro-mechanical systems (MEMS) used in smartphones. The on-board accelerometer and compass are used to provide an estimation of the user’s relative position and orientation in ‘urban canyon’ environments (defined as areas with poor GPS-signal strength such as indoors). Combined with geographical data, this component can also be used to identify user-activity patterns (e.g., walking, running or being in an elevator). The interpretation of sensor values, coupled with different walking models, allows us to ensure continuity in determing the user’s location when they are both indoors and outdoors. However, dead recknoning is subject to cumulative errors that are driven by multiple factors (e.g., sensor drift, missed steps or poor estimation of stride length). These errors can be reduced by fusing various external signals from sources such as GPS and Wi-Fi or by relying on the analysis of a user’s trajectories with the help of a structured map of the environment.
PDR has played a key role in the VENTURI application since the delivery of its AR content is mainly based on the user's estimated location. PDR and the other technologies mentioned in this article aim to provide available information in a ‘user-centric’ way as opposed to a ‘device-centric’. The 2014 demonstration in Grenoble, which combines advanced AR, indoor and outdoor PDR and 3D vision tracking will be this project ‘s final deliverable.
Links:
TYREX team: http://tyrex.inria.fr
VENTURI project: http://venturi.fbk.eu
Reference:
[1] J. Lemordant, T. Michel, M. Razafinahazo: “Mixed Reality Browsers and Pedestrian Navigation in Augmented Cities” in proc. of the Graphical Web Conference, Oct 2013, San Francisco, USA, http://hal.inria.fr/hal-00872721/en
Please contact:
Nabil Layaïda
Inria Rhône-Alpes, France
E-mail:
