The Vgate project introduces a new type of immersive environment that allows full-body immersion and interaction with virtual worlds. The project is a joint initiative between computer scientists from research teams in computer vision, parallel computing and computer graphics at the INRIA Grenoble Rhône-Alpes, and the 4D View Solutions company.
A Virtualization gate virtualizes users into graphical objects that capture both their shape and their appearance in real time. These graphical objects can be plugged into any virtual reality application including immersive and interactive applications where users can see and act on virtual worlds. Geographically distant users can also be immersed into a common virtual environment for collaborative applications.
Images from the VGate project.
Traditional immersive solutions developed in the virtual reality community are generally based on advanced display technology such as head-mounted displays (HMDs) and immersive multi-projector environments like Caves. Though they provide an impressive sense of immersion, users tend to be limited both in their interactions with virtual objects, and in their presence (eg appearance) in the 3D world. The main reason for this lies in the perception capabilities of the environment. Interactions usually rely on sensors that provide local information on position or velocity, for instance, but not full-body or appearance information. In contrast, the Vgate environment relies on cameras that provide both geometric and photometric information on users’ body shapes. The contribution of Vgate is therefore to associate multi-camera 3D modelling, physical simulation and tracked HMDs for immersion and interaction purposes.
Hardware Setup
Vgate uses several video cameras and 3D modelling tools to build a graphical model of the observed shape in real time (about 20 frames per second). This model is fed into a physical simulation where it becomes a solid object that can act upon other objects. User-centred visualization is provided through a head-mounted display that is tracked with an infrared positioning system (Cyclope). Computations are distributed on a PC cluster to enable real-time execution.
Software Components
Vgate uses the following software:
- Computer Vision: silhouette-based models are computed in real time from the video streams. They are represented with meshes onto which the acquired images are back-projected to produce photorealistic models (http://www.4Dviews.com).
- Simulation: the Simulation Open Framework Architecture (SOFA) runs the physical simulation. It allows objects of very different natures to interact, including rigid bodies, deformable objects and fluids (http://sofa-framework.org).
- Middleware: the Vgate application is developed on top of the FlowVR library, a middleware dedicated to high-performance interactive applications. It enforces modular programming through a hierarchical component model that leverages software engineering issues while enabling efficient execution on parallel architectures
(http://flowvr.sourceforge.net/).
Link:
http://vgate.inrialpes.fr/
Please contact:
Edmond Boyer
Grenoble Universities and INRIA Grenoble, France
E-mail:
