by Jacques G. Verly, Jerome Meessen and Benoit Michel
3D MEDIA is part of a new portfolio of R&D projects that will run for five years, financed by the European Regional Development Fund (ERDF). 3D MEDIA covers most scientific and technological aspects concerning the management and processing of audio and video contents. It involves the three largest Universities in Wallonia, the Faculté Polytechnique de Mons (FPMs), the Université Catholique de Louvain (UCL), and the Université de Liège (ULg); it is managed by the Multitel Research Center.
3D MEDIA has already given some spectacular results, such as NeuroTV's capability of projecting in 3D stereo a synthetic avatar (Toon) animated backstage by an artist (see figure).
While not directly financed by the project, industry is a key player of the project. First, the consortium reaches out to the audio and video industry to identify scientific and technical problems that industry does not have the time, the manpower, or the skills to solve. These problems are turned into well-defined challenges; the universities work on them for, say, 18 months, and the results are then transferred back to industry with appropriate licensing agreements. The overall view is a series of challenges staggered in an innovation pipeline. Second, the three Universities, which are fully financed, are also free to explore more challenging and risky avenues and, where warranted, to transfer this technology to industry. Given that Multitel is only partially financed, it generally needs to find contracts with industry, which also helps in finding technical challenges for the consortium to work on. Once an idustrial partner has identified a potential partnership within the project, it can go to the regional government and ask for its own funding.
The consortium partners Multitel, FPMs, UCL, and ULg have complementary expertise. Early on in the 3DMEDIA project, they all listed their existing capabilities and looked for ways to combine them to come up with innovative capabilities. One approach was to identify each partners strengths, and to look for ways in which they could be combined in visual demonstrations, which also proved to be very useful in establishing a dialog with industry.
The different research themes of the 3D MEDIA project can be found at different points along the traditional chain of acquisition, processing, and exploitation/immersion. The project considers advanced acquisition means, such as pairs of stereoscopic cameras, multiple-camera and omni-directional-camera systems, and 3D cameras that can acquire range information directly. The use of several cameras may call for the registration of views, the reconstruction of a 3D scene, and the automatic determination of the position of the cameras. Applications often call for tracking objects and persons, for recognizing them, and for interpreting their motion. Transmission and security calls for compression and for techniques such as watermarking. Mixing real and synthetic data is becoming increasingly important. Match-moving is one technique that allows this to be done automatically. Indexation, annotation, and summarizing of video are important for archiving large amounts of video data. Data mining allows one to quickly retrieve information from large databases of images. Image understanding also plays a role in man-machine interfaces, such as in recognizing gestures and facial expressions. The result of acquisition and processing must ultimately be experienced by the user. Here, we mainly talk about 3D stereo visualization and the creation of immersive acoustical and visual environments.
The application domains of 3D MEDIA are varied, such as 3D cinema, entertainment, sports, and videosurveillance. One concrete example in sports is the automatic tracking of players from multiple cameras for resolving occlusions (TRICTRAC project), which then leads to the automatic analysis of games and, eg, to the possibility of raising an alarm in case of off-side.
As the name indicates, it was judged important by the partners of the consortium that the project have a significant 3D flavour. This is in line with the sudden acceleration of interest worldwide for everything 3D. One of the major factors behind this impetus is the arrival of digital cinema in theatres, and the fact that the jump from 2D to 3D in digital cinema is relatively straightforward, at least in terms of visualization. To meet this exploding interest in 3D, the first author has led the organization of a new event called 3D Stereo MEDIA, which will take place in Liège, Belgium, on 1-3 December 2009, and for which a very successful preview was held on 17 March 2009. A unique feature of this event is the bringing together of artists and engineers who share a passion for 3D acquisition, processing, and stereo visualization. Importantly, the event will feature 3D projection capabilities, not only for movies but, more unusually, for scientific and technical presentations. It is during the March preview of 3D Stereo MEDIA that the Toon avatar was shown for the first time ever in 3D stereo (see figure).
Links:
http://mediatic.multitel.be/platforms/3dmedia.html
http://www.montefiore.ulg.ac.be/
http://www.neurotv.com
http://www.3dmedia2009.com
Please contact:
Jacques G. Verly
University of Liège, Belgium
E-mail: jacques.verly
ulg.ac.be
