ERCIM News 137
May 2024
Special theme Extended Reality
Guest editors Ioannis Chatzigiannakis (Sapienza University of Rome and CNIT), Holger Graf (Fraunhofer IGD); Manos Kamarianakis (University of Crete) and Aris Lalos (ISI)
- Details
- Category: Special Theme
- Hits: 3982
by the guest editors Ioannis Chatzigiannakis (Sapienza University of Rome and CNIT), Holger Graf (Fraunhofer IGD), Manos Kamarianakis (University of Crete), and Aris Lalos (ISI)
Extended Reality (XR) represents a transformative paradigm in the realm of human-computer interaction, blending and fusing physical and digital worlds to create immersive and interactive experiences. XR encompasses a spectrum of technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR), each offering unique capabilities and applications. At its core, XR enables users to engage with digital content in ways that transcend traditional interfaces, opening-up new possibilities for communication, collaboration, and exploration.
- Details
- Category: Special Theme
- Hits: 5515
by Dave Raggett (ERCIM)
The Web grew exponentially on the back of open standards, rapidly eclipsing proprietary alternatives. The same principle will apply to the Immersive Web. This article introduces W3C’s work on related standards and how they fit into the standardisation landscape; key challenges for privacy, accessibility, and scaling; and directions for future work, including Generative AI.
- Details
- Category: Special Theme
- Hits: 3120
by Ioannis Kavouras, Emmanuel Sardis and Anastasios Doulamis (National Technical University of Athens, Greece)
Visualise today the city of tomorrow! The rapid evolution of the computer graphics along with the accessibility in high-capacity 3D geospatial information have led to the emergence of new opportunities for engineers and scientists to better understand, visualise and design the urban environment.
- Details
- Category: Special Theme
- Hits: 3170
by Gianpaolo Palma and Paolo Cignoni (CNR-ISTI)
We introduce a system designed to enhance engagement in VR experiences using sensorised replicas of real objects created through 3D printing. This system lets users interact with physical replicas within the virtual environment while visualising the original object's appearance. Additionally, it facilitates the creation of augmented experiences to manipulate the virtual appearance of the physical replica using personalisation actions, such as painting over the object’s surface or attaching additional virtual objects, taking advantage of its tactile feedback.
- Details
- Category: Special Theme
- Hits: 3790
by Moonisa Ahsan, Irene Viola and Pablo Cesar (CWI)
VOXReality is an ambitious project funded by European Commission, focusing on voice-driven interactions by combining Artificial Intelligence (AI), Natural Language Processing (NLP) and Computer Vision (CV) technologies in the Extended Reality (XR) use cases of VR Conference, AR Theater, and AR Training. We will develop innovative AI models that will combine language as a core interaction medium supported by visual understanding to deliver next-generation XR applications that provide comprehension of users’ goals, surrounding environment and context in mentioned use cases.
- Details
- Category: Special Theme
- Hits: 3539
by Irene Viola, Jack Jansen, and Pablo Cesar (CWI)
Extended Reality (XR) telecommunication systems promise to overcome the limitations of current real-time teleconferencing solutions, by enabling a better sense of immersion and fostering more natural interpersonal interactions. To achieve truly immersive communication, high-fidelity representations of our own bodies and faces are essential. Enter VR2Gather: a customisable end-to-end system to enable multi-party communication with real-time acquisition.
- Details
- Category: Special Theme
- Hits: 3199
by Fabio Carrara (CNR-ISTI)
Immersive and user-friendly experiences will win the Extended Reality (XR) game in the long run. However, setting up good VR/AR scenarios often requires manual asset authoring, which is realistic just when dealing with a limited number of predefined objects and scenes. The Social and hUman ceNtered (SUN) XR project is investigating low-cost, yet effective, solutions to create links between a physical environment and its corresponding one in the virtual world.
- Details
- Category: Special Theme
- Hits: 3092
by Marco Callieri, Daniela Giorgi (CNR-ISTI), Andrea Maggiordomo (University of Milan) and Gianpaolo Palma (CNR-ISTI)
Successful Extended Reality (XR) applications require 3D contents able to provide rich sensory feedback to the users. In the European project SUN, the Visual Computing Lab at CNR-ISTI is investigating novel techniques for 3D asset creation for XR solutions, with 3D objects featuring both accurate appearance and estimated mechanical properties. Our cutting-edge research leverages on Artificial Intelligence (AI), computer graphics, and modern sensing and Computational Fabrication techniques. The application fields include XR-mediated training environments for industries; remote social interaction for psychosocial rehabilitation; and personalised physical rehabilitation.
- Details
- Category: Special Theme
- Hits: 3422
by Marco Di Benedetto, Giulio Federico and Giuseppe Amato (CNR-ISTI)
The power of Generative AI is employed towards a detailed reconstruction of large environments, overcoming short-range and sparsity of common techniques with spatio-temporal 3D diffusion models.
In the rapidly evolving landscape of Extended Reality (XR), the Social and hUman ceNtered (SUN) XR project [L1] emerges as a pioneering endeavour to redefine intuitive 3D data acquisition solutions, promising digitally immersive experiences with unprecedented realism. Spearheaded by a collaborative effort involving esteemed institutions across Europe, SUN aims to set new benchmarks in the creation of physically convincing digital 3D models while facilitating seamless integration with real-world objects.
- Details
- Category: Special Theme
- Hits: 3066
by Alexandre Kabil (Université Paris-Saclay, CNRS, Inria), Ronan Gaugne (Univ Rennes, Inria, CNRS, IRISA), Michel Beaudouin-Lafon (Université Paris-Saclay, CNRS, Inria)
The French National Research Infrastructure CONTINUUM is a unique consortium of 30 platforms located throughout France for advancing interdisciplinary research between computer science, the humanities and social sciences. Through CONTINUUM, 37 research groups develop cutting-edge research focusing on visualisation, immersion, interaction and collaboration, as well as human perception, cognition and behaviour in virtual/augmented reality.
- Details
- Category: Special Theme
- Hits: 3499
by Asterios Leonidis, Maria Korozi and Constantine Stephanidis (FORTH-ICS)
Recent advancements in Extended Reality (XR) technology, have expanded the scope of potential applications, enabling highly immersive solutions that blend digital and physical elements seamlessly. These developments have introduced applications like ARcane Tabletop and CRETE AR, which offer immersive experiences in gaming, historical simulations, and virtual exploration. Additionally, systems like GreenCitiesXR, Research Assistant, and the Digitised Material Preservation Application leverage XR technology to enrich collaboration and creativity.
- Details
- Category: Special Theme
- Hits: 3269
by Michael Prodinger, Rita Stampfl, Marie Deissl-O'Meara (University of Applied Sciences Burgenland)
The innovative and collaborative Extended Reality (XR) learning environment, which empowers flexible and interactive continuing education, is a characteristic of the first distance learning programmes at the FH Burgenland. With their well-organised module structure, the programmes not only give students the freedom to plan their online and asynchronous coursework independently, but also underscore their commitment to offering a contemporary learning experience through the central role of the XR platform MeetYoo. Features of this platform such as its info-point and virtual fireside chats, promote student interaction and networking with experts. The ongoing interaction that follows, coupled with access to resources without time or space constraints reflects the desire to optimise the learning process and offer students exceptional flexibility and support in their academic and professional journey.
- Details
- Category: Special Theme
- Hits: 4149
by Stylianos Mystakidis (University of Patras)
Spatial computing, extended reality (XR) and the Metaverse are emerging technologies with the potential to transform education and training. However, technology-use alone in education without suitable pedagogical strategies and methods cannot guarantee effective learning experiences. Meaningful gamification in the form of escape room games is one viable option to increase sustainable engagement and instil passion in learning.
- Details
- Category: Special Theme
- Hits: 4337
by Filip Škola, Václav Milata and Fotis Liarokapis (CYENS)
This article introduces an intelligent Extended Reality (XR) teaching assistant communicating with the users in natural language. The application uses artificial intelligence (ChatGPT) to understand and communicate with the students using realistic-looking avatars with body language. Teachers can customise the application by adding their slides to be taught or choosing a teaching topic that will be discussed by two intelligent agents.
- Details
- Category: Special Theme
- Hits: 5269
by George Fatouros and John Soldatos (INNOV-ACTS Limited)
The use of Extended Reality (XR) technologies for industrial workers’ training provides immersive and cost-effective learning experiences that improve safety, efficiency, and productivity in the workplace. Nevertheless, most XR-based training applications are based on static and inflexible content, which limits their customisation and personalisation potential. This article introduces a novel approach to developing and deploying XR applications for training industrial workers. The approach leverages Large Language Models (LLMs) and Generative Artificial Intelligence (GenAI) to produce customisable, context-aware, and personalised content that significantly enhances the workers’ training experience.
- Details
- Category: Special Theme
- Hits: 3096
by Kübra Yayan (Eskisehir Osmangazi University), Yunus Emre Esen (LTG), Uğur Yayan (Eskisehir Osmangazi University)
Embarking on an innovative journey, the XR4HRC (Excellence in Extended Reality for Human-Robot Collaboration) project within the XR2Learn initiative redefines vocational training in the robotics sector through Extended Reality (XR) technologies. This endeavour aims to elevate workforce competencies, quality assurance, and human-robot synergy to new heights, perfectly aligning with the progressive vision of Industry 5.0.
- Details
- Category: Special Theme
- Hits: 3232
by Alessandro Pollini, Tania Sabatini and Sara Traversari (BSD Design)
In manufacturing, Human-Line Interactions (HLI) are characterised by three aspects: the operator’s loss of situational awareness due to the opacity of the systems [1], the complexity and interdependency of all the automation functions, and the machine autonomy that always requires to be clearly communicated. The PROXIMA project solves for these aspects by creating an Augmented Reality (AR) Training Kit that supports and enhances human cognition in the act of learning in HLI-based environments.
- Details
- Category: Special Theme
- Hits: 3880
Chatzigiannakis Ioannis (Sapienza University of Rome and CNIT), Giordano Silvia (University of Applied Science of Southern Switzerland - SUPSI), Orphanoudakis Theofanis (Hellenic Open University)
Extended Reality (XR) has become a tangible possibility to revolutionise educational and training scenarios by offering immersive, interactive and personalised experiences. The XR2Learn [L1] project aims to establish a cross-border innovation community for XR in learning, bringing XR technology providers, application designers, education experts, application developers, end-users and decision-makers in direct access to communicate, collaborate and matchmake interests enabling also bottom-up innovation creation.
- Details
- Category: Special Theme
- Hits: 3183
by Valérie Maquil, Lou Schwartz and Adrien Coppens (Luxembourg Institute of Science and Technology)
As part of the ReSurf project running at the Luxembourg Institute of Science and Technology, we are working on the next generation of remote collaboration systems using wall-sized displays. The project aims at studying the collaborative behaviour of team members interacting with such displays to solve decision-making problems. Through our observations, we will develop and assess visual indicators for better supporting awareness of collaborators’ activities when moving to a mixed-presence scenario involving two distant wall-sized displays.
- Details
- Category: Special Theme
- Hits: 5181
by John Soldatos (INNOV-ACTS Limited), Georgios Makridis (University of Piraeus Research Center) and Fotis Liarokapis (CYENS Centre of Excellence)
Extended reality (XR) is a term that covers all real and virtual environments, enabling human-machine interactions through computer-generated content. It includes virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies [1]. State-of-the-art XR applications for manufacturing are built based on a “one-size-fits-all” philosophy that does not consider the characteristics of individual workers. Hence, they fall short when it comes to supporting the emerging wave of Industry 5.0 (I5.0) applications that are destined to be human-centric and emphasise trustworthy human-machine collaboration. I5.0 requires XR visualisations that consider the characteristics, skills, and context of the manufacturing worker [2], along with the peculiarities of their interactions with machinery, automation devices, and cyber physical production systems.
- Details
- Category: Special Theme
- Hits: 3210
by Katharina Hölzle (Fraunhofer Institute for Industrial Engineering IAO and University of Stuttgart) and Matthias Aust and Steffen Braun (Fraunhofer Institute for Industrial Engineering IAO)
The industrial metaverse offers the potential to fundamentally disrupt our understanding of technology-assisted collaboration and therefore requires an interdisciplinary approach across the domains of Extended Reality (XR), AI, ergonomics, and innovation management. Consequently, we develop a unique hard- and software infrastructure for co-located and multi-modal communication-, meeting-, and planning-lab use cases. This will enable different industries to take the first steps into a human-centred metaverse for future product development, engineering, service development, and co-innovation.
- Details
- Category: Special Theme
- Hits: 3544
by Karolina Wylężek, Irene Viola and Pablo Cesar (CWI)
Museums’ objects, often having outlived their owners, continue telling the story of the past and educate us about the history. Sadly, they also do not last forever. Passing time gradually makes them more and more fragile, finally rendering them too vulnerable to be shown on a display ever again. Luckily, new technologies come with solutions to keep relics of history accessible even after their physical “lifetime” ends. At CWI [L1], we decided to develop a social VR system that would allow us to experience and interact with objects long after their time has passed.
- Details
- Category: Special Theme
- Hits: 4210
by Silvia Rossi, Irene Viola (CWI) and Pablo Cesar (CWI and TU Delft)
Thanks to the recent advance of Extended Reality (XR) and Artificial Intelligence (AI) technologies, researchers in the European TRANSMIXR project are working on the future of media experiences by advancing the state-of-the-art in media production, delivery and consumption. In this context, we developed an open-source system for volumetric real-time interaction for social VR, collected user requirements and proposed new metrics to evaluate the quality and user experience in XR environments.
- Details
- Category: Special Theme
- Hits: 3770
by Julien Lomet (Univ Paris 8, Univ Rennes 2, IRISA), Ronan Gaugne (Univ Rennes, Inria, CNRS, IRISA) and Valérie Gouranton (Univ Rennes, INSA Rennes, Inria, CNRS, IRISA)
Collaborative virtual spaces allow to establish rich relations between involved actors in an artistic perspective. Our work contributes to the design and study of artistic collaborative virtual environments through the presentation of immersive and interactive digital artwork installation, named “Creative Harmony”. In this project, we explore different levels of collaborations proposed by the “Creative Harmony” environment, and we question the experience and interactivity of several actors immersed in this environment, through different technological implementations.
- Details
- Category: Special Theme
- Hits: 3681
by Muhannad Ismael, Roderick McCall and Joan Baixauli (Luxembourg Institute of Science and Technology)
Using mixed reality (MR) head-mounted displays (HMDs) for fluid simulations (FS) is challenging. This is due to the capacity in MR-HMDs to perform FS in real time with reasonable frames per second. FSMR (fluid simulation in mixed reality) models position-based fluids via an MR-HMD using remote particles system computing.
- Details
- Category: Special Theme
- Hits: 3016
by Maria di Summa (CNR-STIIMA), Nicola Mosca (CNR-STIIMA), and Angelo Cardellicchio (CNR-STIIMA)
The railway sector has always played a key role in communication systems and today, safety standards within this sector has reached very high levels. However, despite these high standards, the industry's goal remains achieving a completely accident-free status by leveraging resources and technologies.
- Details
- Category: Special Theme
- Hits: 3007
by Gerasimos Arvanitis (University of Patras, AviSense.ai), Aris S. Lalos (ISI, ATHENA R.C.), and Konstantinos Moustakas (University of Patras)
In recent years, Extended Reality (XR) technologies have transformed the interaction between humans and machines, especially in dynamic environments like automotive and transportation. By providing drivers with relevant Augmented Reality (AR) information about their surroundings, we can enhance drivers’ capabilities and potentially improve their behaviour, safety, and decision-making in critical situations within rapidly changing environments. Connected Autonomous Vehicles (CAVs) offer the advantage of intercommunication, creating a network of connected vehicles known as cooperative situational awareness. This connectivity presents numerous opportunities, including the sharing of AR information related to the observed objects-of-interest between vehicles within the system (simultaneous and/or asynchronous). Such information transmission allows one vehicle to relay valuable observations to another, enhancing overall system awareness and efficiency.
- Details
- Category: Special Theme
- Hits: 3740
by Anderson Maciel (Técnico Lisboa), Catarina Moreira (University of Technology Sydney), and Joaquim Jorge (Técnico Lisboa)
Desktop-based virtual colonoscopy has been proven to be an asset in identifying colon anomalies. The process is accurate, although time consuming. In this article, we present a new design exploring elements of the VR paradigm to make the immersive analysis more efficient while still effective.