by Katharina Hölzle (Fraunhofer Institute for Industrial Engineering IAO and University of Stuttgart) and Matthias Aust and Steffen Braun (Fraunhofer Institute for Industrial Engineering IAO)
The industrial metaverse offers the potential to fundamentally disrupt our understanding of technology-assisted collaboration and therefore requires an interdisciplinary approach across the domains of Extended Reality (XR), AI, ergonomics, and innovation management. Consequently, we develop a unique hard- and software infrastructure for co-located and multi-modal communication-, meeting-, and planning-lab use cases. This will enable different industries to take the first steps into a human-centred metaverse for future product development, engineering, service development, and co-innovation.
The significance of virtual or, better, extended environments is nothing new. Previous contributions in this field have addressed physical aftereffects and the sense of presence in virtual environments decades ago and formulated a strategic research and development agenda [1]. Nevertheless, the technological progress over the past 30 months since the announcement of ‘the’ metaverse by one specific company has created a new hype cycle with a significant expected impact on several industries. Metaverse markets are predicted to reach volumes of up to USD 394 billion by 2025 and USD 5 trillion by 2030 [2] [L1]. A scientific stakeholder analysis in 2023 has identified over 700 companies and institutions working on metaverse-relevant technologies and applications on the state level of Baden-Württemberg alone (ibid.). Consequently, companies and researchers alike are trying out various architectures, applications, and use cases for the industrial metaverse.
Based on these premises, an interdisciplinary research team at Fraunhofer IAO has started the applied research project INSTANCE (Immersive iNduSTriAl iNnovation eCosystEms) to develop, prototype and transfer an XR infrastructure within and between different locations of the research institute, focusing on collaboration in new product development and engineering. The key research question is how metaverse technologies will transform these fields and, at the same time, open up new fields for future industries.
Motivation for Immersive Industrial Innovation Ecosystems (INSTANCE)
The concept of the industrial metaverse continues to evolve rapidly, as numerous industry applications are emerging and the scientific discourse is just starting to gain momentum (e.g. [3]). We view the industrial metaverse as a vision of an immersive environment at the interface between reality and virtuality, in which human and artificial avatars collaborate virtually, economically, socially, and creatively [2] [L1]. It can be characterised by persistence, ubiquity, scalability (cf. cross-instance identity, connectivity, infrastructure), immersion (cf. XR, sensor technology), interoperability (cf. open standards, decentralisation), and value-add (cf. digital innovations). Its key features involve the seamless integration of digital twins as well as AI solutions, driving innovations across the value chain – e.g. in open foresight, new product development, training and development, or predictive maintenance. To this end, substantial challenges must be resolved, such as ensuring its users’ digital competencies and providing an adequate infrastructure based on open standards to allow for interoperability.
We argue that the innovation-driven, knowledge-intensive work in an industrial metaverse is best captured by the notion of an innovation ecosystem – that is, the industrial metaverse will be shaped by complementary contributions of various actors toward collaborative value creation. Notably, this perspective transcends geographical and organisational boundaries, differentiating it from today’s mostly isolated digital twins or XR solutions. Considering this, the industrial metaverse offers the potential to transform our understanding and the implementation of the human-machine interface in industrial contexts and, consequently, requires an interdisciplinary approach across the domains of XR, AI, ergonomics, and innovation management.
Research Approach and Methodology
INSTANCE brings together expertise and experience from various fields, including human factors engineering, design, innovation research, generative AI, immersive technologies (such as virtual and augmented reality), urban systems research, and corporate culture and transformation research. We approach our research questions through two parallel streams, each focusing on a different aspect: (i) infrastructure and (ii) application, both with the goal of building prototypes.
In the infrastructure stream (i), we analyse hardware and software components. We conduct market analyses to determine if available metaverse software platforms meet our requirements, including the use of open standards, open source, independence from proprietary systems, decentralised network architecture, data security, and the ability to host our own prototypical system. The required hardware infrastructure extends beyond networking infrastructure. Leveraging decades of experience in large Virtual Reality Systems, such as Powerwalls and CAVEs, we will extend existing infrastructure by updating systems, and equipping more labs with Powerwalls and Head Mounted Displays. The envisioned result is a network of "connected XR portals" into an innovation ecosystem distributed across multiple locations across Germany (see Figure 1).
![Figure 1: The CAVE in Fraunhofer IAO's Immersive Participation Lab [L2] will be updated with new projection- and computing hardware and become one of the main INSTANCE portals.](/images/stories/EN137/hoelzle.jpg)
Figure 1: The CAVE in Fraunhofer IAO's Immersive Participation Lab [L2] will be updated with new projection- and computing hardware and become one of the main INSTANCE portals.
The application stream (ii) intersects with the infrastructure stream through the user-interface hardware. The portals in different locations are operated by different research teams, each with their own research focus. Multiple metaverse applications will be conceptualised and implemented as prototypes. Trainings will be offered, and experiments run to learn in-depth how humans interact in and with metaverse technologies, which skills are needed, how organisational structure and processes are impacted, and what new forms of technology-assisted collaboration will look like. Examples include AI-assisted innovation workshops, experiments enhancing creativity through virtual surroundings, and facilitation of advanced systems engineering. These applications range from sandbox systems for researchers to customer-ready virtual engineering systems.
Outlook on Future Research Towards Neuroscience-Based Evaluation
Our general approach involves progressing from experimentation to scientific evaluation and optimisation. Each step will provide valuable experiences and best practices that customers and research partners will be able to benefit from.
Based on the experiences enabled by pilot applications using the INSTANCE infrastructure, it is planned to focus on scientific evaluation of the quality of XR sessions using both quantitative and qualitative methods. For quantitative metrics, we will mainly use data-based methodologies of our brain-computer-interaction scientists; for qualitative metrics, we will use AI-based conversation transcription and evaluation. Based on established user-experience methods, we will then be able to measure and systematise different XR configurations and help to find best-fitting solutions for specific use cases.
Applied research at Fraunhofer IAO aims to translate scientific discoveries into real-world applications to benefit industry and society. INSTANCE will be a sandbox for many industries towards exploiting metaverse potentials with humans first and second.
Link:
[L1] https://publica.fraunhofer.de/entities/publication/2344f406-3e01-42e7-a318-c07a4dc43bd5/details
References:
[1] K. Stanney and G. Salvendy, “Aftereffects and sense of presence in virtual environments: formulation of a research and development agenda,” International Journal of Human-Computer Interaction, 1998. https://doi.org/10.1207/s15327590ijhc1002_3
[2] K. Hölzle, et al., “CyberLänd. Potenziale des Metaverse für Unternehmen in Baden-Württemberg,” Fraunhofer IAO, Fraunhofer IPA, VDC Fellbach, 2023. https://doi.org/10.24406/publica-2135
[3] K.-B. Ooi, et al., “The metaverse in engineering management: overview, opportunities, challenges, and future research agenda,” IEEE Transactions on Engineering Management, 2023. https://doi.org/10.1109/TEM.2023.3307562
Please contact:
Katharina Hölzle and Matthias Aust, Fraunhofer Institute for Industrial Engineering IAO, Stuttgart, Germany
