by Marina Toumpouri, Václav Milata, and Fotis Liarokapis (CYENS - Centre of Excellence)
AI-powered chatbots and virtual guides are transforming cultural heritage, making it more interactive, immersive, and accessible. Leading this transformation, the EU-funded XR4ED project is driving innovation in XR-based education and training. A key outcome of the project is the Intelligent XR Assistant, an AI-driven system enhancing user interaction, adaptive learning, and real-time engagement in XR environments. As part of this effort, an AI-powered guide for Nicosia brings its historic centre to life with AI avatars. By leveraging advanced XR and AI technologies, the assistant transforms the way cultural heritage is experienced.
The emergence of artificial intelligence (AI) has significantly transformed the way cultural heritage is perceived, experienced, and consumed, as it has the capacity to enhance visitor experiences through personalised, immersive, and interactive technologies [1]. Notably, AI-powered chatbots and virtual guides can provide real-time, multilingual information about historical sites, museums, and landmarks. AI driven augmented and virtual reality (AR/VR) facilitate immersive experiences of historical environments, including lost architectural heritage, thus providing deeper insights into past civilisations and cultures. Complementing this, AI-driven recommendation systems can tailor itineraries based on visitors’ age, preferences, educational level, and other factors, thereby enriching their interaction and engagement with cultural heritage. Furthermore, machine-learning algorithms can contribute to the analysis and digital preservation of artefacts, facilitating the creation of virtual exhibitions and high-quality 3D models that can be shared globally. By leveraging AI, these advancements make cultural heritage tourism more engaging and educational, while effectively bridging the past with modern technology.
XR4ED [L1] is a project funded by the European Union, designed to foster collaboration between the EdTech and XR (Extended Reality) communities, with the objective of establishing a centralised European platform for XR-based learning and training. This initiative seeks to enhance digital education by providing access to XR content, tools, and solutions, thereby supporting start-ups, SMEs, and educational institutions in developing market-ready XR applications. In alignment with this objective, XR4ED encompasses key components, such as an open marketplace for XR educational tools, an Ethics Observatory to ensure compliance with EU privacy and inclusiveness standards, and a sustainable ecosystem that fosters collaboration across diverse sectors. The project is coordinated by CYENS Centre of Excellence [L2] in Cyprus and involves partners from eight European countries. Among the deliverables of XR4ED is the Intelligent XR Assistant, an AI-driven system designed to enhance user interaction with XR learning environments by providing personalized guidance, adaptive content recommendations, and real-time support for both educators and learners.
Intelligent XR Assistant: Bridging AI and XR
The Intelligent XR Assistant integrates, not only XR but also computer animation and AI to develop highly interactive and immersive learning environments for various application domains, including cultural heritage. As part of this endeavour, an AI-powered guide for Nicosia is introduced, offering an enriched exploration of the city’s historic centre, which is enclosed by the Venetian Walls (Figure 1). Built by the Venetians in the 16th century to replace the existing Frankish fortifications, it stands as the city’s most significant, recognisable, and well-preserved cultural heritage asset. Moreover, Nicosia’s historic centre is home to a wealth of cultural heritage treasures, including historical sites and monuments, as well as significant archaeological, archival, and fine art collections preserved in museums and private institutions.
Figure 1: Intelligent XR Assistants while guiding visitors at the Venetian Walls, in the old city of Nicosia.
The XR Assistant features realistic avatars created with Character Creator 4, enhanced by Ready Player Me’s animation library [L3] to provide natural body language, including gestures, facial expressions, and subtle movements while speaking and when idle. Communication is facilitated through Azure Cognitive Services [L4], enabling multilingual speech-to-text and text-to-speech capabilities, with viseme generation ensuring accurate lip-syncing across different languages [3]. At its core, the assistant integrates OpenAI’s ChatGPT [L5] API to process user inputs and generate contextually relevant responses, allowing dynamic interactions during presentations. Additionally, the system supports a dual-agent mode, where an AI-driven avatar engages in discussions with users on selected topics, offering information, insightful analysis, and complementary viewpoints or contrasting interpretations based on the context [2]. The avatar is depicted in Figure 1.
Built using Unity game engine (version 2022.3.1f1) and Vuforia Engine 10.25 for XR functionalities, the assistant is compatible with both Magic Leap 2 [L6] AR head-mounted displays and Android devices, ensuring accessibility across various platforms [2]. To enhance immersion, the assistant features a panoramic image mode, enabling the on-screen display of contextual backgrounds relevant to the educational content. This capability is particularly valuable in the case of historical sites and monuments, as it facilitates exploration and engagement with locations or environments tied to specific historical contexts, as shown in Figure 2.
Figure 2: Panoramic view of the Intelligent XR Assistants in the old city of Nicosia. In the background is the Church of Panagia Faneromeni (the Revealed One in Greek), built in 1872 to replace an older ecclesiastical structure that existed on the same site.
The system’s architecture also supports collaborative learning, allowing multiple users to connect, interact, and engage with the content simultaneously, fostering a shared educational experience [2]. Overall, the Intelligent XR Assistant leverages cutting-edge XR and AI technologies to provide a customizable, interactive, and immersive educational platform, aiming to transform traditional learning methodologies.
This work has received funding from the European Union’s H2020 Programme under Grant Agreement No 739578 (RISE Project) and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. It has also received funding from the European Union under Grant Agreement No 101093159 (XR4ED Project).
Links:
[L1] https://xr4ed.eu/
[L2] https://ex.cyens.org.cy/
[L3] https://docs.readyplayer.me/ready-player-me/integration-guides/unity/animations/ready-player-me-animation-library
[L4] https://azure.microsoft.com/en-us/products/ai-services
[L5] https://chat.openai.com/
[L6] https://www.magicleap.com/magic-leap-2
References:
[1] S. Boiano et al., “Chatbots and new audience opportunities for museums and heritage organisations,” in Electronic Visualisation and the Arts (EVA) 2018, pp. 164–171. https://doi.org/10.14236/ewic/EVA2018.33
[2] F. Liarokapis, V. Milata, and F. Škola, “Extended reality educational system with virtual teacher interaction for enhanced learning,” Multimodal Technologies and Interaction, vol. 8, no. 9, p. 83, 2024. https://doi.org/10.3390/mti8090083
[3] F. Škola, V. Milata, and F. Liarokapis, “Extended reality intelligent assistant for education,” ERCIM News, no. 137, 13 May 2024. [ISSN: 0926-4981].
Please contact:
Fotis Liarokapis, CYENS – Centre of Excellence, Nicosia, Cyprus
