ERCIM News 136
January 2024
Special theme Large Language Models
Guest editors Diego Collarana Vargas (Fraunhofer FIT) and Nassos Katsamanis (Athena RC)
Large Language Models - Introduction to the Special Theme
- Details
- Category: Special Theme
- Hits: 5057
by the guest editors Diego Collarana Vargas (Fraunhofer FIT) and Nassos Katsamanis (Athena RC)
Natural Language Processing (NLP) has witnessed significant advancements in recent years, with a focus on the development of Large Language Models (LLMs) capable of processing and generating human-like text. These models have shown tremendous potential in various applications, including ChatGPT, ranging from content creation and customer service to more sensitive areas such as mental health support and education. However, integrating LLMs into real-world settings presents challenges related to size, energy consumption, knowledge management, ethical concerns, interpretability, and governance.
Role-Playing Simulation Games using ChatGPT
- Details
- Category: Special Theme
- Hits: 7232
by Rita Stampfl (University of Applied Sciences Burgenland), Igor Ivkić (University of Applied Sciences Burgenland and Lancaster University) and Barbara Geyer (University of Applied Sciences Burgenland)
Since the COVID-19 pandemic, educational institutions have embarked on digital transformation projects. The success of these projects depends on integrating new technologies and understanding the needs of digitally literate students. The “learning by doing” approach suggests that real success in learning new skills is achieved when students can try out and practise these skills. In this article, we demonstrate how Large Language Models (LLMs) can enhance the quality of teaching by using ChatGPT in a role-playing simulation game scenario to promote active learning. Moreover, we discuss how LLMs can boost students’ interest in learning by allowing them to practice real-life scenarios using ChatGPT.
ChatGPT as a Learning Assistant in Distance Learning
- Details
- Category: Special Theme
- Hits: 8658
by Michael Prodinger, Rita Stampfl and Marie Deissl-O'Meara (University of Applied Sciences Burgenland)
The following article deals with the implementation of a Learning Assistant, an advanced tool based on artificial intelligence (AI) that provides continuous learning support to students. The assistant is used in distance learning programmes at FH Burgenland Continuing Education. With ChatGPT installed and integrated into the Learning Management System (LMS), it functions as an assistant with the ability to answer questions from students whenever they arise. Additionally, a course teacher verifies and corrects the AI’s responses within 24 hours to guarantee the system’s correctness and dependability.
Using LLMs in a Language Teaching and Learning Application
- Details
- Category: Special Theme
- Hits: 4533
by Ioanna Antoniou-Kritikou, Voula Giouli, George Tsoulouhas and Constandina Economou (ATHENA Research Centre)
Our research is aimed at employing Large Language Models (LLMs) to build a multi-dimensional paraphrase tool in view of enhancing reading comprehension and writing skills of learners of Greek as mother tongue, second or foreign language (L1, L2/FL). The idea is to integrate ChatGPT into classroom settings, thus turning Generative AI from a threat to an assistant.
Chatbots & Socrates: Dialogues in Learning
- Details
- Category: Special Theme
- Hits: 8992
by Barbara Geyer, Rita Stampfl and Elisabeth Hauser (University of Applied Sciences Burgenland)
It is becoming increasingly important to integrate educational chatbots, particularly those that use Socratic methods. The development of Large Language Models such as GPT-4 have made it possible to create more nuanced and interactive conversations, enabling chatbots to support more complex learning processes and provide personalised learning experiences. Two examples of chatbots that use Socratic methods to help students understand and learn the Scrum framework are the Scrum Tutor and the Scrum Assistant. These tutors were developed using GPTs and were evaluated through controlled testing and user feedback to ensure technical functionality and didactic effectiveness. The Socratic teaching method encourages critical thinking and self-reflection among learners, making it an effective approach for developing educational chatbots that actively involve students in the learning process.
A Conversational AI Assistant for Teaching and Learning
- Details
- Category: Special Theme
- Hits: 6033
by Spyridoula Stamouli, George Paraskevopoulos and Nassos Katsamanis (Athena Research Center)
The AI4EDU project develops an innovative approach to incorporating AI in education, featuring conversational tools like “Study Buddy” and “Teacher Workmate”. This initiative is reshaping learning and teaching, emphasising ethical and inclusive AI practices, and is bringing together education and technology experts to enrich educational experiences.
Using LLMs in Professional Training: Criminal Investigators Improve their Skills in Children’s Interrogations
- Details
- Category: Special Theme
- Hits: 4276
by Don Tuggener (Zurich University of Applied Sciences) and Susanna Niehaus (Lucerne University of Applied Sciences and Arts)
How do you teach a knowledge-based chatbot to reveal its information only conditionally? An unusual setting in the age of information and chatty dialogue systems – and precisely the premise of the Virtual Kids project.
Unveiling Ethical Biases in Generative AI
- Details
- Category: Special Theme
- Hits: 3621
by Sergio Morales (Universitat Oberta de Catalunya), Robert Clarisó (Universitat Oberta de Catalunya) and Jordi Cabot (Luxembourg Institute of Science and Technology)
Generative AI models are widely used for generating documents, videos, images, and so on; however, they can exhibit ethical biases that could be harmful or offensive. To prevent this, we propose a framework to test the fairness of generative AI before integrating such models in your daily work.
Exciting Opportunities and Necessary Safeguards for Large Language Models in the Public Sector
- Details
- Category: Special Theme
- Hits: 3764
by Peter Biegelbauer, Alexander Schindler, Rodrigo Conde-Jimenez, and Pia Weinlinger (AIT Austrian Institute of Technology)
Large Language Models (LLMs) have the potential to support the civil service. They can be used to automate tasks such as document classification, summarisation, and translation, among others. However, there are also risks and challenges associated with their use.
What Do LLMs Know about Deception in Text?
- Details
- Category: Special Theme
- Hits: 5367
by Katerina Papantoniou, Panagiotis Papadakos and Dimitris Plexousakis (ICS-FORTH)
At ICS-FORTH, we explore what LLMs know regarding the task of verbal deception detection. We evaluate the performance of two well-known LLMs and compare them with a fine-tuned BERT-based model. Finally, we explore whether the LLMs are aware of culture-specific linguistic deception detection cues.
Towards AI-Assisted Data Storytelling
- Details
- Category: Special Theme
- Hits: 4990
by Angelica Lo Duca (CNR-IIT)
At CNR-IIT we are investigating if generative AI can be used to improve Data Storytelling and generate more engaging and informative data stories.
VOCTRACTOR: AI-Powered Vocabulary Design and Keyword Extraction Tool
- Details
- Category: Special Theme
- Hits: 4366
by Aradina Chettakattu and Denis Havlik (AIT Austrian Institute of Technology GmbH)
Building and using a common terminology is of great importance for all collaborative work. On the other hand, well implemented tagging of documents by several “orthogonal” vocabularies provides a concise representation of context and topics of a textual data, which facilitates search and retrieval, as well as automated matching of “similar” documents in knowledge management systems. Manually finding relevant keywords for documents is a daunting task, time-consuming and prone to errors. Our “Voctractor” (from “Vocabulary Extractor”) application prototype [L1] addresses this challenge by streamlining both the vocabulary design and the keyword extraction workflow.
From Chats to Models: Assessing the Potential of LLMs in the World of Software Modeling
- Details
- Category: Special Theme
- Hits: 3720
by Javier Cámara, Javier Troya and Lola Burgueño (ITIS Software / Universidad de Málaga)
There is a growing body of work assessing the capabilities of Large Language Models (LLMs) for writing code. Comparatively, the analysis of the current state of LLMs with respect to software modeling has received little attention. Are LLMs capable of generating useful software models? What factors determine the quality of such models? The work we are conducting at ITIS-UMA investigates the capabilities and main shortcomings of current LLMs in software modeling.
Enhancing Conversational Search with Large Language Models
- Details
- Category: Special Theme
- Hits: 4397
by Guido Rocchietti, Cristina Ioana Muntean, Franco Maria Nardini (CNR-ISTI)
In the context of the Horizon Europe EFRA research project [L1], we explore the innovative use of Large Language Models (LLMs), both instructed and fine-tuned, in improving the quality of conversational search. The focus is on applying these models in rewriting conversational utterances to enhance the capability of conversational agents to retrieve accurate responses.
Knowledge Graph Treatments for Hallucinating Large Language Models
- Details
- Category: Special Theme
- Hits: 6068
by Diego Collarana, Moritz Busch, and Christoph Lange (Fraunhofer FIT)
Despite the excitement about Large Language Models (LLMs), they still fail in unpredictable ways in knowledge-intensive tasks. In this article, we explore the integration of LLMs with Knowledge Graphs (KGs) to develop cognitive conversational assistants with improved accuracy. To address the current challenges of LLMs, such as hallucination, updateability and provenance, we propose a layered solution that leverages the structured, factual data of KGs alongside the generative capabilities of LLMs. The outlined strategy includes constructing domain-specific KGs, interfacing them with LLMs for specialised tasks, integrating them with enterprise information systems and processes, and adding guardrails to validate their output, thereby presenting a comprehensive framework for deploying more reliable and context-aware AI applications in various industries.
Revolutionising Theatre Archives: Using Large Language Models to Interact with Structured Archival Content
- Details
- Category: Special Theme
- Hits: 3961
by Chara Tsoukala, Georgios Paraskevopoulos and Athanasios Katsamanis (Athena Research Center)
The Greek National Theatre has introduced an advanced chatbot based on Large Language Models (LLMs) as a novel means of accessing the content of the digital archive on the web. This state-of-the-art chatbot, which utilises Text2SQL conversion through LLMs, offers a more intuitive user experience, enabling complex searches with simple natural language. This addition marks a significant step forward in making theatrical performance data accessible in a more interactive way.
Improving the Sample Efficiency of Pre-training Language Models
- Details
- Category: Special Theme
- Hits: 4130
by Gábor Berend (University of Szeged)
The use of transformer-based pretrained language models (PLMs) arguably dominates the natural language processing (NLP) landscape. The pre-training of such models, however, is known to be notoriously data and resource hungry, which hinders their creation in low-resource settings, making it a privilege to those few (mostly corporate) actors, who have access to sufficient computational resource and/or pre-training data. The main goal of our research is to develop such a novel sample-efficient pre-training paradigm of PLMs, which makes their use available in the low data and/or computational budget regime, helping the democratisation of this disruptive technology beyond the current status quo.
ChatGPT Multilingual Querying Consistency – A Test Case
- Details
- Category: Special Theme
- Hits: 2893
by George Tambouratzis (Athena Research Centre)
Conversational agents and chatbots have developed rapidly in the past year to provide answers to user queries, drawing information from huge collections of data. From the user-side, the usefulness of conversational agents hinges on the accuracy of response in addition to user-friendliness and response speed. Here we briefly evaluate one of the most widely used chatbots, ChatGPT over a set of queries posed using multiple languages, to test its robustness and consistency, while running the experiment at two timepoints to monitor ChatGPT’'s evolution.
A Pipeline for Validating ChatGPT Responses Using Knowledge Graphs and Embeddings
- Details
- Category: Special Theme
- Hits: 4030
by Michalis Mountantonakis and Yannis Tzitzikas (FORTH-ICS and University of Crete)
Since it is challenging to combine ChatGPT (which has been trained by using data from web sources), with popular RDF Knowledge Graphs (that include high quality information), we present a generic pipeline that exploits RDF Knowledge Graphs and short sentence embeddings for enabling the validation of ChatGPT responses, and an evaluation by using a benchmark containing 2,000 facts for popular Greek persons, places and events.
A Unified Model for Automated Evaluation of Text Generation Systems
- Details
- Category: Special Theme
- Hits: 4216
by Jan Deriu and Mark Cieliebak (Zurich University of Applied Sciences)
With all the recent hype around Large Language Models and ChatGPT in particular, one crucial question is still unanswered: how do we evaluate generated text, and how can this be automated? In this SNF project, we develop a theoretical framework to answer these questions.
TERMINET: Pioneering the Future of Smart Interconnected IoT
- Details
- Category: Research and Innovation
- Hits: 9454
by Peter Kunz (ERCIM)
January 2024 marks the conclusion of the ambitious European H2020 project, TERMINET, which envisions a revolutionary next-generation IoT architecture. Combining state-of-the-art technologies like SDN, multiple-access edge computing, and virtualization, TERMINET aims to transform the IoT landscape with intelligent devices tailored for low-latency, market-driven use cases.
Tailoring Decarbonisation and Resilience Strategies to Drive Regional and Local Action
- Details
- Category: Research and Innovation
- Hits: 3152
by Christiane Walter (PIK), Luis Costa (PIK) and Sara Dorato (T6)
LOCALISED is a four- year H2020-funded research project (October 2021 – September 2025) that develops, in a co-design process, tailored end-user products and services for local and regional policy-makers, administrations, businesses and citizens, to speed up sub-national decarbonisation processes while considering climate risks and adaptation needs. The flagship outputs are the Decarbonisation Profiler and the Net-Zero Business Consultant, providing information for all NUTS3-regions in Europe, currently under development.
Taranis AI: Applying Natural Language Processing for Advanced Open-Source Intelligence Analysis
- Details
- Category: Research and Innovation
- Hits: 8440
by Florian Skopik and Benjamin Akhras (Austrian Institute of Technology)
Open-source intelligence (OSINT) provides up-to-date information about new cyber-attack techniques, attacker groups, changes in IT products, updates of policies, recent security events and much more. Often dozens of analysts search a multitude of sources and collect, categorise, cluster, and rank news items from the clear and dark web in order to prepare the most relevant information for decision makers. A tool that supports this job is “Taranis NG” from the Slovakian CERT. This solution ingests information from many types of sources such as websites, RSS feeds, emails and social media channels and makes them searchable. It also supports the creation of reports and daily summaries. However, the number of sources and news items is continuously growing, making it increasingly difficult to search them purely manually. These circumstances call for the application of novel natural language processing (NLP) methods to make OSINT analysis more efficient.
From Cultural Heritage Assets to Cultural Linked Data – the Case of the Archaeological Museum of Messara
- Details
- Category: Research and Innovation
- Hits: 3406
by Dimitris Angelakis, Lida Charami, Pavlos Fafalios and Christos Georgis (FORTH-ICS)
The new Archaeological Museum of Messara, which opened its doors to visitors on 22 April 2023, is entirely dedicated to the antiquities of the Messara region in Heraklion, Greece. The Centre for Cultural Informatics of FORTH-ICS has undertaken the task to provide a comprehensive data management solution for the scientific and administrative documentation, research and promotion of the new museum’s important assets.
Controllable Artificial Intelligence
- Details
- Category: Research and Innovation
- Hits: 5414
by Peter Kieseberg, Simon Tjoa (St. Pölten UAS) and Andreas Holzinger (University of Natural Resources and Life Sciences)
The burgeoning landscape of AI legislation and the ubiquitous integration of machine learning into daily computing underscore the imperative for trustworthy AI. Yet, prevailing definitions of this concept often dwell in the realm of the abstract, imposing robust demands for explainability. In light of this, we propose a novel paradigm that mirrors the strategies employed in navigating the opaqueness of human decision-making. This approach offers a pragmatic and relatable pathway to cultivating trust in AI systems, potentially revolutionising our interaction with these transformative technologies.
Report on the 4th Joint JST/ERCIM Workshop
- Details
- Category: Joint ERCIM Actions
- Hits: 7549
by Katsumi Emura (Fukushima Institute for Research, Education and Innovation) and Dimitris Plexousakis (FORTH-ICS)
This article provides a brief report on the 4th Workshop jointly organized by ERCIM and the Japan Science and Technology Agency (JST). The workshop, themed “Exploring New Research Challenges and Collaborations in Artificial Intelligence, Big Data, Human-Computer Interaction, and the Internet of Things,” took place in October 2023 in Kyoto, Japan. Hosted by JST as part of the Advanced Integrated Intelligence Platform project, the event offered European and Japanese participants an opportunity to report on recent research results in the aforementioned areas and to explore collaboration prospects within the framework of European Commission programs or corresponding initiatives of JST.
Diversity Matters - INESC TEC's Diversity and Inclusion Commission
- Details
- Category: Joint ERCIM Actions
- Hits: 3450
INESC TEC's Diversity and Inclusion Commission (D&IC), led by Ana Sequeira. In this interview, Ana Sequeira highlights D&IC’s pioneering initiatives in promoting gender equality, supporting disabilities, and fostering intercultural understanding within the research institute.
Dagstuhl Seminars and Perspectives Workshops
- Details
- Category: Announcements
- Hits: 2672
Call for Proposals
Schloss Dagstuhl – Leibniz-Zentrum für Informatik is accepting proposals for scientific seminars/workshops in all areas of computer science, in particular also in connection with other fields.
CWI Received Two Test of Time Awards
- Details
- Category: Announcements
- Hits: 3190
Marten van Dijk, head of CWI’s Computer Security group, won the ACM CCS 2023 Test-of-Time Award, and members of CWI’s Database Architectures group won the 2024 CIDR Test of Time Award.
43rd SAFECOMP 2024, 19th DECSoS Workshop - Call for Papers
- Details
- Category: Announcements
- Hits: 3911
Florence, Italy, 17-20 September 2024
The renowned international SAFECOMP Conference will be held this year in Florence, Italy, from September 17-20, with the first day reserved for several parallel workshops (Workshop Day). The key theme is Safety in a cyber-physical interconnected world.
Interdisciplinary Information Management Talks - IDIMT 2024 - Call for Papers
- Details
- Category: Announcements
- Hits: 3812
Hradec Kralove, Czech Republic, 4-6 September 2024
With over 30 years of history, IDIMT conferences have established themselves as an interdisciplinary international forum for exchanging concepts and visions in the areas of software-intensive systems, management and engineering of information and knowledge, social media, business engineering, and related topics. The conference has been organized since its inception by the University of Economics and Business in Prague and the Johannes Kepler University in Linz, Austria. The proceedings are published by Trauner Verlag, Linz, in the University Edition, Informatics series. Papers are peer-reviewed and indexed by Scopus and Web of Science. Past proceedings can be found at https://idimt.org/proceedings/.