ERCIM News 136
January 2024
Special theme Large Language Models
Guest editors Diego Collarana Vargas (Fraunhofer FIT) and Nassos Katsamanis (Athena RC)
- Details
- Category: Special Theme
- Hits: 5068
by the guest editors Diego Collarana Vargas (Fraunhofer FIT) and Nassos Katsamanis (Athena RC)
Natural Language Processing (NLP) has witnessed significant advancements in recent years, with a focus on the development of Large Language Models (LLMs) capable of processing and generating human-like text. These models have shown tremendous potential in various applications, including ChatGPT, ranging from content creation and customer service to more sensitive areas such as mental health support and education. However, integrating LLMs into real-world settings presents challenges related to size, energy consumption, knowledge management, ethical concerns, interpretability, and governance.
- Details
- Category: Special Theme
- Hits: 7268
by Rita Stampfl (University of Applied Sciences Burgenland), Igor Ivkić (University of Applied Sciences Burgenland and Lancaster University) and Barbara Geyer (University of Applied Sciences Burgenland)
Since the COVID-19 pandemic, educational institutions have embarked on digital transformation projects. The success of these projects depends on integrating new technologies and understanding the needs of digitally literate students. The “learning by doing” approach suggests that real success in learning new skills is achieved when students can try out and practise these skills. In this article, we demonstrate how Large Language Models (LLMs) can enhance the quality of teaching by using ChatGPT in a role-playing simulation game scenario to promote active learning. Moreover, we discuss how LLMs can boost students’ interest in learning by allowing them to practice real-life scenarios using ChatGPT.
- Details
- Category: Special Theme
- Hits: 8685
by Michael Prodinger, Rita Stampfl and Marie Deissl-O'Meara (University of Applied Sciences Burgenland)
The following article deals with the implementation of a Learning Assistant, an advanced tool based on artificial intelligence (AI) that provides continuous learning support to students. The assistant is used in distance learning programmes at FH Burgenland Continuing Education. With ChatGPT installed and integrated into the Learning Management System (LMS), it functions as an assistant with the ability to answer questions from students whenever they arise. Additionally, a course teacher verifies and corrects the AI’s responses within 24 hours to guarantee the system’s correctness and dependability.
- Details
- Category: Special Theme
- Hits: 4558
by Ioanna Antoniou-Kritikou, Voula Giouli, George Tsoulouhas and Constandina Economou (ATHENA Research Centre)
Our research is aimed at employing Large Language Models (LLMs) to build a multi-dimensional paraphrase tool in view of enhancing reading comprehension and writing skills of learners of Greek as mother tongue, second or foreign language (L1, L2/FL). The idea is to integrate ChatGPT into classroom settings, thus turning Generative AI from a threat to an assistant.
- Details
- Category: Special Theme
- Hits: 9028
by Barbara Geyer, Rita Stampfl and Elisabeth Hauser (University of Applied Sciences Burgenland)
It is becoming increasingly important to integrate educational chatbots, particularly those that use Socratic methods. The development of Large Language Models such as GPT-4 have made it possible to create more nuanced and interactive conversations, enabling chatbots to support more complex learning processes and provide personalised learning experiences. Two examples of chatbots that use Socratic methods to help students understand and learn the Scrum framework are the Scrum Tutor and the Scrum Assistant. These tutors were developed using GPTs and were evaluated through controlled testing and user feedback to ensure technical functionality and didactic effectiveness. The Socratic teaching method encourages critical thinking and self-reflection among learners, making it an effective approach for developing educational chatbots that actively involve students in the learning process.
- Details
- Category: Special Theme
- Hits: 8361
by Spyridoula Stamouli, George Paraskevopoulos and Nassos Katsamanis (Athena Research Center)
The AI4EDU project develops an innovative approach to incorporating AI in education, featuring conversational tools like “Study Buddy” and “Teacher Workmate”. This initiative is reshaping learning and teaching, emphasising ethical and inclusive AI practices, and is bringing together education and technology experts to enrich educational experiences.
- Details
- Category: Special Theme
- Hits: 4292
by Don Tuggener (Zurich University of Applied Sciences) and Susanna Niehaus (Lucerne University of Applied Sciences and Arts)
How do you teach a knowledge-based chatbot to reveal its information only conditionally? An unusual setting in the age of information and chatty dialogue systems – and precisely the premise of the Virtual Kids project.
- Details
- Category: Special Theme
- Hits: 3636
by Sergio Morales (Universitat Oberta de Catalunya), Robert Clarisó (Universitat Oberta de Catalunya) and Jordi Cabot (Luxembourg Institute of Science and Technology)
Generative AI models are widely used for generating documents, videos, images, and so on; however, they can exhibit ethical biases that could be harmful or offensive. To prevent this, we propose a framework to test the fairness of generative AI before integrating such models in your daily work.
- Details
- Category: Special Theme
- Hits: 3782
by Peter Biegelbauer, Alexander Schindler, Rodrigo Conde-Jimenez, and Pia Weinlinger (AIT Austrian Institute of Technology)
Large Language Models (LLMs) have the potential to support the civil service. They can be used to automate tasks such as document classification, summarisation, and translation, among others. However, there are also risks and challenges associated with their use.
- Details
- Category: Special Theme
- Hits: 5386
by Katerina Papantoniou, Panagiotis Papadakos and Dimitris Plexousakis (ICS-FORTH)
At ICS-FORTH, we explore what LLMs know regarding the task of verbal deception detection. We evaluate the performance of two well-known LLMs and compare them with a fine-tuned BERT-based model. Finally, we explore whether the LLMs are aware of culture-specific linguistic deception detection cues.
- Details
- Category: Special Theme
- Hits: 5016
by Angelica Lo Duca (CNR-IIT)
At CNR-IIT we are investigating if generative AI can be used to improve Data Storytelling and generate more engaging and informative data stories.
- Details
- Category: Special Theme
- Hits: 4382
by Aradina Chettakattu and Denis Havlik (AIT Austrian Institute of Technology GmbH)
Building and using a common terminology is of great importance for all collaborative work. On the other hand, well implemented tagging of documents by several “orthogonal” vocabularies provides a concise representation of context and topics of a textual data, which facilitates search and retrieval, as well as automated matching of “similar” documents in knowledge management systems. Manually finding relevant keywords for documents is a daunting task, time-consuming and prone to errors. Our “Voctractor” (from “Vocabulary Extractor”) application prototype [L1] addresses this challenge by streamlining both the vocabulary design and the keyword extraction workflow.
- Details
- Category: Special Theme
- Hits: 3739
by Javier Cámara, Javier Troya and Lola Burgueño (ITIS Software / Universidad de Málaga)
There is a growing body of work assessing the capabilities of Large Language Models (LLMs) for writing code. Comparatively, the analysis of the current state of LLMs with respect to software modeling has received little attention. Are LLMs capable of generating useful software models? What factors determine the quality of such models? The work we are conducting at ITIS-UMA investigates the capabilities and main shortcomings of current LLMs in software modeling.
- Details
- Category: Special Theme
- Hits: 4423
by Guido Rocchietti, Cristina Ioana Muntean, Franco Maria Nardini (CNR-ISTI)
In the context of the Horizon Europe EFRA research project [L1], we explore the innovative use of Large Language Models (LLMs), both instructed and fine-tuned, in improving the quality of conversational search. The focus is on applying these models in rewriting conversational utterances to enhance the capability of conversational agents to retrieve accurate responses.
- Details
- Category: Special Theme
- Hits: 6105
by Diego Collarana, Moritz Busch, and Christoph Lange (Fraunhofer FIT)
Despite the excitement about Large Language Models (LLMs), they still fail in unpredictable ways in knowledge-intensive tasks. In this article, we explore the integration of LLMs with Knowledge Graphs (KGs) to develop cognitive conversational assistants with improved accuracy. To address the current challenges of LLMs, such as hallucination, updateability and provenance, we propose a layered solution that leverages the structured, factual data of KGs alongside the generative capabilities of LLMs. The outlined strategy includes constructing domain-specific KGs, interfacing them with LLMs for specialised tasks, integrating them with enterprise information systems and processes, and adding guardrails to validate their output, thereby presenting a comprehensive framework for deploying more reliable and context-aware AI applications in various industries.
- Details
- Category: Special Theme
- Hits: 3977
by Chara Tsoukala, Georgios Paraskevopoulos and Athanasios Katsamanis (Athena Research Center)
The Greek National Theatre has introduced an advanced chatbot based on Large Language Models (LLMs) as a novel means of accessing the content of the digital archive on the web. This state-of-the-art chatbot, which utilises Text2SQL conversion through LLMs, offers a more intuitive user experience, enabling complex searches with simple natural language. This addition marks a significant step forward in making theatrical performance data accessible in a more interactive way.
- Details
- Category: Special Theme
- Hits: 4158
by Gábor Berend (University of Szeged)
The use of transformer-based pretrained language models (PLMs) arguably dominates the natural language processing (NLP) landscape. The pre-training of such models, however, is known to be notoriously data and resource hungry, which hinders their creation in low-resource settings, making it a privilege to those few (mostly corporate) actors, who have access to sufficient computational resource and/or pre-training data. The main goal of our research is to develop such a novel sample-efficient pre-training paradigm of PLMs, which makes their use available in the low data and/or computational budget regime, helping the democratisation of this disruptive technology beyond the current status quo.
- Details
- Category: Special Theme
- Hits: 2903
by George Tambouratzis (Athena Research Centre)
Conversational agents and chatbots have developed rapidly in the past year to provide answers to user queries, drawing information from huge collections of data. From the user-side, the usefulness of conversational agents hinges on the accuracy of response in addition to user-friendliness and response speed. Here we briefly evaluate one of the most widely used chatbots, ChatGPT over a set of queries posed using multiple languages, to test its robustness and consistency, while running the experiment at two timepoints to monitor ChatGPT’'s evolution.
- Details
- Category: Special Theme
- Hits: 4050
by Michalis Mountantonakis and Yannis Tzitzikas (FORTH-ICS and University of Crete)
Since it is challenging to combine ChatGPT (which has been trained by using data from web sources), with popular RDF Knowledge Graphs (that include high quality information), we present a generic pipeline that exploits RDF Knowledge Graphs and short sentence embeddings for enabling the validation of ChatGPT responses, and an evaluation by using a benchmark containing 2,000 facts for popular Greek persons, places and events.
- Details
- Category: Special Theme
- Hits: 4238
by Jan Deriu and Mark Cieliebak (Zurich University of Applied Sciences)
With all the recent hype around Large Language Models and ChatGPT in particular, one crucial question is still unanswered: how do we evaluate generated text, and how can this be automated? In this SNF project, we develop a theoretical framework to answer these questions.