ERCIM News 134
July 2023
Special theme Explainable AI
Guest editors Manjunatha Veerappa (Fraunhofer IOSB) and Salvo Rinzivillo (CNR-ISTI)
- Details
- Category: Special Theme
- Hits: 4959
by Manjunatha Veerappa (Fraunhofer IOSB) and Salvo Rinzivillo (CNR-ISTI)
Artificial Intelligence (AI) has witnessed remarkable advancements in recent years, transforming various domains and enabling groundbreaking capabilities. However, the increasing complexity of AI models, such as convolutional neural networks (CNNs) and deep learning architectures, has raised concerns regarding their interpretability and explainability. As AI systems become integral to critical decision-making processes, it becomes essential to understand and trust the reasoning behind their outcomes. This need has given rise to the field of explainable AI (XAI), which focuses on developing methods and frameworks to enhance the interpretability and transparency of AI models, bridging the gap between accuracy and explainability.
- Details
- Category: Special Theme
- Hits: 12993
by Mihály Héder (SZTAKI)
Understandability of computers has been a research topic from the very early days, but more systematically from the 1980s, when human-computer interaction started to take shape. In their book published in 1986, Winograd and Flores [1] extensively dealt with the issues of explanations and transparency. They set out to replace vague terms like “user-friendly”, “easy-to-learn” and “self-explaining” with scientifically grounded design principles. They did this by relying on phenomenology and, especially, cognitive science. Their key message was that a system needs to reflect how the user's mental representation of the domain of use is structured. From our current vantage point, almost four decades later, we can see that this was the user-facing variation of a similar idea, but for developers – object-oriented programming, a method on the rise at the time.
- Details
- Category: Special Theme
- Hits: 3788
by Andżelika Zalewska-Küpçü (QED Software), Andrzej Janusz (University of Warsaw & QED Software) and Dominik Ślęzak (University of Warsaw & QED Software)
BrightBox technology presents a novel approach to investigating mistakes in machine learning model operations.
- Details
- Category: Special Theme
- Hits: 4235
by Francesco Spinnato (Scuola Normale Superiore and ISTI-CNR), Riccardo Guidotti (University of Pisa) and Anna Monreale (University of Pisa )
We present LASTS, an XAI framework that addresses the lack of explainability in black-box time series classifiers. LASTS utilises saliency maps, instance-based explanations and rule-based explanations to provide interpretable insights into the predictions made by these classifiers. LASTS aims to bridge the gap between accuracy and explainability, specifically in critical domains.
- Details
- Category: Special Theme
- Hits: 3688
by Antonio Bruno, Giacomo Ignesti and Massimo Martinelli (CNR-ISTI).
Correct classification is the main aspect in evaluating the quality of an artificial intelligence system, but what happens when you reach top accuracy and no method explains how it works? In our study, we aim at addressing the black-box problem using an ad-hoc built classifier for lung ultrasound images.- Details
- Category: Special Theme
- Hits: 3881
by Luigi Briguglio, Francesca Morpurgo and Carmela Occhipinti (CyberEthics Lab.)
How can clinicians be deemed responsible for basing their decisions on diagnoses generated by artificial intelligence and derived in a way that cannot be fully understood? How can patients rely and accept decisions if they are based on “black boxes” of data and algorithms? In the context of the MES-CoBraD project, CyberEthics Lab. defines a model for governing and assessing “Ethical Artificial Intelligence” (ETHAI).
- Details
- Category: Special Theme
- Hits: 3833
by Nikolaos Rodis (Harokopio University of Athens), Christos Sardianos (Harokopio University of Athens) and Georgios Th. Papadopoulos (Harokopio University of Athens)
Despite the outstanding advances in Artificial Intelligence (AI) and its widespread adoption in several application domains, there are still significant challenges that need to be addressed regarding the explanation of how decisions are reached to the end-user. The latter need becomes more complex and demanding when multiple types of data are involved in the AI-based generated decisions; hence, leading to the emergence of the so-called multimodal explainable AI (MXAI) field. The above challenges become even more imperative for some critical domains, for example, medical applications (where human lives are involved).
- Details
- Category: Special Theme
- Hits: 3624
by Alexandre Lädermann (Hôpital de La Tour, Meyrin, Switzerland), Philippe Collin (American Hospital of Paris, France) and Patrick J. Denard (Oregon Shoulder Institute, Medford, Oregon, USA)
Surgery is said to be indicated when conservative treatment fails. Previous studies reported that around 20% of patients do not improve sufficiently after surgery, inducing frustration, high societal costs, and an overload of healthcare systems. The consortium members investigated the efficacy of machine learning methods in detecting outcomes. They achieved a promising model with a recall of 32% of the cases that were inappropriate candidates for an operation.
- Details
- Category: Special Theme
- Hits: 3240
by Michaela Areti Zervou, Effrosyni Doutsi, Panagiotis Tsakalides (University of Crete and FORTH-ICS)
Precision medicine holds the promise of personalised treatment based on an individual's genetic makeup. However, the lengthy and costly process of drug discovery hinders progress. Can artificial intelligence (AI), specifically generative models offer a solution? Is it possible to efficiently select the most promising drug candidates for validation? Our research focuses on developing robust tools that streamline the validation process of drug design – saving time and resources.
- Details
- Category: Special Theme
- Hits: 3988
by Gianluigi Folino, Massimo Guarascio, Luigi Pontieri and Paolo Zicari (CNR-ICAR)
Pushing intelligence and integrating explainable tools in the new generation of ticket-management systems is crucial for supporting customer-support activities. To this aim, we defined a comprehensive ticket-classification framework, which integrates deep ensemble methods and AI-based interpretation techniques to help both the operator identify misclassification errors and the analyst improve the model. Tests on real data demonstrate the quality of the predictions returned by the framework and the practical value of their associated explanations.
- Details
- Category: Special Theme
- Hits: 4450
by Danilo Brajovic and Marco F. Huber (Fraunhofer Institute for Manufacturing Engineering and Automation IPA)
The debate about reliable, transparent and thus, explainable AI applications is in full swing. Despite that, there is a lack of experience in how to integrate AI-specific safety aspects into standard software development. In the veoPipe research project, Fraunhofer IPA, Huber Automotive, and ROI-EFESO Management Consulting work on a joint approach to integrate these AI-specific aspects into an automotive-development process. In this article, we share details on one component of this framework – reporting the AI development.
- Details
- Category: Special Theme
- Hits: 3316
by Anahid Jalali (AIT), Andreas Rauber (TUWien), Jasmin Lampert (AIT)
Deep learning models for time series prediction have become popular with the rise of IoT and sensor data availability. However, their lack of explainability hampers their use in critical industrial applications. While existing model-agnostic approaches like LIME and SHAP have been used in time series classification applications, it is worth mentioning that they may have limitations in their suitability. For example, the random sampling process used by LIME leads to unstable explanations. We propose a counterfactual explanation approach for interpretable insights into time series predictions to address this issue. We choose an industrial use case, determining machine health, and employ k-means clustering and Dynamic Time Warping (DTW) to handle the temporal dimension. DTW compares and aligns two time series by discovering the optimal path of alignment that minimizes disparities in their temporal patterns. We explain the model's decisions using local surrogate decision trees, analysing feature importance and decision cuts.
- Details
- Category: Special Theme
- Hits: 5710
by Michalis Mountantonakis and Yannis Tzitzikas (FORTH-ICS and University of Crete)
The novel artificial intelligence ChatGPT chatbot offers detailed responses across many domains of knowledge; however, quite often it returns erroneous facts even for popular persons, events and places. To tackle this problem, we present GPT•LODS, a novel prototype that annotates and validates ChatGPT responses by leveraging one or more RDF knowledge graphs.
- Details
- Category: Special Theme
- Hits: 3588
by Christoforos Prasatzakis, Theodore Patkos and Dimitris Plexousakis (ICS-FORTH)
In this article, we present a new, easy-to-understand and flexible chatbot architecture, which banks on ease of use and modularity. It relies on the Event Calculus, in order to perform high-level reasoning on world events and agents’ knowledge, and can answer questions regarding the other agents’ belief state, in addition to what is happening in the chatbot’s world in general.
- Details
- Category: Special Theme
- Hits: 3690
by Alessia Amelio, Gianluca Bonifazi, Domenico Ursino and Luca Virgili (Polytechnic University of Marche
We propose an approach to map a convolutional neural network (CNN) into a multilayer network. It allows the interpretability of the internal structure of deep learning architectures. Then, we use this representation to compress the CNN.
- Details
- Category: Special Theme
- Hits: 3861
by Anahid Jalali, Alexander Schindler (AIT) and Anita Zolles (BFW)
Explainable Artificial Intelligence (XAI) is gaining importance in various fields, including forestry and tree-growth modelling. However, challenges such as evaluating model interpretability, lack of transparency in some XAI methods, inconsistent terminology, and bias towards specific data types hinder its integration. This article proposes combining long short-term memories (LSTMs) with example-based explanations to enhance tree-growth models' interpretability. By generating counterfactual examples and engaging domain experts, critical features impacting outcomes can be identified. Addressing privacy protection and selecting appropriate reference models are also crucial. Overcoming these challenges will lead to more interpretable models, supporting informed decision-making in forestry and climate change mitigation.
- Details
- Category: Special Theme
- Hits: 3734
by Davide Ceolin (CWI) and Ji Qi (Netherlands eScience Center)
The spread of disinformation affects society as a whole and the recent developments in AI are likely to aggravate the problem. This calls for automated solutions that assist humans in the task of automated information quality assessment, a task that can be perceived as subjective or biased, and that thus requires a high level of transparency and customisation. At CWI, together with the Netherlands eScience Center, we investigate how to design AI pipelines that are fully transparent and tunable by an end user. These pipelines will be applied to automated information quality assessment, using reasoning, natural language processing (NLP), and crowdsourcing components, and are available in the form of open source workflows.
- Details
- Category: Special Theme
- Hits: 5592
by Mahmoud Jaziri (Luxembourg Institute of Science and Technology) and Olivier Parisot (Luxembourg Institute of Science and Technology)
AI is an indispensable part of the astronomer's toolbox, particularly for detecting new deep space objects like gas clouds from the immense image databases filled every day by ground and space telescopes. We applied explainable AI (XAI) techniques for computer vision to ensure that deep sky objects classification models are working as intended and are free of bias.