ERCIM News 116
January 2019
Special theme Transparency in Algorithmic Decision Making
Guest editors Andreas Rauber (TU Wien and SBA), Roberto Trasarti and Fosca Giannotti (ISTI-CNR)
High-Level Expert Group on Artificial Intelligence
- Details
- Category: Keynote
- Hits: 5550
by Sabine Theresia Köszegi
On 25 April 2018, the Europaen Commission published a Communication in which it announced an ambitious European Strategy for Artificial Intelligence (AI). The major advances in AI over the last decade revealed its capacity as a general-purpose technology and pushed inventions in areas of mobility, healthcare, home & service robotics, education and cyber security, to name just a few. These AI-enabled developments have the capability to generate tremendous benefits not only for individuals but also for the society as a whole. AI has also promising capabilities when it comes to address and resolve the grand challenges, such as climate change or global health and wellbeing, as expressed in the United Nations Sustainable Development goals. In competition with other key players, like the United States and China, Europe needs to leverage its current strengths, foster the enablers for innovation and technology uptake and find its unique selling proposition in AI to ensure a competitive advantage and a prosperous economic development in its Member States. At the same time, AI comes with risks and challenges associated to fundamental human rights and ethics. Europe therefore must ensure to craft a strategy that maximizes the benefits of AI while minimizing its risks.
Transparency in Algorithmic Decision Making
- Details
- Category: Special Theme
- Hits: 6656
by Andreas Rauber (TU Wien and SBA), Roberto Trasarti, Fosca Giannotti (ISTI-CNR)
The past decade has seen increasing deployment of powerful automated decision-making systems in settings ranging from smile detection on mobile phone cameras to control of safety-critical systems. While evidently powerful in solving complex tasks, these systems are typically completely opaque, i.e. they provide hardly any mechanisms to explore and understand their behaviour and the reasons underlying their decisions. This opaqueness raises numerous legal, ethical and practical concerns, which have led to initiatives and recommendations on how to address these problems, calling for greater scrutiny in the deployment of automated decision-making systems. Clearly, joint efforts are required across technical, legal, sociological and ethical domains to address these increasingly pressing issues.
The AI Black Box Explanation Problem
- Details
- Category: Special Theme
- Hits: 10531
by Riccardo Guidotti, Anna Monreale and Dino Pedreschi (KDDLab, ISTI-CNR Pisa and University of Pisa)
Explainable AI is an essential component of a “Human AI”, i.e., an AI that expands human experience, instead of replacing it. It will be impossible to gain the trust of people in AI tools that make crucial decisions in an opaque way without explaining the rationale followed, especially in areas where we do not want to completely delegate decisions to machines.
About Deep Learning, Intuition and Thinking
- Details
- Category: Special Theme
- Hits: 7665
by Fabrizio Falchi, (ISTI-CNR)
In recent years, expert intuition has been a hot topic within the discipline of psychology and decision making. The results of this research can help in understanding deep learning; the driving force behind the AI renaissance, which started in 2012.
Public Opinion and Algorithmic Bias
- Details
- Category: Special Theme
- Hits: 5699
by Alina Sîrbu (University of Pisa), Fosca Giannotti (ISTI-CNR), Dino Pedreschi (University of Pisa) and János Kertész (Central European University)
Does the use of online platforms to share opinions contribute to the polarization of the public debate? An answer from a modelling perspective.
Detecting Adversarial Inputs by Looking in the Black Box
- Details
- Category: Special Theme
- Hits: 6996
by Fabio Carrara, Fabrizio Falchi, Giuseppe Amato (ISTI-CNR), Rudy Becarelli and Roberto Caldelli (CNIT Research Unit at MICC ‒ University of Florence)
The astonishing and cryptic effectiveness of Deep Neural Networks comes with the critical vulnerability to adversarial inputs — samples maliciously crafted to confuse and hinder machine learning models. Insights into the internal representations learned by deep models can help to explain their decisions and estimate their confidence, which can enable us to trace, characterise, and filter out adversarial attacks.
Inspecting the Behaviour of Deep Learning Neural Networks
- Details
- Category: Special Theme
- Hits: 4349
by Alexander Dür, Peter Filzmoser (TU Wien) and Andreas Rauber (TU Wien and Secure Business Austria)
With the desire and need to be able to trust decision making systems, understanding the inner workings of complex deep learning neural network architectures may soon replace qualitative or quantitative performance as the primary focus of investigation and measure of success. We report on a study investigating a complex deep learning neural network architecture aimed at detecting causality relations between pairs of statements. It demonstrates the need to obtain a better understanding of what actually constitutes sufficient and useful insights into the behaviour of such architectures that go beyond mere transformation into rule-based representations.
Personalisable Clinical Decision Support System
- Details
- Category: Special Theme
- Hits: 4125
by Tamara Müller and Pietro Lió (University of Cambridge)
We introduce a Clinical Decision Support System (CDSS) as an operation of translational medicine. It is based on random forests, is personalisable and allows a clear insight into the decision making process. A well-structured rule set is created and every rule of the decision making process can be observed by the user (physician). Furthermore, the user has an impact on the creation of the final rule set and the algorithm allows the comparison of different diseases as well as regional differences in the same disease.
Putting Trust First in the Translation of AI for Healthcare
- Details
- Category: Special Theme
- Hits: 44855
by Anirban Mukhopadhyay, David Kügler (TU Darmstadt), Andreas Bucher (University Hospital Frankfurt), Dieter Fellner (Fraunhofer IGD and TU Darmstadt) and Thomas Vogl (University Hospital Frankfurt)
From screening diseases to personalised precision treatments, AI is showing promise in healthcare. But how comfortable should we feel about giving black box algorithms the power to heal or kill us?
Ethical and Legal Implications of AI Recruiting Software
- Details
- Category: Special Theme
- Hits: 8252
by Carmen Fernández and Alberto Fernández (Universidad Rey Juan Carlos)
Artificial Intelligence (AI) applications may have different ethical and legal implications depending on the domain. One application of AI is analysis of video-interviews during the recruitment process. There are pros and cons to using AI in this context, and potential ethical and legal consequences for candidates, companies and states. There is a deficit of regulation of these systems, and a need for external and neutral auditing of the types of analysis made in interviews. We propose a multi-agent system architecture for further control and neutral auditing to guarantee a fair, inclusive and accurate AI and to reduce the potential for discrimination, for example on the basis of race or gender, in the job market.
Towards Increased Transparency in Digital Insurance
- Details
- Category: Special Theme
- Hits: 5003
by Ulrik Franke (RISE SICS)
Automated decision-making has the potential to increase both productivity and competitiveness as well as compensate for well-known human biases and cognitive flaws [1]. But today’s powerful machine-learning based technical solutions also bring about problems of their own – not least in terms of being uncomfortably black-box like. A new research project at RISE Research Institutes of Sweden, in collaboration with KTH Royal Institute of Technology, has recently been set up to study transparency in the insurance industry, a sector that is poised to undergo technological disruption.
INDICÆTING – Automatically Detecting, Extracting, and Correlating Cyber Threat Intelligence from Raw Computer Log Data
- Details
- Category: Special Theme
- Hits: 5572
by Max Landauer and Florian Skopik (AIT Austrian Institute of Technology)
“Cyber threat intelligence” is security-relevant information, often directly derived from cyber incidents that enables comprehensive protection against upcoming cyber-attacks. However, collecting and transforming the available low-level data into high-level threat intelligence is usually time-consuming and requires extensive manual work as well as in-depth domain knowledge. INDICÆTING supports this procedure by developing and applying machine learning algorithms that automatically detect anomalies in the monitored system behaviour, correlate affected events to generate multi-step attack models and aggregate them to generate usable threat intelligence.
Why are Work Orders Scheduled too late? – A Practical Approach to Understand a Production Scheduler
- Details
- Category: Special Theme
- Hits: 4567
by Markus Berg (proALPHA) and Sebastian Velten (Fraunhofer ITWM)
In complex production environments, understanding the results of a scheduling algorithm is a challenging task. To avoid tardy work orders, proALPHA and Fraunhofer ITWM developed a component for identifying practical tardiness reasons and appropriate countermeasures.
Ethics in Research - Introduction
- Details
- Category: Research and Society
- Hits: 3131
by Claude Kirchner (Inria) and James Larrus (EPFL)
Science is in revolution. The formidable scientific and technological developments of the last century have dramatically transformed the way in which we conduct scientific research. The knowledge and applications that science produces has profound consequences on our society, both at the global level (for example, climate change) and the individual level (for example, impact of mobile devices on our daily lives). These developments also have a profound impact on the way scientists are working today and will work in the future. In particular, informatics and mathematics have changed the way we deal with data, simulations, models and digital twins, publications, and importantly, also with ethics.
How to Include Ethics in Machine Learning Research
- Details
- Category: Research and Society
- Hits: 5019
by Michele Loi and Markus Christen (University of Zurich)
The use of machine learning in decision-making has triggered an intense debate about “fair algorithms”. Given that fairness intuitions differ and can led to conflicting technical requirements, there is a pressing need to integrate ethical thinking into research and design of machine learning. We outline a framework showing how this can be done.
Fostering Reproducible Research
- Details
- Category: Research and Society
- Hits: 2943
by Arnaud Legrand (Univ. Grenoble Alpes/CNRS/Inria)
To accelerate the adoption of reproducible research methods, researchers from CNRS and Inria have designed a MOOC targeting PhD students, research scientists and engineers working in any scientific domain.
Research Ethics and Integrity Training for Doctoral Candidates: Face-to-Face is Better!
- Details
- Category: Research and Society
- Hits: 2989
by Catherine Tessier (Université de Toulouse)
The University of Toulouse and Inria have set up face-to-face training in research ethics and integrity for doctoral candidates based on debates about their own theses.
Efficient Accumulation of Scientific Knowledge, Research Waste and Accumulation Bias
- Details
- Category: Research and Society
- Hits: 4166
by Judith ter Schure (CWI)
An estimated 85 % of global health research investment is wasted [1]; a total of one hundred billion US dollars in the year 2009 when it was estimated. The movement to reduce this waste recommends that previous studies be taken into account when prioritising, designing and interpreting new research. Yet current practice to summarize previous studies ignores two crucial aspects: promising initial results are more likely to develop into (large) series of studies than their disappointing counterparts, and conclusive studies are more likely to trigger meta-analyses than not so noteworthy findings. Failing to account for these apects introduces ‘accumulation bias’, a term coined by our Machine Learning research group to study all possible dependencies potentially involved in meta-analysis. Accumulation bias asks for new statistical methods to limit incorrect decisions from health research while avoiding research waste.
Using Augmented Reality for Radiological Incident Training
- Details
- Category: Research and Innovation
- Hits: 4745
by Santiago Maraggi, Joan Baixauli and Roderick McCall (LIST)
Training for radiological events is time consuming and risky. In contrast to real sources, a prototype augmented reality system lets trainees and trainers safely learn about the necessary detection, identification and decontamination steps.
Building upon Modularity in Artificial Neural Networks
- Details
- Category: Research and Innovation
- Hits: 4471
by Zoltán Fazekas, Gábor Balázs, and Péter Gáspár (MTA SZTAKI)
In a pilot-study, an urban road environment detection function was considered for smart cars, as well as for self-driving cars. The implemented artificial neural network (ANN) based algorithms use the traffic sign (TS) and/or crossroad (CR) occurrences, along a route, as input. The TS-based and the CR-based classifiers were then merged into a compound one. The way this was accomplished serves as a simple, practical example of how to build upon modularity and how to retain some degree of it in functioning ANNs.
BBTalk: An Online Service for Collaborative and Transparent Thesaurus Curation
- Details
- Category: Research and Innovation
- Hits: 4416
by Christos Georgis, George Bruseker and Eleni Tsouloucha (ICS-FORTH)
BBTalk is an online service designed to support collaborative interdisciplinary development and extension of thesauri. At present, it serves to support the curation of the BackBone Thesaurus (BBT), a meta-thesaurus for the humanities. This service allows for the transparent, community development of the BackBone Thesaurus by enabling users to submit suggestions for changes and additions to the terminology, as well as link specialist thesauri to the meta-thesaurus terms, while enabling the thesauri curators to jointly edit, add and delete terminology. This model of cooperative editing is linked to an online discussion system that allows thesauri curators to confer with one another, exchange views and ideas and finally determine any necessary changes to the BBT.
Understandable Deep Neural Networks for Predictive Maintenance in the Manufacturing Industry
- Details
- Category: Research and Innovation
- Hits: 8513
by Anahid N.Jalali, Alexander Schindler and Bernhard Haslhofer
Data driven prognostic systems enable us to send out an early warning of machine failure in order to reduce the cost of failures and maintenance and to improve the management of the maintenance schedule. For this purpose, robust prognostic algorithms such as deep neural networks are used whose put is often difficult to interpret and comprehend. We investigate these models with the aim of moving towards a transparent and understandable model which can be applied on critical applications such as within the manufacturing industry.
Is My Definition the Same as Yours?
- Details
- Category: Research and Innovation
- Hits: 4326
by Gerhard Chroust (Johannes Kepler University Linz) and Georg Neubauer (Austrian Institute of Technology)
Effective and efficient communication and cooperation needs a semantically precise terminology, especially in disaster management, owing to the inherent urgency, time pressure, stress and often cultural differences of interventions. The European project Driver+ aims to measure the similarities between different countries’ terminologies surrounding disaster management. Each definition is characterised by a set of “descriptors” selected from a predefined anthology (the “bag-of-words”). The number of identical/different descriptors serves as a measure of the semantic similarity/difference of individual definitions and is translated into a numeric “degree of similarity”. The translation considers logical and intuitive aspects. Human judgment and mechanical derivation in the process are clearly separated and identified. By exchanging the ontology this method will also be applicable to other domains.
Science2Society Project Unveils the Effective Use of Big Research Data Transfer
- Details
- Category: Research and Innovation
- Hits: 3498
by Ricard Munné Caldés (ATOS)
The Science2Society project improves collaboration between science and industry by leveraging big research data through sustainable business models.
Informed Machine Learning for Industry
- Details
- Category: Research and Innovation
- Hits: 6247
by Christian Bauckhage, Daniel Schulz and Dirk Hecker (Fraunhofer IAIS)
Deep neural networks have pushed the boundaries of artificial intelligence but their training requires vast amounts of data and high performance hardware. While truly digitised companies easily cope with these prerequisites, traditional industries still often lack the kind of data or infrastructures the current generation of end-to-end machine learning depends on. The Fraunhofer Center for Machine Learning therefore develops novel solutions which are informed by expert knowledge. These typically require less training data and are more transparent in their decision-making processes.
POEMA - 15 Doctoral Student Positions Available
- Details
- Category: In Brief
- Hits: 3266
on the subject of Polynomial Optimization, Efficiency through Moments and Algebra at eleven European Research Institutes and Universitites
The Innovative Training Network POEMA is hiring 15 Doctoral Students starting from September 2019. The proposed projects will be investigating the development of new algebraic and geometric methods combined with computer algebra techniques for global non-linear optimization problems. Applications will focus on smarter cities challenges, urban traffic management, water network management, energy flow control, or environmental monitoring.
Dagstuhl Seminars and Perspectives Workshops
- Details
- Category: In Brief
- Hits: 3055
Call for Proposals
Schloss Dagstuhl – Leibniz-Zentrum für Informatik is accepting proposals for scientific seminars/workshops in all areas of computer science, in particular also in connection with other fields.
CWI, EIT Digital, Spirit, and UPM launch Innovation Activity “G-Moji”
- Details
- Category: In Brief
- Hits: 3095
Announcement
The Dutch youth care organization Spirit, CWI and Universidad Politécnica de Madrid (UPM) join forces in the new innovation activity of EIT Digital “G-Moji” ‒ a smartphone application to improve lives of youth at risk. G-Moji aims to support youth with mental health issues. In recent years,
New EU Project Data Market Services
- Details
- Category: In Brief
- Hits: 3305
Announcement
ERCIM participates in DataMarketServices (DMS), a new H2020 project whose objective is to overcome the barriers of data-centric European SMEs and start-ups by providing free support services around data skills, entrepreneurial opportunities, legal issues and standardization. The expected project deliverables are a 100-data-based companies portfolio, twelve free support services, webinars and training in topics such as GDPR, IPR, etc.
New W3C Web Experts Videos
- Details
- Category: In Brief
- Hits: 3179
Announcement
World renowned Web experts were attending the W3C TPAC ‘18 meeting in October in Lyon, and the W3C developer relations team seized this opportunity to organize a developer meetup featuring five prominent speakers and twelve demonstrations.