Explainable Artificial Intelligence for neural networks and its evaluation

Transparency in the context of an AI system is a fundamental property which is remarked by the EU AI Act and its foundational document, the Ethical Guidelines for Trustworthy AI: indeed, modern-day AI models, such as neural networks, are often so complex that their predictive dynamics are unintelligible to humans. One of the ways for enhancing transparency is by providing explanations, human-understandable tokens of information that approximate the functioning of said models. The branch behind the study of techniques for generating explanations is called Explainable AI (XAI). Other regulations, such as the GDPR, introduce a “right for explanation” for users whose data are processed automatically by other entities, further fueling the necessity for reliable XAI tools. However, the reliability of these methods has often been questioned, and the formal evaluation of XAI quality remains an open challenge, hindering the widespread applicability of XAI to real-world applications.

This intensive 5-day course provides an accelerated, interactive introduction to XAI in the specific case of neural networks. The course is composed of lectures and practical activities. The lectures will be blending frontal lectures and active learning, with activities based on concept mapping and collaborative peer analysis. The practical part will include labs and group work aimed at solving small challenges on the topic of (X)AI.

The practical activities will be part of the course assessment.

• Describe the main ways in which an Explainable AI tool can be assessed.
• Criticize the various approaches for Explainable AI with regards to their application, strengths, and weaknesses.
• Evaluate which facets of an Explainable AI tool can be important with regards to the various stakeholders of an AI system.

Prerequisite knowledge of Deep Learning

4 lectures on the themes of XAI, its evaluation, and socio-technical perspectives.

3 labs on the implementations of the lecture themes

Extra work for the implementation of the group project

The two-level mutual trust-based quality assurance scheme has been adopted:

• at the university level: Technische Universiteit Delft has applied its internal quality assurance procedures and structures to the proposal of XAI evaluation course it submitted to ENHANCE and to its implementation - the related learning activities,
• at the Alliance level: the body composed of Education Officers has made decisions regarding the inclusion of XAI evaluation course proposed by Technische Universiteit Delft to the Innovative Learning Campus part of the joint ENHANCE educational offer, based on the compliance with the formal requirements and ENHANCE goals.

TBD

The attainment of the learning outcomes will be assessed with group-based practical work and its presentation in front of the teaching team and the other students. Additionally, groups will be required to provide a reflection of the socio-technical implications of their work.

Completing the form does not grant automatic access to the course, the enrollment with be confirmed after manual check that prerequisite knowledge is valid.

TU Delft

Additional study hours and group works will apply.

The application period will close as soon as 40 students have enrolled. A selection procedure will then take place. This means that applications may close earlier than the communicated deadline.

Please contact your home university mobility officer to check whether there is travel support funding available.