Explainable Artificial Intelligence for neural networks and its evaluation

Transparency in the context of an AI system is a fundamental property which is remarked by the EU AI Act and its foundational document, the Ethical Guidelines for Trustworthy AI: indeed, modern-day AI models, such as neural networks, are often so complex that their predictive dynamics are unintelligible to humans. One of the ways for enhancing transparency is by providing explanations, human-understandable tokens of information that approximate the functioning of said models. The branch behind the study of techniques for generating explanations is called Explainable AI (XAI). Other regulations, such as the GDPR, introduce a “right for explanation” for users whose data are processed automatically by other entities, further fueling the necessity for reliable XAI tools. However, the reliability of these methods has often been questioned, and the formal evaluation of XAI quality remains an open challenge, hindering the widespread applicability of XAI to real-world applications.

This intensive 5-day course provides an accelerated, interactive introduction to XAI in the specific case of neural networks. The first half of the course will be aimed at providing the basis of XAI, while the second half will dive into the topics of explanations quality. The lectures will be blending frontal lectures and active learning, with activities based on concept mapping and collaborative peer analysis. The final day will be dedicated to assessment, a multiple-choice exam for Bachelor’s and Master’s students and an evaluative group discussion for PhD candidates.

• Describe the main ways in which an Explainable AI tool can be assessed.
• Criticize the various approaches for Explainable AI with regards to their application, strengths, and weaknesses.
• Evaluate which facets of an Explainable AI tool can be important with regards to the various stakeholders of an AI system.

Prerequisite knowledge of Deep Learning

4 lectures:

1. Recap of Deep Learning fundamentals; Intro to XAI - main tasks
2. Techniques for feature attribution, counterfactual examples, concepts
3. Evaluation of XAI Part I
4. Evaluation of XAI Part II and stakeholder perspective over XAI - preparation for final exam

Final day:

written exam and group discussion

The two-level mutual trust-based quality assurance scheme has been adopted:

• at the university level: Technische Universiteit Delft has applied its internal quality assurance procedures and structures to the proposal of XAI evaluation course it submitted to ENHANCE and to its implementation - the related learning activities,
• at the Alliance level: the body composed of Education Officers has made decisions regarding the inclusion of XAI evaluation course proposed by Technische Universiteit Delft to the Innovative Learning Campus part of the joint ENHANCE educational offer, based on the compliance with the formal requirements and ENHANCE goals.

TBD

MSc/BSc students: written exam - PhD candidates: group discussion

MSc/BSc students with an ongoing (research) AI project may request to do a group discussion instead of the exam.

Completing the form does not grant automatic access to the course, the enrollment with be confirmed after manual check that prerequisite knowledge is valid

TU Delft

Additional study hours and group works will apply, granting the final ECTS at the end.