Umeå University's logo

Graph neural networks have helped researchers overcome the challenges of deep learning on graphs in non-Euclidean space. Like most deep learning algorithms, although the prediction of the models produces good results, explaining the predictions of the model is often challenging. This paper will focus on applying graph neural networks to predict the properties of the various molecules in the molecular datasets. The aim is to explore the generation of explanations for molecule property predictions. The four graph neural networks and seven explainers are chosen to generate and compare the quality of the explanations that are given by the explainers for each of the model predictions. The quality of this explanation is measured by sparsity, fidelity, and fidelity inverse. It is observed that various models find it difficult to learn the node embeddings when there is a class imbalance; despite the models achieving a 75% accuracy and the F1_Score was 66%. It is also observed that for all datasets, sparsity had a statistically significant effect on fidelity; that is, as more important features are masked, the quality of the explanation reduces. The effect of sparsity on fidelity inverse varied from dataset to dataset; as more unimportant features were masked, the quality of the explanations improved in some datasets, yet the change was not significant in other datasets. Finally, it was observed that the explanation quality differs across models. However, larger neural networks produced better predictions in our experiments, and the quality of the explanation of those predictions was not lower than that of smaller neural networks.

The availability of easy-to-use and reliable software implementations is important for allowing researchers in academia and industry to test, assess and take into use eXplainable AI (XAI) methods. This paper describes the py-ciu Python implementation of the Contextual Importance and Utility (CIU) model-agnostic, post-hoc explanation method and illustrates capabilities of CIU that go beyond the current state-of-the-art that could be useful for XAI practitioners in general.

 This volume constitutes the papers of several workshops which were held in conjunction with the 6th International Workshop on Explainable and Transparent AI and Multi-Agent Systems, EXTRAAMAS 2024, in Auckland, New Zealand, during May 6–10, 2024.

The 13 full papers presented in this book were carefully reviewed and selected from 25 submissions. The papers are organized in the following topical sections: User-centric XAI; XAI and Reinforcement Learning; Neuro-symbolic AI and Explainable Machine Learning; and XAI & Ethics.

The United Nations launched sustainable development goals in 2015 that include goals for sustainable energy. From global energy consumption, households consume 20–30% of energy in Europe, North America and Asia; furthermore, the overall global energy consumption has steadily increased in the recent decades. Consequently, to meet the increased energy demand and to promote efficient energy consumption, there is a persistent need to develop applications enhancing utilization of energy in buildings. However, despite the potential significance of AI in this area, few surveys have systematically categorized these applications. Therefore, this paper presents a systematic review of the literature, and then creates a novel taxonomy for applications of smart building energy utilization. The contributions of this paper are (a) a systematic review of applications and machine learning methods for smart building energy utilization, (b) a novel taxonomy for the applications, (c) detailed analysis of these solutions and techniques used for the applications (electric grid, smart building energy management and control, maintenance and security, and personalization), and, finally, (d) a discussion on open issues and developments in the field.

Contextual Importance and Utility (CIU) is a model-agnostic method for explaining outcomes of AI systems. CIU has succeeded in producing meaningful explanations where state-of-the-art methods fail, e.g. for detecting bleeding in gastroenterological images. This paper presents a Python implementation of CIU for explaining image classifications.

The Internet of things (IoT) is expected to have an impact on business and the world at large in a way comparable to the Internet itself. An IoT product is a physical product with an associated virtual counterpart connected to the internet with computational as well as communication capabilities. The possibility to collect information from internet-connected products and sensors gives unprecedented possibilities to improve and optimize product use and maintenance. Virtual counterpart and digital twin (DT) concepts have been proposed as a solution for providing the necessary information management throughout the whole product lifecycle, which we here call product lifecycle information management (PLIM). Security in these systems is imperative due to the multiple ways in which opponents can attack the system during the whole lifecycle of an IoT product. To address this need, the current study proposes a security architecture for the IoT, taking into particular consideration the requirements of PLIM. The security architecture has been designed for the Open Messaging Interface (O-MI) and Open Data Format (O-DF) standards for the IoT and product lifecycle management (PLM) but it is also applicable to other IoT and PLIM architectures. The proposed security architecture is capable of hindering unauthorized access to information and restricts access levels based on user roles and permissions. Based on our findings, the proposed security architecture is the first security model for PLIM to integrate and coordinate the IoT ecosystem, by dividing the security approaches into two domains: user client and product domain. The security architecture has been deployed in smart city use cases in three different European cities, Helsinki, Lyon, and Brussels, to validate the security metrics in the proposed approach. Our analysis shows that the proposed security architecture can easily integrate the security requirements of both clients and products providing solutions for them as demonstrated in the implemented use cases.

Contextual Importance and Utility (CIU) is a model-agnostic method for producing situation- or instance-specific explanations of the outcome of so-called black-box systems. A major difference between CIU and other outcome explanation methods (also called post-hoc methods) is that CIU produces explanations without producing any intermediate interpretable model. CIU’s notion of importance is similar as in Decision Theory but differs from how importance is defined for other outcome explanation methods. Utility is also a well-known concept from Decision Theory that is largely ignored in current Explainable AI research. CIU is here validated by providing explanations for the two popular medical data sets - heart disease and breast cancer in order to show the applicability of CIU explanations on medical predictions and with different black-box models. The explanations are compared with corresponding ones produced by the Local Interpretable Model-agnostic Explanations (LIME) method [17], which is currently one of the most used post-hoc explanation methods. The paper’s main contribution is to provide new CIU results and insights on several benchmark data sets and showing in what way CIU differs from LIME-based explanations.

Contextual Importance and Utility (CIU) is a model-agnostic method for post-hoc explanation of prediction outcomes. In this paper we describe and show new functionality in the R implementation of CIU for tabular data. Much of that functionality is specific to CIU and goes beyond the current state of the art.

This work introduces the notion of intermediate concepts based on levels structure to aid explainability for black-box models. The levels structure is a hierarchical structure in which each level corresponds to features of a dataset (i.e., a player-set partition). The level of coarseness increases from the trivial set, which only comprises singletons, to the set, which only contains the grand coalition. In addition, it is possible to establish meronomies, i.e., part-whole relationships, via a domain expert that can be utilised to generate explanations at an abstract level. We illustrate the usability of this approach in a real-world car model example and the Titanic dataset, where intermediate concepts aid in explainability at different levels of abstraction.