Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models
Giorgio Visania, Enrico Bagli, Federico Chesani, Alessandro Poluzzi and Davide Capuzzo
ABSTRACT of Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models
Nowadays we are witnessing a transformation of the business processes towards a more computation driven approach. The ever increasing usage of Machine Learning techniques is the clearest example of such trend. This sort of revolution is often providing advantages, such as an increase in prediction accuracy and a reduced time to obtain the results. However, these methods present a major drawback: it is very difficult to understand on what grounds the algorithm took the decision. To address this issue we consider the LIME method. We give a general background on LIME then, we focus on the stability issue: employing the method repeated times, under the same conditions, may yield to different explanations. Two complementary indices are proposed, to measure LIME stability. It is important for the practitioner to be aware of the issue, as well as to have a tool for spotting it. Stability guarantees LIME explanations to be reliable therefore a stability assessment, made through the proposed indices, is crucial. As a case study, we apply both Machine Learning and classical statistical techniques to Credit Risk data. We test LIME on the Machine Learning algorithm and check its stability. Eventually, we examine the goodness of the explanations returned.
Introduction to Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models
Nowadays, more and more interest is devoted to the concept of "learning from the data", i.e. using the data collected about the process to predict its outcome (Hastie, Tibshirani, & Friedman, 2009). The main ingredients of its recent success are the huge availability of data sources and the increased computational power, which allows complex algorithms to deliver results in a relatively short time.
In statistics, making predictions about the future is a particularly relevant topic. To address the subject, simple algorithms and methods have been developed over the years, the most famous being Linear Regression and Generalised Linear Models (Greene, 2003). However, with the advent of powerful computing tools, more sophisticated techniques have been developed. In particular, Machine Learning models are able to perform intelligent tasks usually done by humans, supporting the automation of data driven processes.
Despite the enhanced accuracy, Machine Learning models display weakness especially when it comes to interpretability, i.e. \the ability to explain or to present the results, in understandable terms, to a human" (Hall & Gill, 2018). They usually adopt large model structures and refine the prediction using a huge number of iterations. The logic underlying the model ends up hidden under potentially many strata of mathematical calculations, as well as scattered across a too vast architecture, preventing humans from grasping it.
To achieve the interpretability, quite a few techniques have been proposed in re- cent literature. These approaches can be grouped based on different criteria Molnar (2020a), Guidotti et al. (2018) such as i) Model agnostic or model specific ii) Local, global or example based iii) Intrinsic or post-hoc iv) Perturbation or saliency based. Herein, we focus on LIME (Local Interpretable Model-agnostic Explanations), a local interpretability framework, developed by Ribeiro, Singh, & Guestrin, 2016.
The technique may suffer from a lack of stability, namely repeated applications of the method under the same conditions may obtain different results. This is a particularly delicate issue however it is rarely taken into consideration. Even worse, many times the issue is not spotted at all, e.g. when just a single call to the method is done and the result is considered to be okay without further checks. In this paper, we introduce a pair of complementary stability indices, useful to measure LIME stability and spot potential issues. They represent an innovative contribution to the scientific community, addressing an important research question.
The indices are calculated on repeated calls of the method, to evaluate the similarity of the results. They may be applied on every trained LIME method and will allow the practitioner to be aware about potential instability of the results, otherwise to ensure that the trained method is consistent.
Hereafter, a brief introduction on the explainability techniques is presented in Chapter 2. The LIME technique is exhaustively analysed in Chapter 4, including its weak points. A thorough discussion about LIME stability can be found in Chapter 4, along with a description of some recent works tackling the issue. Our proposition is extensively discussed in Chapter 5. Eventually, a practical application of the method in the Credit Risk Modelling field is shown in Chapter 6. Chapter 7 is dedicated to Discussion and Conclusions. The code used for the experiments is available at https://github.com/giorgiovisani/LIME stability.
ARE YOU A DEVELOPER?
Check out all the resources for TPPs and developers on the Crif Platform development portal.