Shap Lime Interpretability | AI Final Year Project Report

Complete Technical Paper Content — Shap Lime Interpretability

1. Executive Abstract, Problem Formulation & Background

Explaining Models with LIME and SHAP This repository provides a notebook with examples in explaining 6 models (Naive Bayes, Logistic Regression, Decision Tree, Random Forest, Gradient Boosted Tree, Multilayer Perceptron) using LIME and SHAP. At Cloudera Fast Forward , we see model interpretability as an important step in the data science workflow. Being able to explain how a model works serves many purposes, including building trust in the model's output, satisfying regulatory requirements, model debugging, and verifying model safety, amongst other things. We have written a research report (access it free here ) that discusses this topic in detail. In this article, we revisit two industry standard algorithms for interpretability - LIME and SHAP. We discuss how these two algorithms work, and show some code examples of how to implement them in Python. At the end of this notebook , you should be familiar with: An overview of model interpretability Interpreting white box models, such as Linear/Logistic Regression (using model coefficients) and Tree models (using feature importance scores) Interpreting black box models with LIME and SHAP (KernelExplainer, TreeExplainer) and how to implement this in Python Good practices for "debugging" LIME and SHAP explanations Limitations of LIME/SHAP (a.k.a., when to choose LIME over SHAP) Figure 1. Plots of UMAP and PCA dimensionality reduction (two dimensions) applied to the dataset. Figure 2. Local explanations created with LIME for a given test data instance across 6 models. Figure 3. Local explanations created with SHAP Kernel Explainer for a given test data instance across 6 models. Figure 2 and 3 above show the local explanations created with LIME and SHAP for a given test data instance across 6 models. We see agreement in magnitude and direction across all models for both explanation methods (except for the Decision Tree). LIME vs SHAP : When to Use What? LIME and SHAP are both good methods for explaining models. In theory, SHAP is the better approach as it provides mathematical guarantees for the accuracy and consistency of explanations. In practice, the model agnostic implementation of SHAP (KernelExplainer) is slow, even with approximations. This speed issue is of much less concern if you are using a tree based model and can take advantage of the optimizations implemented in SHAP TreeExplainer (we saw it could be up to 100x faster than KernelExplainer). Some additional limitations of both methods are mentioned below: LIME is not designed to work with one hot encoded data. Considering that each data point is perturbed to create the approximate model, perturbing a one hot encoded variable may result in unexpected (meaningless) features. See discussion here . LIME depends on the ability to perturb samples in meaningful ways. This perturbation is use case specific. E.g., for tabular data, this entails adding random noise to each feature; for images, this entails replacing superpixels within the image with some mean value or zeroes; for text, this entails removing words from the text. It is often useful to think through any side effects of these perturbation strategies with respect to your data to further build trust in the explanation. In some cases, the local model built by LIME may fail to approximate the behaviour of the original model. It is good practice to check for such inconsistencies before trusting LIME explanations. LIME works with models that output probabilities for classification problems. Models like SVMs are not particularly designed to output probabilities (though they can be coerced into this with some issues .). This may introduce some bias into the explanations. SHAP depends on background datasets to infer a baseline/expected value. For large datasets, it is computationally expensive to use the entire dataset and we have to rely on appromixations (e.g. subsample the data). This has implications for the accuracy of the explanation. SHAP explains the deviation of a prediction from the expected/baseline value which is estimated using the training dataset. Depending on the specific use case, it may be more meaningful to compute the expected value using a specific subset of the training set as opposed to the entire training set. For example, it may be more meaningful to explain a churn prediction with respect to how it deviates from customers who did not churn. Here, we might want to use the dataset of customers who did not churn as our background dataset. See this issue here . Deploying on Cloudera Machine Learning (CML) There are three ways to launch this notebook on CML: From Prototype Catalog - Navigate to the Prototype Catalog in a CML workspace, select the "Explaining Models with LIME and SHAP" tile, click "Launch as Project", click "Configure Project" As ML Prototype - In a CML workspace, click "New Project", add a Project Name, select "ML Prototype" as the Initial Setup option, copy in the repo URL , click "Create Project", click "Configure Project" Manual Setup - In a CML workspace, click "New Project", add a Project Name, select "Git" as the Initial Setup option, copy in the repo URL , click "Create Project". Once the project has been initialized in a CML workspace, run the notebook by starting a Python 3 Jupyter notebook server session with at least 4GB of RAM. All library and model dependencies are installed inline in the notebook.

This repository provides a notebook with examples in explaining 6 models (Naive Bayes, Logistic Regression, Decision Tree, Random Forest, Gradient Boosted Tree, Multilayer Perceptron) using LIME and SHAP.

At Cloudera Fast Forward , we see model interpretability as an important step in the data science workflow. Being able to explain how a model works serves many purposes, including building trust in the model's output, satisfying regulatory requirements, model debugging, and verifying model safety, amongst other things. We have written a research report (access it free here ) that discusses this topic in detail. In this article, we revisit two industry standard algorithms for interpretability - LIME and SHAP. We discuss how these two algorithms work, and show some code examples of how to implement them in Python. At the end of this notebook , you should be familiar with:

2. Related Work, Literature Survey & Algorithmic Foundations

Figure 2 and 3 above show the local explanations created with LIME and SHAP for a given test data instance across 6 models. We see agreement in magnitude and direction across all models for both explanation methods (except for the Decision Tree).

LIME and SHAP are both good methods for explaining models. In theory, SHAP is the better approach as it provides mathematical guarantees for the accuracy and consistency of explanations. In practice, the model agnostic implementation of SHAP (KernelExplainer) is slow, even with approximations. This speed issue is of much less concern if you are using a tree based model and can take advantage of the optimizations implemented in SHAP TreeExplainer (we saw it could be up to 100x faster than KernelExplainer).

LIME is not designed to work with one hot encoded data. Considering that each data point is perturbed to create the approximate model, perturbing a one hot encoded variable may result in unexpected (meaningless) features. See discussion here .

3. Dataset Ingestion, Preprocessing & Stochastic Augmentations

LIME depends on the ability to perturb samples in meaningful ways. This perturbation is use case specific. E.g., for tabular data, this entails adding random noise to each feature; for images, this entails replacing superpixels within the image with some mean value or zeroes; for text, this entails removing words from the text. It is often useful to think through any side effects of these perturbation strategies with respect to your data to further build trust in the explanation.

In some cases, the local model built by LIME may fail to approximate the behaviour of the original model. It is good practice to check for such inconsistencies before trusting LIME explanations.

LIME works with models that output probabilities for classification problems. Models like SVMs are not particularly designed to output probabilities (though they can be coerced into this with some issues .). This may introduce some bias into the explanations.

4. Neural Network Backbone, Layer Specifications & Loss Formulations

SHAP depends on background datasets to infer a baseline/expected value. For large datasets, it is computationally expensive to use the entire dataset and we have to rely on appromixations (e.g. subsample the data). This has implications for the accuracy of the explanation.

SHAP explains the deviation of a prediction from the expected/baseline value which is estimated using the training dataset. Depending on the specific use case, it may be more meaningful to compute the expected value using a specific subset of the training set as opposed to the entire training set. For example, it may be more meaningful to explain a churn prediction with respect to how it deviates from customers who did not churn. Here, we might want to use the dataset of customers who did not churn as our background dataset. See this issue here .

Once the project has been initialized in a CML workspace, run the notebook by starting a Python 3 Jupyter notebook server session with at least 4GB of RAM. All library and model dependencies are installed inline in the notebook.

5. Training Protocol, Convergence Behavior & Experimental Evaluation

Advanced Interpretability of Black-Box Models Using SHAP and LIME Project Overview This project demonstrates advanced model interpretability techniques applied to a non-linear, “black-box” classification model. Using LightGBM as the predictive model, we explore both global and local interpretability through SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations). The focus is not just on high predictive performance, but also on understanding feature importance, interactions, and individual predictions , providing actionable insights for business stakeholders. Repository Contents File Description shap_analysis.py Main Python script to train LightGBM, generate SHAP and LIME explanations, and save the model. final_lgbm_model.pkl Trained LightGBM model serialized using Joblib. lime_explanation.html Interactive LIME explanation for a selected test instance. README.md Project overview, methodology, and results. Dataset Synthetic classification dataset with 1000 samples and 20 features . 10 informative features and 5 redundant features. Split into train (50%) and test (50%) sets. (In a real-world scenario, this could be a credit risk or customer churn dataset.) Model Training Model : LightGBM Classifier Hyperparameters : n_estimators=500 learning_rate=0.05 subsample=0.8 colsample_bytree=0.8 Evaluation Metric : AUC (Area Under the ROC Curve) Early Stopping : 30 rounds Performance on Test Set: AUC: 0.981 Accuracy: 0.92 Precision, Recall, F1-score: Class 0 -> Precision: 0.96, Recall: 0.88, F1: 0.92 Class 1 -> Precision: 0.88, Recall: 0.96, F1: 0.92 Interpretability 1. SHAP (Global & Local) Global Feature Importance: SHAP summary plot highlights the top influential features across all predictions. Local Explanation: SHAP waterfall plot explains individual predictions, showing contributions of each feature . Feature Interaction: Dependency plots reveal interactions between features that drive model predictions. Example Global Top Features: Feature Mean Absolute SHAP f17 1.09 f5 0.73 f16 0.59 f14 0.55 f4 0.48 2. LIME (Local Explanation) Local interpretability for a specific instance (high-confidence correct prediction). Provides human-understandable explanation by approximating the black-box model locally. Output is saved as lime_explanation.html for interactive visualization. Comparative Analysis Aspect SHAP LIME Scope Global + Local Local only Output Contribution values of all features Interpretable linear surrogate for one instance Feature Interaction Captured via dependency plots Limited Use Case Identify global important features & individual prediction drivers Explain single predictions to stakeholders Strength Consistent, theoretically sound (Shapley values) Highly intuitive for non-technical stakeholders Insights: SHAP and LIME explanations converge on the most influential features for the selected instance. Divergence may appear for features with weak impact, as LIME only approximates locally. SHAP’s global view helps identify blind spots and refine decision-making criteria. Usage Run the analysis script: python shap_analysis.py Outputs generated: final_lgbm_model.pkl → trained model SHAP summary plot (interactive) SHAP waterfall plot for selected instance lime_explanation.html → LIME interactive explanationOutputs generated: final_lgbm_model.pkl → trained model SHAP summary plot (interactive) SHAP waterfall plot for selected instance lime_explanation.html → LIME interactive explanation Dependencies Python 3.8+ Libraries: numpy, pandas, lightgbm, scikit-learn, shap, joblib, lime, matplotlib Install via: pip install numpy pandas scikit-learn lightgbm shap joblib lime matplotlib Conclusion This project demonstrates how state-of-the-art interpretability techniques can be applied to black-box models, providing both global insights and local explanations. The combination of SHAP and LIME ensures a comprehensive understanding, suitable for both data scientists and business stakeholders.

This project demonstrates advanced model interpretability techniques applied to a non-linear, “black-box” classification model. Using LightGBM as the predictive model, we explore both global and local interpretability through SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations).

The focus is not just on high predictive performance, but also on understanding feature importance, interactions, and individual predictions , providing actionable insights for business stakeholders.