PaddlePaddle/InterpretDL

PaddlePaddle/InterpretDL

Description: InterpretDL: Interpretation of Deep Learning Models，基于『飞桨』的模型可解释性算法库。

Language: Python

License: Apache-2.0

Stars: 261

Forks: 39

Open issues: 6

Created: 2020-06-22T01:26:45Z

Pushed: 2024-09-04T01:07:30Z

Default branch: master

Fork: no

Archived: no

README: [中文](./README_CN.md) | English

 

!logo

InterpretDL: Interpretation of Deep Learning Models based on PaddlePaddle

InterpretDL, short for *interpretations of deep learning models*, is a model interpretation toolkit for PaddlePaddle models. This toolkit contains implementations of many interpretation algorithms, including LIME, Grad-CAM, Integrated Gradients and more. Some SOTA and new interpretation algorithms are also implemented.

*InterpretDL is under active construction and all contributions are welcome!*


Why InterpretDL

The increasingly complicated deep learning models make it impossible for people to understand their internal workings. Interpretability of black-box models has become the research focus of many talented researchers. InterpretDL provides a collection of both classical and new algorithms for interpreting models.

By utilizing these helpful methods, people can better understand why models work and why they don't, thus contributing to the model development process.

For researchers working on designing new interpretation algorithms, InterpretDL gives an easy access to existing methods that they can compare their work with.

:fire: :fire: :fire: News :fire: :fire: :fire:

• (2024.6) A paper investigating feature attributions of LLMs via Optimal Transport got accepted by ICML'24. See implementations at GiLOT.

> Xuhong Li, Jiamin Chen, Yekun Chai, Haoyi Xiong. GILOT: Interpreting Generative Language Models via Optimal Transport. ICML 2024. paper link.

• (2023.10) M4 XAI Benchmark got accepted by Neurips'23 Datasets and Benchmarks. See implementations at M4_XAI_Benchmark.

> Xuhong Li, Mengnan Du, Jiamin Chen, Yekun Chai, Himabindu Lakkaraju, Haoyi Xiong. “M4: A Unified XAI Benchmark for Faithfulness Evaluation of Feature Attribution Methods across Metrics, Modalities and Models.” Neurips 2023, Dataset and Benchmark Track. paper link.

• (2023.2) One paper on explaining Transformers got accepted by TMLR. See implementations at BT.

> Jiamin Chen, Xuhong Li, Lei Yu, Dejing Dou, Haoyi Xiong. “Beyond Intuition: Rethinking Token Attributions inside Transformers.” TMLR. paper link.

Demo

Interpretation algorithms give a hint of why a black-box model makes its decision.

The following table gives visualizations of several interpretation algorithms applied to the original image to tell us why the model predicts "bull_mastiff."

| Original Image | IntGrad (demo) | SG (demo) | LIME (demo) | Grad-CAM (demo) | | :------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | |  |  |  |  |  |

For sentiment analysis task, the reason why a model gives positive/negative predictions can be visualized as follows. A quick demo can be found here. Samples in Chinese are also available here.


Contents

• [InterpretDL: Interpretation of Deep Learning Models based on PaddlePaddle](#interpretdl-interpretation-of-deep-learning-models-based-on-paddlepaddle)
• [Why InterpretDL](#why-interpretdl)
• [:fire: :fire: :fire: News :fire: :fire: :fire:](#fire-fire-fire-news-fire-fire-fire)
• [Demo](#demo)
• [Contents](#contents)
• [Installation](#installation)
• [Pip installation](#pip-installation)
• [Developer installation](#developer-installation)
• [Unit Tests](#unit-tests)
• [Documentation](#documentation)
• [Getting Started](#getting-started)
• [Examples and Tutorials](#examples-and-tutorials)
• [Roadmap](#roadmap)
• [Implemented Algorithms with Taxonomy](#implemented-algorithms-with-taxonomy)
• [Implemented Trustworthiness Evaluation Algorithms](#implemented-trustworthiness-evaluation-algorithms)
• [Presentations](#presentations)
• [References of Algorithms](#references-of-algorithms)
• [Copyright and License](#copyright-and-license)
• [Recent News](#recent-news)

Installation

It requires the deep learning framework paddlepaddle, versions with CUDA support are recommended.

Pip installation

pip install interpretdl

# or with tsinghua mirror
pip install interpretdl -i https://pypi.tuna.tsinghua.edu.cn/simple

Developer installation

git clone https://github.com/PaddlePaddle/InterpretDL.git
# ... fix bugs or add new features
cd InterpretDL && pip install -e .
# welcome to propose pull request…