# Visualizing the Feature Importance for Black Box Models

Source: [Visualizing the Feature Importance for Black Box Models](<https://www.sota2.com/research/paper/68fa67e81e209131cea190b9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:56:43.504Z

Authors: Giuseppe Casalicchio, Christoph Molnar, Bernd Bischl

Published: 2018-04-18T09:35:38.000Z

## Abstract

In recent years, a large amount of model-agnostic methods to improve the transparency, trustability and interpretability of machine learning models have been developed. We introduce local feature importance as a local version of a recent model-agnostic global feature importance method. Based on local feature importance, we propose two visual tools: partial importance (PI) and individual conditional importance (ICI) plots which visualize how changes in a feature affect the model performance on average, as well as for individual observations. Our proposed methods are related to partial dependence (PD) and individual conditional expectation (ICE) plots, but visualize the expected (conditional) feature importance instead of the expected (conditional) prediction. Furthermore, we show that averaging ICI curves across observations yields a PI curve, and integrating the PI curve with respect to the distribution of the considered feature results in the global feature importance. Another contribution of our paper is the Shapley feature importance, which fairly distributes the overall performance of a model among the features according to the marginal contributions and which can be used to compare the feature importance across different models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Local Feature Importance on Corners Simulation v1](<https://www.sota2.com/research/sota/local-feature-importance-on-corners-simulation-v1.md>): Local Feature Importance; Corners Simulation v1; FP-MAE: 0.566.

- [Local Feature Importance on Regression Interaction Simulation](<https://www.sota2.com/research/sota/local-feature-importance-on-regression-interaction-simulation.md>): Local Feature Importance; Regression Interaction Simulation; FP-MAE: 0.306.

- [Local Feature Importance on AND Gate Simulation](<https://www.sota2.com/research/sota/local-feature-importance-on-and-gate-simulation.md>): Local Feature Importance; AND Gate Simulation; FP-MAE: 0.466.

- [Local Feature Importance on Corners Simulation v2](<https://www.sota2.com/research/sota/local-feature-importance-on-corners-simulation-v2.md>): Local Feature Importance; Corners Simulation v2; FP-MAE: 0.368.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1804.06620>)

[PDF](<https://arxiv.org/pdf/1804.06620.pdf>)
