Dataset Level Variable Importance as Change in Loss Function after Variable Permutations

Source: R/model_parts.R

From DALEX version 1.0 this function calls the feature_importance Find information how to use this function here: https://pbiecek.github.io/ema/featureImportance.html.

model_parts(
  explainer,
  loss_function = loss_default(explainer$model_info$type),
  ...,
  type = "variable_importance",
  N = n_sample,
  n_sample = 1000
)

Arguments

explainer

a model to be explained, preprocessed by the explain function
loss_function

a function that will be used to assess variable importance. By default it is 1-AUC for classification, cross entropy for multilabel classification and RMSE for regression.
...

other parameters
type

character, type of transformation that should be applied for dropout loss. variable_importance and raw results raw drop lossess, ratio returns drop_loss/drop_loss_full_model while difference returns drop_loss - drop_loss_full_model
N

number of observations that should be sampled for calculation of variable importance. If negative then variable importance will be calculated on whole dataset (no sampling).
n_sample

alias for N held for backwards compatibility. number of observations that should be sampled for calculation of variable importance.

Value

An object of the class feature_importance. It's a data frame with calculated average response.

References

Explanatory Model Analysis. Explore, Explain and Examine Predictive Models. https://pbiecek.github.io/ema/

Examples

 # \dontrun{
library("ranger")
titanic_ranger_model <- ranger(survived~., data = titanic_imputed, num.trees = 50,
                               probability = TRUE)
explainer_ranger  <- explain(titanic_ranger_model, data = titanic_imputed[,-8],
                             y = titanic_imputed$survived)
#> Preparation of a new explainer is initiated
#>   -> model label       :  ranger  (  default  )
#>   -> data              :  2207  rows  7  cols 
#>   -> target variable   :  2207  values 
#>   -> predict function  :  yhat.ranger  will be used (  default  )
#>   -> predicted values  :  numerical, min =  0.01960806 , mean =  0.3211121 , max =  0.9969517  
#>   -> model_info        :  package ranger , ver. 0.12.1 , task classification (  default  ) 
#>   -> residual function :  difference between y and yhat (  default  )
#>   -> residuals         :  numerical, min =  -0.7950186 , mean =  0.001044707 , max =  0.8950206  
#>   A new explainer has been created!  
vi_ranger <- model_parts(explainer_ranger, type = "raw")
head(vi_ranger, 8)
#>       variable mean_dropout_loss  label
#> 1 _full_model_         0.1038296 ranger
#> 2        parch         0.1207129 ranger
#> 3        sibsp         0.1235869 ranger
#> 4     embarked         0.1246202 ranger
#> 5          age         0.1750194 ranger
#> 6         fare         0.1848718 ranger
#> 7        class         0.2040485 ranger
#> 8       gender         0.3126926 ranger
plot(vi_ranger)

titanic_glm_model <- glm(survived~., data = titanic_imputed, family = "binomial")
explainer_glm <- explain(titanic_glm_model, data = titanic_imputed[,-8],
                         y = titanic_imputed$survived)
#> Preparation of a new explainer is initiated
#>   -> model label       :  lm  (  default  )
#>   -> data              :  2207  rows  7  cols 
#>   -> target variable   :  2207  values 
#>   -> predict function  :  yhat.glm  will be used (  default  )
#>   -> predicted values  :  numerical, min =  0.008128381 , mean =  0.3221568 , max =  0.9731431  
#>   -> model_info        :  package stats , ver. 4.0.0 , task classification (  default  ) 
#>   -> residual function :  difference between y and yhat (  default  )
#>   -> residuals         :  numerical, min =  -0.9628583 , mean =  -2.569729e-10 , max =  0.9663346  
#>   A new explainer has been created!  
logit <- function(x) exp(x)/(1+exp(x))
vi_glm <- model_parts(explainer_glm, type = "raw",
                        loss_function = function(observed, predicted)
                                     sum((observed - logit(predicted))^2))
head(vi_glm, 8)
#>       variable mean_dropout_loss label
#> 1 _full_model_          562.1659    lm
#> 2        parch          562.1322    lm
#> 3         fare          562.4590    lm
#> 4        sibsp          563.2325    lm
#> 5     embarked          564.4570    lm
#> 6          age          567.9731    lm
#> 7        class          584.2344    lm
#> 8       gender          612.2269    lm
plot(vi_glm)
 # }