Score models by suitable metrics
score_models(
models,
predictions,
observed,
type,
metrics = "auto",
sort_by = "auto",
metric_function = NULL,
metric_function_name = NULL,
metric_function_decreasing = TRUE,
engine = NULL,
tuning = NULL
)A list of models trained by `train_models()` function.
A list of predictions of every engine from the test data.
A vector of true values from the test data.
A string, determines if the future task is `binary_clf` or `regression`.
A vector of metrics names. By default param set for `auto`, most important metrics are returned. For `all` all metrics are returned. For `NULL` no metrics returned but still sorted by `sort_by`.
String with name of metric to sort by. For `auto` models going to be sorted by `mse` for regression and `f1` for classification.
The self-created function. It should look like name(predictions, observed) and return the numeric value. In case of using `metrics` param with value other than `auto` or `all`, is needed to use value `metric_function` in order to see given metric in report. If `sort_by` is equal to `auto` models are sorted by `metric_function`.
The name of the column with values of param `metric_function`. By default `metric_function_name` is `metric_function`.
A logical value indicating how metric_function should be sorted. `TRUE` by default.
A vector of strings containing information of engine in `models` list.
A vector of strings containing information of tuning method in `models` list.
A data.frame with 'no.' - number of model from models, 'engine' - name of the model from models, other metrics columns.
iris_bin <- iris[1:100, ]
iris_bin$Species <- factor(iris_bin$Species)
type <- guess_type(iris_bin, 'Species')
preprocessed_data <- preprocessing(iris_bin, 'Species', type)
#> Error in if (advanced) { del_cor <- delete_correlated_values(pre_data, y, verbose = verbose) pre_data <- del_cor$data pre_data <- delete_id_columns(pre_data) pre_data <- boruta_selection(pre_data, y)}: argument is not interpretable as logical
preprocessed_data <- preprocessed_data$data
#> Error in eval(expr, envir, enclos): object 'preprocessed_data' not found
split_data <-
train_test_balance(preprocessed_data,
'Species',
balance = FALSE)
#> Error in train_test_balance(preprocessed_data, "Species", balance = FALSE): object 'preprocessed_data' not found
train_data <-
prepare_data(split_data$train,
'Species',
engine = c('ranger', 'xgboost', 'decision_tree', 'lightgbm', 'catboost'))
#> Error in as.data.frame(unclass(data), stringsAsFactors = TRUE): object 'split_data' not found
test_data <-
prepare_data(split_data$test,
'Species',
engine = c('ranger', 'xgboost', 'decision_tree', 'lightgbm', 'catboost'),
predict = TRUE,
train = split_data$train)
#> Error in as.data.frame(unclass(data), stringsAsFactors = TRUE): object 'split_data' not found
suppressWarnings(
model <-
train_models(train_data,
'Species',
engine = c('ranger', 'xgboost', 'decision_tree', 'lightgbm', 'catboost'),
type = type)
)
#> Error in ranger::ranger(dependent.variable.name = y, data = data$ranger_data, classification = TRUE, probability = TRUE): object 'train_data' not found
predictions <-
predict_models(model,
test_data,
'Species',
engine = c('ranger', 'xgboost', 'decision_tree', 'lightgbm', 'catboost'),
type = type)
#> Error in predict(models[[i]], data$ranger_data): object 'model' not found
score <-
score_models(model,
predictions,
observed = split_data$test$Species,
type = type)
#> Error in matrix(nrow = length(models), ncol = length(metrics_binary_clf) + nr_add_col): object 'model' not found