Calulate EPP score for all models
calculate_elo( results, decreasing_metric = TRUE, compare_in_split = TRUE, keep_columns = FALSE, keep_model = FALSE, reference = NULL, keep_data = TRUE, estimation = "glmnet" )
| results | data frame with results for one dataset. Data should be in the following format. First 3 columns should correspond to: model, split, score. See more in 'details' section. |
|---|---|
| decreasing_metric | Logical. If TRUE used metric is considered as decreasing, that means a model with higher score value is considered as better model. If FALSE used metric will be considered as increasing. |
| compare_in_split | Logical. If TRUE compares models only in the same fold. If FALSE compares models across folds. |
| keep_columns | Logical. If TRUE original data frame with new 'epp' column will be returned. |
| keep_model | Logical. If TRUE logistic regression model to compute EPP will be returned. |
| reference | Model that should be a reference level for EPP scores. It should be a name of one of the models from 'results' data frame. If NULL, none of the models will be chosen. |
| keep_data | If all the meta-data shoul be keept in result. |
| estimation | Method of estimating elo coefficients, 'glm' or 'glmnet'. |
Format of the data frame passed via results parameter.
First column should correspond to a model. Dofferent settings of hyperparameters of the same model should have different values in this column.
Second column corresponds to indexes of splits. As EPP is based on Elo rating system, power of model is assessed by comparing its results
with other models on multiple data splits. Therefore, each model should be evaluated on multiple train-test splits.
Indexes of splits should be in this column. And should match across models.
Third column contains score used to evaluate models. It can be both decreasing or increasing metric,
just remember to set the decreasing_metric parameter accordingly.
The following columns can be of any kind.
#> | | | 0% | | | 1% | |= | 1% | |= | 2% | |== | 2% | |== | 3% | |== | 4% | |=== | 4% | |=== | 5% | |==== | 5% | |==== | 6% | |===== | 7% | |===== | 8% | |====== | 8% | |====== | 9% | |======= | 10% | |======= | 11% | |======== | 11% | |======== | 12% | |========= | 12% | |========= | 13% | |========= | 14% | |========== | 14% | |========== | 15% | |=========== | 15% | |=========== | 16% | |============ | 17% | |============ | 18% | |============= | 18% | |============= | 19% | |============== | 19% | |============== | 20% | |============== | 21% | |=============== | 21% | |=============== | 22% | |================ | 22% | |================ | 23% | |================ | 24% | |================= | 24% | |================= | 25% | |================== | 25% | |================== | 26% | |=================== | 27% | |=================== | 28% | |==================== | 28% | |==================== | 29% | |===================== | 29% | |===================== | 30% | |===================== | 31% | |====================== | 31% | |====================== | 32% | |======================= | 32% | |======================= | 33% | |======================== | 34% | |======================== | 35% | |========================= | 35% | |========================= | 36% | |========================== | 37% | |========================== | 38% | |=========================== | 38% | |=========================== | 39% | |============================ | 39% | |============================ | 40% | |============================ | 41% | |============================= | 41% | |============================= | 42% | |============================== | 42% | |============================== | 43% | |=============================== | 44% | |=============================== | 45% | |================================ | 45% | |================================ | 46% | |================================= | 47% | |================================= | 48% | |================================== | 48% | |================================== | 49% | |=================================== | 49% | |=================================== | 50% | |=================================== | 51% | |==================================== | 51% | |==================================== | 52% | |===================================== | 52% | |===================================== | 53% | |====================================== | 54% | |====================================== | 55% | |======================================= | 55% | |======================================= | 56% | |======================================== | 57% | |======================================== | 58% | |========================================= | 58% | |========================================= | 59% | |========================================== | 59% | |========================================== | 60% | |========================================== | 61% | |=========================================== | 61% | |=========================================== | 62% | |============================================ | 62% | |============================================ | 63% | |============================================= | 64% | |============================================= | 65% | |============================================== | 65% | |============================================== | 66% | |=============================================== | 67% | |=============================================== | 68% | |================================================ | 68% | |================================================ | 69% | |================================================= | 69% | |================================================= | 70% | |================================================= | 71% | |================================================== | 71% | |================================================== | 72% | |=================================================== | 72% | |=================================================== | 73% | |==================================================== | 74% | |==================================================== | 75% | |===================================================== | 75% | |===================================================== | 76% | |====================================================== | 76% | |====================================================== | 77% | |====================================================== | 78% | |======================================================= | 78% | |======================================================= | 79% | |======================================================== | 79% | |======================================================== | 80% | |======================================================== | 81% | |========================================================= | 81% | |========================================================= | 82% | |========================================================== | 82% | |========================================================== | 83% | |=========================================================== | 84% | |=========================================================== | 85% | |============================================================ | 85% | |============================================================ | 86% | |============================================================= | 86% | |============================================================= | 87% | |============================================================= | 88% | |============================================================== | 88% | |============================================================== | 89% | |=============================================================== | 89% | |=============================================================== | 90% | |================================================================ | 91% | |================================================================ | 92% | |================================================================= | 92% | |================================================================= | 93% | |================================================================== | 94% | |================================================================== | 95% | |=================================================================== | 95% | |=================================================================== | 96% | |==================================================================== | 96% | |==================================================================== | 97% | |==================================================================== | 98% | |===================================================================== | 98% | |===================================================================== | 99% | |======================================================================| 99% | |======================================================================| 100% #> | | | 0% | |======================================================================| 100%#> $epp #> model epp #> 1 catboost_1 -1.1629124 #> 2 catboost_2 2.6224285 #> 3 catboost_3 -3.0180459 #> 4 gbm_1 -62.4999305 #> 5 gbm_10 1.4834819 #> 6 gbm_11 2.6120344 #> 7 gbm_2 0.9935102 #> 8 gbm_3 -0.6756740 #> 9 gbm_4 -49.4795242 #> 10 gbm_5 -25.6438440 #> 11 gbm_6 -37.5579575 #> 12 gbm_7 0.8349048 #> 13 gbm_8 -14.4927770 #> 14 gbm_9 0.8914450 #> 15 randomForest_1 2.1560226 #> 16 randomForest_2 1.5532968 #> 17 randomForest_3 1.0210257 #> 18 randomForest_4 2.2675649 #> 19 randomForest_5 2.0558015 #> 20 randomForest_6 0.0000000 #> #> $residual_deviance #> $residual_deviance$value #> [1] 198.2772 #> #> $residual_deviance$df #> NULL #> #> #> $actual_score #> winner loser match loses wins #> 1 catboost_1 catboost_2 20 19.0 1.0 #> 2 catboost_1 catboost_3 20 1.0 19.0 #> 3 catboost_1 gbm_1 20 0.0 20.0 #> 4 catboost_1 gbm_10 20 17.0 3.0 #> 5 catboost_1 gbm_11 20 20.0 0.0 #> 6 catboost_1 gbm_2 20 18.0 2.0 #> 7 catboost_1 gbm_3 20 15.0 5.0 #> 8 catboost_1 gbm_4 20 0.0 20.0 #> 9 catboost_1 gbm_5 20 0.0 20.0 #> 10 catboost_1 gbm_6 20 0.0 20.0 #> 11 catboost_1 gbm_7 20 19.0 1.0 #> 12 catboost_1 gbm_8 20 0.0 20.0 #> 13 catboost_1 gbm_9 20 16.0 4.0 #> 14 catboost_1 randomForest_1 20 20.0 0.0 #> 15 catboost_1 randomForest_2 20 19.0 1.0 #> 16 catboost_1 randomForest_3 20 19.0 1.0 #> 17 catboost_1 randomForest_4 20 19.0 1.0 #> 18 catboost_1 randomForest_5 20 19.0 1.0 #> 19 catboost_1 randomForest_6 20 19.0 1.0 #> 20 catboost_2 catboost_1 20 1.0 19.0 #> 21 catboost_2 catboost_3 20 0.0 20.0 #> 22 catboost_2 gbm_1 20 0.0 20.0 #> 23 catboost_2 gbm_10 20 7.0 13.0 #> 24 catboost_2 gbm_11 20 9.0 11.0 #> 25 catboost_2 gbm_2 20 4.0 16.0 #> 26 catboost_2 gbm_3 20 1.0 19.0 #> 27 catboost_2 gbm_4 20 0.0 20.0 #> 28 catboost_2 gbm_5 20 0.0 20.0 #> 29 catboost_2 gbm_6 20 0.0 20.0 #> 30 catboost_2 gbm_7 20 4.0 16.0 #> 31 catboost_2 gbm_8 20 0.0 20.0 #> 32 catboost_2 gbm_9 20 4.0 16.0 #> 33 catboost_2 randomForest_1 20 6.0 14.0 #> 34 catboost_2 randomForest_2 20 5.0 15.0 #> 35 catboost_2 randomForest_3 20 5.0 15.0 #> 36 catboost_2 randomForest_4 20 6.0 14.0 #> 37 catboost_2 randomForest_5 20 6.0 14.0 #> 38 catboost_2 randomForest_6 20 6.0 14.0 #> 39 catboost_3 catboost_1 20 19.0 1.0 #> 40 catboost_3 catboost_2 20 20.0 0.0 #> 41 catboost_3 gbm_1 20 0.0 20.0 #> 42 catboost_3 gbm_10 20 20.0 0.0 #> 43 catboost_3 gbm_11 20 20.0 0.0 #> 44 catboost_3 gbm_2 20 19.0 1.0 #> 45 catboost_3 gbm_3 20 19.0 1.0 #> 46 catboost_3 gbm_4 20 0.0 20.0 #> 47 catboost_3 gbm_5 20 0.0 20.0 #> 48 catboost_3 gbm_6 20 0.0 20.0 #> 49 catboost_3 gbm_7 20 19.0 1.0 #> 50 catboost_3 gbm_8 20 0.0 20.0 #> 51 catboost_3 gbm_9 20 18.0 2.0 #> 52 catboost_3 randomForest_1 20 20.0 0.0 #> 53 catboost_3 randomForest_2 20 19.0 1.0 #> 54 catboost_3 randomForest_3 20 20.0 0.0 #> 55 catboost_3 randomForest_4 20 20.0 0.0 #> 56 catboost_3 randomForest_5 20 20.0 0.0 #> 57 catboost_3 randomForest_6 20 20.0 0.0 #> 58 gbm_1 catboost_1 20 20.0 0.0 #> 59 gbm_1 catboost_2 20 20.0 0.0 #> 60 gbm_1 catboost_3 20 20.0 0.0 #> 61 gbm_1 gbm_10 20 20.0 0.0 #> 62 gbm_1 gbm_11 20 20.0 0.0 #> 63 gbm_1 gbm_2 20 20.0 0.0 #> 64 gbm_1 gbm_3 20 20.0 0.0 #> 65 gbm_1 gbm_4 20 20.0 0.0 #> 66 gbm_1 gbm_5 20 20.0 0.0 #> 67 gbm_1 gbm_6 20 20.0 0.0 #> 68 gbm_1 gbm_7 20 20.0 0.0 #> 69 gbm_1 gbm_8 20 20.0 0.0 #> 70 gbm_1 gbm_9 20 20.0 0.0 #> 71 gbm_1 randomForest_1 20 20.0 0.0 #> 72 gbm_1 randomForest_2 20 20.0 0.0 #> 73 gbm_1 randomForest_3 20 20.0 0.0 #> 74 gbm_1 randomForest_4 20 20.0 0.0 #> 75 gbm_1 randomForest_5 20 20.0 0.0 #> 76 gbm_1 randomForest_6 20 20.0 0.0 #> 77 gbm_10 catboost_1 20 3.0 17.0 #> 78 gbm_10 catboost_2 20 13.0 7.0 #> 79 gbm_10 catboost_3 20 0.0 20.0 #> 80 gbm_10 gbm_1 20 0.0 20.0 #> 81 gbm_10 gbm_11 20 17.0 3.0 #> 82 gbm_10 gbm_2 20 7.0 13.0 #> 83 gbm_10 gbm_3 20 2.0 18.0 #> 84 gbm_10 gbm_4 20 0.0 20.0 #> 85 gbm_10 gbm_5 20 0.0 20.0 #> 86 gbm_10 gbm_6 20 0.0 20.0 #> 87 gbm_10 gbm_7 20 6.0 14.0 #> 88 gbm_10 gbm_8 20 0.0 20.0 #> 89 gbm_10 gbm_9 20 6.0 14.0 #> 90 gbm_10 randomForest_1 20 12.0 8.0 #> 91 gbm_10 randomForest_2 20 12.0 8.0 #> 92 gbm_10 randomForest_3 20 9.0 11.0 #> 93 gbm_10 randomForest_4 20 12.0 8.0 #> 94 gbm_10 randomForest_5 20 14.0 6.0 #> 95 gbm_10 randomForest_6 20 13.0 7.0 #> 96 gbm_11 catboost_1 20 0.0 20.0 #> 97 gbm_11 catboost_2 20 11.0 9.0 #> 98 gbm_11 catboost_3 20 0.0 20.0 #> 99 gbm_11 gbm_1 20 0.0 20.0 #> 100 gbm_11 gbm_10 20 3.0 17.0 #> 101 gbm_11 gbm_2 20 1.5 18.5 #> 102 gbm_11 gbm_3 20 0.0 20.0 #> 103 gbm_11 gbm_4 20 0.0 20.0 #> 104 gbm_11 gbm_5 20 0.0 20.0 #> 105 gbm_11 gbm_6 20 0.0 20.0 #> 106 gbm_11 gbm_7 20 1.0 19.0 #> 107 gbm_11 gbm_8 20 0.0 20.0 #> 108 gbm_11 gbm_9 20 2.0 18.0 #> 109 gbm_11 randomForest_1 20 9.0 11.0 #> 110 gbm_11 randomForest_2 20 6.0 14.0 #> 111 gbm_11 randomForest_3 20 5.0 15.0 #> 112 gbm_11 randomForest_4 20 9.0 11.0 #> 113 gbm_11 randomForest_5 20 10.0 10.0 #> 114 gbm_11 randomForest_6 20 7.0 13.0 #> 115 gbm_2 catboost_1 20 2.0 18.0 #> 116 gbm_2 catboost_2 20 16.0 4.0 #> 117 gbm_2 catboost_3 20 1.0 19.0 #> 118 gbm_2 gbm_1 20 0.0 20.0 #> 119 gbm_2 gbm_10 20 13.0 7.0 #> 120 gbm_2 gbm_11 20 18.5 1.5 #> 121 gbm_2 gbm_3 20 1.0 19.0 #> 122 gbm_2 gbm_4 20 0.0 20.0 #> 123 gbm_2 gbm_5 20 0.0 20.0 #> 124 gbm_2 gbm_6 20 0.0 20.0 #> 125 gbm_2 gbm_7 20 9.0 11.0 #> 126 gbm_2 gbm_8 20 0.0 20.0 #> 127 gbm_2 gbm_9 20 9.0 11.0 #> 128 gbm_2 randomForest_1 20 14.0 6.0 #> 129 gbm_2 randomForest_2 20 14.0 6.0 #> 130 gbm_2 randomForest_3 20 14.0 6.0 #> 131 gbm_2 randomForest_4 20 14.0 6.0 #> 132 gbm_2 randomForest_5 20 14.0 6.0 #> 133 gbm_2 randomForest_6 20 14.0 6.0 #> 134 gbm_3 catboost_1 20 5.0 15.0 #> 135 gbm_3 catboost_2 20 19.0 1.0 #> 136 gbm_3 catboost_3 20 1.0 19.0 #> 137 gbm_3 gbm_1 20 0.0 20.0 #> 138 gbm_3 gbm_10 20 18.0 2.0 #> 139 gbm_3 gbm_11 20 20.0 0.0 #> 140 gbm_3 gbm_2 20 19.0 1.0 #> 141 gbm_3 gbm_4 20 0.0 20.0 #> 142 gbm_3 gbm_5 20 0.0 20.0 #> 143 gbm_3 gbm_6 20 0.0 20.0 #> 144 gbm_3 gbm_7 20 20.0 0.0 #> 145 gbm_3 gbm_8 20 0.0 20.0 #> 146 gbm_3 gbm_9 20 18.0 2.0 #> 147 gbm_3 randomForest_1 20 17.0 3.0 #> 148 gbm_3 randomForest_2 20 17.0 3.0 #> 149 gbm_3 randomForest_3 20 18.0 2.0 #> 150 gbm_3 randomForest_4 20 18.0 2.0 #> 151 gbm_3 randomForest_5 20 18.0 2.0 #> 152 gbm_3 randomForest_6 20 18.0 2.0 #> 153 gbm_4 catboost_1 20 20.0 0.0 #> 154 gbm_4 catboost_2 20 20.0 0.0 #> 155 gbm_4 catboost_3 20 20.0 0.0 #> 156 gbm_4 gbm_1 20 0.0 20.0 #> 157 gbm_4 gbm_10 20 20.0 0.0 #> 158 gbm_4 gbm_11 20 20.0 0.0 #> 159 gbm_4 gbm_2 20 20.0 0.0 #> 160 gbm_4 gbm_3 20 20.0 0.0 #> 161 gbm_4 gbm_5 20 20.0 0.0 #> 162 gbm_4 gbm_6 20 20.0 0.0 #> 163 gbm_4 gbm_7 20 20.0 0.0 #> 164 gbm_4 gbm_8 20 20.0 0.0 #> 165 gbm_4 gbm_9 20 20.0 0.0 #> 166 gbm_4 randomForest_1 20 20.0 0.0 #> 167 gbm_4 randomForest_2 20 20.0 0.0 #> 168 gbm_4 randomForest_3 20 20.0 0.0 #> 169 gbm_4 randomForest_4 20 20.0 0.0 #> 170 gbm_4 randomForest_5 20 20.0 0.0 #> 171 gbm_4 randomForest_6 20 20.0 0.0 #> 172 gbm_5 catboost_1 20 20.0 0.0 #> 173 gbm_5 catboost_2 20 20.0 0.0 #> 174 gbm_5 catboost_3 20 20.0 0.0 #> 175 gbm_5 gbm_1 20 0.0 20.0 #> 176 gbm_5 gbm_10 20 20.0 0.0 #> 177 gbm_5 gbm_11 20 20.0 0.0 #> 178 gbm_5 gbm_2 20 20.0 0.0 #> 179 gbm_5 gbm_3 20 20.0 0.0 #> 180 gbm_5 gbm_4 20 0.0 20.0 #> 181 gbm_5 gbm_6 20 0.0 20.0 #> 182 gbm_5 gbm_7 20 20.0 0.0 #> 183 gbm_5 gbm_8 20 20.0 0.0 #> 184 gbm_5 gbm_9 20 20.0 0.0 #> 185 gbm_5 randomForest_1 20 20.0 0.0 #> 186 gbm_5 randomForest_2 20 20.0 0.0 #> 187 gbm_5 randomForest_3 20 20.0 0.0 #> 188 gbm_5 randomForest_4 20 20.0 0.0 #> 189 gbm_5 randomForest_5 20 20.0 0.0 #> 190 gbm_5 randomForest_6 20 20.0 0.0 #> 191 gbm_6 catboost_1 20 20.0 0.0 #> 192 gbm_6 catboost_2 20 20.0 0.0 #> 193 gbm_6 catboost_3 20 20.0 0.0 #> 194 gbm_6 gbm_1 20 0.0 20.0 #> 195 gbm_6 gbm_10 20 20.0 0.0 #> 196 gbm_6 gbm_11 20 20.0 0.0 #> 197 gbm_6 gbm_2 20 20.0 0.0 #> 198 gbm_6 gbm_3 20 20.0 0.0 #> 199 gbm_6 gbm_4 20 0.0 20.0 #> 200 gbm_6 gbm_5 20 20.0 0.0 #> 201 gbm_6 gbm_7 20 20.0 0.0 #> 202 gbm_6 gbm_8 20 20.0 0.0 #> 203 gbm_6 gbm_9 20 20.0 0.0 #> 204 gbm_6 randomForest_1 20 20.0 0.0 #> 205 gbm_6 randomForest_2 20 20.0 0.0 #> 206 gbm_6 randomForest_3 20 20.0 0.0 #> 207 gbm_6 randomForest_4 20 20.0 0.0 #> 208 gbm_6 randomForest_5 20 20.0 0.0 #> 209 gbm_6 randomForest_6 20 20.0 0.0 #> 210 gbm_7 catboost_1 20 1.0 19.0 #> 211 gbm_7 catboost_2 20 16.0 4.0 #> 212 gbm_7 catboost_3 20 1.0 19.0 #> 213 gbm_7 gbm_1 20 0.0 20.0 #> 214 gbm_7 gbm_10 20 14.0 6.0 #> 215 gbm_7 gbm_11 20 19.0 1.0 #> 216 gbm_7 gbm_2 20 11.0 9.0 #> 217 gbm_7 gbm_3 20 0.0 20.0 #> 218 gbm_7 gbm_4 20 0.0 20.0 #> 219 gbm_7 gbm_5 20 0.0 20.0 #> 220 gbm_7 gbm_6 20 0.0 20.0 #> 221 gbm_7 gbm_8 20 0.0 20.0 #> 222 gbm_7 gbm_9 20 10.0 10.0 #> 223 gbm_7 randomForest_1 20 15.0 5.0 #> 224 gbm_7 randomForest_2 20 15.0 5.0 #> 225 gbm_7 randomForest_3 20 15.0 5.0 #> 226 gbm_7 randomForest_4 20 15.0 5.0 #> 227 gbm_7 randomForest_5 20 15.0 5.0 #> 228 gbm_7 randomForest_6 20 15.0 5.0 #> 229 gbm_8 catboost_1 20 20.0 0.0 #> 230 gbm_8 catboost_2 20 20.0 0.0 #> 231 gbm_8 catboost_3 20 20.0 0.0 #> 232 gbm_8 gbm_1 20 0.0 20.0 #> 233 gbm_8 gbm_10 20 20.0 0.0 #> 234 gbm_8 gbm_11 20 20.0 0.0 #> 235 gbm_8 gbm_2 20 20.0 0.0 #> 236 gbm_8 gbm_3 20 20.0 0.0 #> 237 gbm_8 gbm_4 20 0.0 20.0 #> 238 gbm_8 gbm_5 20 0.0 20.0 #> 239 gbm_8 gbm_6 20 0.0 20.0 #> 240 gbm_8 gbm_7 20 20.0 0.0 #> 241 gbm_8 gbm_9 20 20.0 0.0 #> 242 gbm_8 randomForest_1 20 20.0 0.0 #> 243 gbm_8 randomForest_2 20 20.0 0.0 #> 244 gbm_8 randomForest_3 20 20.0 0.0 #> 245 gbm_8 randomForest_4 20 20.0 0.0 #> 246 gbm_8 randomForest_5 20 20.0 0.0 #> 247 gbm_8 randomForest_6 20 20.0 0.0 #> 248 gbm_9 catboost_1 20 4.0 16.0 #> 249 gbm_9 catboost_2 20 16.0 4.0 #> 250 gbm_9 catboost_3 20 2.0 18.0 #> 251 gbm_9 gbm_1 20 0.0 20.0 #> 252 gbm_9 gbm_10 20 14.0 6.0 #> 253 gbm_9 gbm_11 20 18.0 2.0 #> 254 gbm_9 gbm_2 20 11.0 9.0 #> 255 gbm_9 gbm_3 20 2.0 18.0 #> 256 gbm_9 gbm_4 20 0.0 20.0 #> 257 gbm_9 gbm_5 20 0.0 20.0 #> 258 gbm_9 gbm_6 20 0.0 20.0 #> 259 gbm_9 gbm_7 20 10.0 10.0 #> 260 gbm_9 gbm_8 20 0.0 20.0 #> 261 gbm_9 randomForest_1 20 14.0 6.0 #> 262 gbm_9 randomForest_2 20 14.0 6.0 #> 263 gbm_9 randomForest_3 20 12.0 8.0 #> 264 gbm_9 randomForest_4 20 14.0 6.0 #> 265 gbm_9 randomForest_5 20 14.0 6.0 #> 266 gbm_9 randomForest_6 20 14.0 6.0 #> 267 randomForest_1 catboost_1 20 0.0 20.0 #> 268 randomForest_1 catboost_2 20 14.0 6.0 #> 269 randomForest_1 catboost_3 20 0.0 20.0 #> 270 randomForest_1 gbm_1 20 0.0 20.0 #> 271 randomForest_1 gbm_10 20 8.0 12.0 #> 272 randomForest_1 gbm_11 20 11.0 9.0 #> 273 randomForest_1 gbm_2 20 6.0 14.0 #> 274 randomForest_1 gbm_3 20 3.0 17.0 #> 275 randomForest_1 gbm_4 20 0.0 20.0 #> 276 randomForest_1 gbm_5 20 0.0 20.0 #> 277 randomForest_1 gbm_6 20 0.0 20.0 #> 278 randomForest_1 gbm_7 20 5.0 15.0 #> 279 randomForest_1 gbm_8 20 0.0 20.0 #> 280 randomForest_1 gbm_9 20 6.0 14.0 #> 281 randomForest_1 randomForest_2 20 4.0 16.0 #> 282 randomForest_1 randomForest_3 20 2.0 18.0 #> 283 randomForest_1 randomForest_4 20 11.0 9.0 #> 284 randomForest_1 randomForest_5 20 9.0 11.0 #> 285 randomForest_1 randomForest_6 20 9.0 11.0 #> 286 randomForest_2 catboost_1 20 1.0 19.0 #> 287 randomForest_2 catboost_2 20 15.0 5.0 #> 288 randomForest_2 catboost_3 20 1.0 19.0 #> 289 randomForest_2 gbm_1 20 0.0 20.0 #> 290 randomForest_2 gbm_10 20 8.0 12.0 #> 291 randomForest_2 gbm_11 20 14.0 6.0 #> 292 randomForest_2 gbm_2 20 6.0 14.0 #> 293 randomForest_2 gbm_3 20 3.0 17.0 #> 294 randomForest_2 gbm_4 20 0.0 20.0 #> 295 randomForest_2 gbm_5 20 0.0 20.0 #> 296 randomForest_2 gbm_6 20 0.0 20.0 #> 297 randomForest_2 gbm_7 20 5.0 15.0 #> 298 randomForest_2 gbm_8 20 0.0 20.0 #> 299 randomForest_2 gbm_9 20 6.0 14.0 #> 300 randomForest_2 randomForest_1 20 16.0 4.0 #> 301 randomForest_2 randomForest_3 20 6.0 14.0 #> 302 randomForest_2 randomForest_4 20 16.0 4.0 #> 303 randomForest_2 randomForest_5 20 12.0 8.0 #> 304 randomForest_2 randomForest_6 20 13.0 7.0 #> 305 randomForest_3 catboost_1 20 1.0 19.0 #> 306 randomForest_3 catboost_2 20 15.0 5.0 #> 307 randomForest_3 catboost_3 20 0.0 20.0 #> 308 randomForest_3 gbm_1 20 0.0 20.0 #> 309 randomForest_3 gbm_10 20 11.0 9.0 #> 310 randomForest_3 gbm_11 20 15.0 5.0 #> 311 randomForest_3 gbm_2 20 6.0 14.0 #> 312 randomForest_3 gbm_3 20 2.0 18.0 #> 313 randomForest_3 gbm_4 20 0.0 20.0 #> 314 randomForest_3 gbm_5 20 0.0 20.0 #> 315 randomForest_3 gbm_6 20 0.0 20.0 #> 316 randomForest_3 gbm_7 20 5.0 15.0 #> 317 randomForest_3 gbm_8 20 0.0 20.0 #> 318 randomForest_3 gbm_9 20 8.0 12.0 #> 319 randomForest_3 randomForest_1 20 18.0 2.0 #> 320 randomForest_3 randomForest_2 20 14.0 6.0 #> 321 randomForest_3 randomForest_4 20 20.0 0.0 #> 322 randomForest_3 randomForest_5 20 17.0 3.0 #> 323 randomForest_3 randomForest_6 20 20.0 0.0 #> 324 randomForest_4 catboost_1 20 1.0 19.0 #> 325 randomForest_4 catboost_2 20 14.0 6.0 #> 326 randomForest_4 catboost_3 20 0.0 20.0 #> 327 randomForest_4 gbm_1 20 0.0 20.0 #> 328 randomForest_4 gbm_10 20 8.0 12.0 #> 329 randomForest_4 gbm_11 20 11.0 9.0 #> 330 randomForest_4 gbm_2 20 6.0 14.0 #> 331 randomForest_4 gbm_3 20 2.0 18.0 #> 332 randomForest_4 gbm_4 20 0.0 20.0 #> 333 randomForest_4 gbm_5 20 0.0 20.0 #> 334 randomForest_4 gbm_6 20 0.0 20.0 #> 335 randomForest_4 gbm_7 20 5.0 15.0 #> 336 randomForest_4 gbm_8 20 0.0 20.0 #> 337 randomForest_4 gbm_9 20 6.0 14.0 #> 338 randomForest_4 randomForest_1 20 9.0 11.0 #> 339 randomForest_4 randomForest_2 20 4.0 16.0 #> 340 randomForest_4 randomForest_3 20 0.0 20.0 #> 341 randomForest_4 randomForest_5 20 9.0 11.0 #> 342 randomForest_4 randomForest_6 20 7.0 13.0 #> 343 randomForest_5 catboost_1 20 1.0 19.0 #> 344 randomForest_5 catboost_2 20 14.0 6.0 #> 345 randomForest_5 catboost_3 20 0.0 20.0 #> 346 randomForest_5 gbm_1 20 0.0 20.0 #> 347 randomForest_5 gbm_10 20 6.0 14.0 #> 348 randomForest_5 gbm_11 20 10.0 10.0 #> 349 randomForest_5 gbm_2 20 6.0 14.0 #> 350 randomForest_5 gbm_3 20 2.0 18.0 #> 351 randomForest_5 gbm_4 20 0.0 20.0 #> 352 randomForest_5 gbm_5 20 0.0 20.0 #> 353 randomForest_5 gbm_6 20 0.0 20.0 #> 354 randomForest_5 gbm_7 20 5.0 15.0 #> 355 randomForest_5 gbm_8 20 0.0 20.0 #> 356 randomForest_5 gbm_9 20 6.0 14.0 #> 357 randomForest_5 randomForest_1 20 11.0 9.0 #> 358 randomForest_5 randomForest_2 20 8.0 12.0 #> 359 randomForest_5 randomForest_3 20 3.0 17.0 #> 360 randomForest_5 randomForest_4 20 11.0 9.0 #> 361 randomForest_5 randomForest_6 20 10.5 9.5 #> 362 randomForest_6 catboost_1 20 1.0 19.0 #> 363 randomForest_6 catboost_2 20 14.0 6.0 #> 364 randomForest_6 catboost_3 20 0.0 20.0 #> 365 randomForest_6 gbm_1 20 0.0 20.0 #> 366 randomForest_6 gbm_10 20 7.0 13.0 #> 367 randomForest_6 gbm_11 20 13.0 7.0 #> 368 randomForest_6 gbm_2 20 6.0 14.0 #> 369 randomForest_6 gbm_3 20 2.0 18.0 #> 370 randomForest_6 gbm_4 20 0.0 20.0 #> 371 randomForest_6 gbm_5 20 0.0 20.0 #> 372 randomForest_6 gbm_6 20 0.0 20.0 #> 373 randomForest_6 gbm_7 20 5.0 15.0 #> 374 randomForest_6 gbm_8 20 0.0 20.0 #> 375 randomForest_6 gbm_9 20 6.0 14.0 #> 376 randomForest_6 randomForest_1 20 11.0 9.0 #> 377 randomForest_6 randomForest_2 20 7.0 13.0 #> 378 randomForest_6 randomForest_3 20 0.0 20.0 #> 379 randomForest_6 randomForest_4 20 13.0 7.0 #> 380 randomForest_6 randomForest_5 20 9.5 10.5 #> players #> 1 catboost_1 catboost_2 #> 2 catboost_1 catboost_3 #> 3 catboost_1 gbm_1 #> 4 catboost_1 gbm_10 #> 5 catboost_1 gbm_11 #> 6 catboost_1 gbm_2 #> 7 catboost_1 gbm_3 #> 8 catboost_1 gbm_4 #> 9 catboost_1 gbm_5 #> 10 catboost_1 gbm_6 #> 11 catboost_1 gbm_7 #> 12 catboost_1 gbm_8 #> 13 catboost_1 gbm_9 #> 14 catboost_1 randomForest_1 #> 15 catboost_1 randomForest_2 #> 16 catboost_1 randomForest_3 #> 17 catboost_1 randomForest_4 #> 18 catboost_1 randomForest_5 #> 19 catboost_1 randomForest_6 #> 20 catboost_2 catboost_1 #> 21 catboost_2 catboost_3 #> 22 catboost_2 gbm_1 #> 23 catboost_2 gbm_10 #> 24 catboost_2 gbm_11 #> 25 catboost_2 gbm_2 #> 26 catboost_2 gbm_3 #> 27 catboost_2 gbm_4 #> 28 catboost_2 gbm_5 #> 29 catboost_2 gbm_6 #> 30 catboost_2 gbm_7 #> 31 catboost_2 gbm_8 #> 32 catboost_2 gbm_9 #> 33 catboost_2 randomForest_1 #> 34 catboost_2 randomForest_2 #> 35 catboost_2 randomForest_3 #> 36 catboost_2 randomForest_4 #> 37 catboost_2 randomForest_5 #> 38 catboost_2 randomForest_6 #> 39 catboost_3 catboost_1 #> 40 catboost_3 catboost_2 #> 41 catboost_3 gbm_1 #> 42 catboost_3 gbm_10 #> 43 catboost_3 gbm_11 #> 44 catboost_3 gbm_2 #> 45 catboost_3 gbm_3 #> 46 catboost_3 gbm_4 #> 47 catboost_3 gbm_5 #> 48 catboost_3 gbm_6 #> 49 catboost_3 gbm_7 #> 50 catboost_3 gbm_8 #> 51 catboost_3 gbm_9 #> 52 catboost_3 randomForest_1 #> 53 catboost_3 randomForest_2 #> 54 catboost_3 randomForest_3 #> 55 catboost_3 randomForest_4 #> 56 catboost_3 randomForest_5 #> 57 catboost_3 randomForest_6 #> 58 gbm_1 catboost_1 #> 59 gbm_1 catboost_2 #> 60 gbm_1 catboost_3 #> 61 gbm_1 gbm_10 #> 62 gbm_1 gbm_11 #> 63 gbm_1 gbm_2 #> 64 gbm_1 gbm_3 #> 65 gbm_1 gbm_4 #> 66 gbm_1 gbm_5 #> 67 gbm_1 gbm_6 #> 68 gbm_1 gbm_7 #> 69 gbm_1 gbm_8 #> 70 gbm_1 gbm_9 #> 71 gbm_1 randomForest_1 #> 72 gbm_1 randomForest_2 #> 73 gbm_1 randomForest_3 #> 74 gbm_1 randomForest_4 #> 75 gbm_1 randomForest_5 #> 76 gbm_1 randomForest_6 #> 77 gbm_10 catboost_1 #> 78 gbm_10 catboost_2 #> 79 gbm_10 catboost_3 #> 80 gbm_10 gbm_1 #> 81 gbm_10 gbm_11 #> 82 gbm_10 gbm_2 #> 83 gbm_10 gbm_3 #> 84 gbm_10 gbm_4 #> 85 gbm_10 gbm_5 #> 86 gbm_10 gbm_6 #> 87 gbm_10 gbm_7 #> 88 gbm_10 gbm_8 #> 89 gbm_10 gbm_9 #> 90 gbm_10 randomForest_1 #> 91 gbm_10 randomForest_2 #> 92 gbm_10 randomForest_3 #> 93 gbm_10 randomForest_4 #> 94 gbm_10 randomForest_5 #> 95 gbm_10 randomForest_6 #> 96 gbm_11 catboost_1 #> 97 gbm_11 catboost_2 #> 98 gbm_11 catboost_3 #> 99 gbm_11 gbm_1 #> 100 gbm_11 gbm_10 #> 101 gbm_11 gbm_2 #> 102 gbm_11 gbm_3 #> 103 gbm_11 gbm_4 #> 104 gbm_11 gbm_5 #> 105 gbm_11 gbm_6 #> 106 gbm_11 gbm_7 #> 107 gbm_11 gbm_8 #> 108 gbm_11 gbm_9 #> 109 gbm_11 randomForest_1 #> 110 gbm_11 randomForest_2 #> 111 gbm_11 randomForest_3 #> 112 gbm_11 randomForest_4 #> 113 gbm_11 randomForest_5 #> 114 gbm_11 randomForest_6 #> 115 gbm_2 catboost_1 #> 116 gbm_2 catboost_2 #> 117 gbm_2 catboost_3 #> 118 gbm_2 gbm_1 #> 119 gbm_2 gbm_10 #> 120 gbm_2 gbm_11 #> 121 gbm_2 gbm_3 #> 122 gbm_2 gbm_4 #> 123 gbm_2 gbm_5 #> 124 gbm_2 gbm_6 #> 125 gbm_2 gbm_7 #> 126 gbm_2 gbm_8 #> 127 gbm_2 gbm_9 #> 128 gbm_2 randomForest_1 #> 129 gbm_2 randomForest_2 #> 130 gbm_2 randomForest_3 #> 131 gbm_2 randomForest_4 #> 132 gbm_2 randomForest_5 #> 133 gbm_2 randomForest_6 #> 134 gbm_3 catboost_1 #> 135 gbm_3 catboost_2 #> 136 gbm_3 catboost_3 #> 137 gbm_3 gbm_1 #> 138 gbm_3 gbm_10 #> 139 gbm_3 gbm_11 #> 140 gbm_3 gbm_2 #> 141 gbm_3 gbm_4 #> 142 gbm_3 gbm_5 #> 143 gbm_3 gbm_6 #> 144 gbm_3 gbm_7 #> 145 gbm_3 gbm_8 #> 146 gbm_3 gbm_9 #> 147 gbm_3 randomForest_1 #> 148 gbm_3 randomForest_2 #> 149 gbm_3 randomForest_3 #> 150 gbm_3 randomForest_4 #> 151 gbm_3 randomForest_5 #> 152 gbm_3 randomForest_6 #> 153 gbm_4 catboost_1 #> 154 gbm_4 catboost_2 #> 155 gbm_4 catboost_3 #> 156 gbm_4 gbm_1 #> 157 gbm_4 gbm_10 #> 158 gbm_4 gbm_11 #> 159 gbm_4 gbm_2 #> 160 gbm_4 gbm_3 #> 161 gbm_4 gbm_5 #> 162 gbm_4 gbm_6 #> 163 gbm_4 gbm_7 #> 164 gbm_4 gbm_8 #> 165 gbm_4 gbm_9 #> 166 gbm_4 randomForest_1 #> 167 gbm_4 randomForest_2 #> 168 gbm_4 randomForest_3 #> 169 gbm_4 randomForest_4 #> 170 gbm_4 randomForest_5 #> 171 gbm_4 randomForest_6 #> 172 gbm_5 catboost_1 #> 173 gbm_5 catboost_2 #> 174 gbm_5 catboost_3 #> 175 gbm_5 gbm_1 #> 176 gbm_5 gbm_10 #> 177 gbm_5 gbm_11 #> 178 gbm_5 gbm_2 #> 179 gbm_5 gbm_3 #> 180 gbm_5 gbm_4 #> 181 gbm_5 gbm_6 #> 182 gbm_5 gbm_7 #> 183 gbm_5 gbm_8 #> 184 gbm_5 gbm_9 #> 185 gbm_5 randomForest_1 #> 186 gbm_5 randomForest_2 #> 187 gbm_5 randomForest_3 #> 188 gbm_5 randomForest_4 #> 189 gbm_5 randomForest_5 #> 190 gbm_5 randomForest_6 #> 191 gbm_6 catboost_1 #> 192 gbm_6 catboost_2 #> 193 gbm_6 catboost_3 #> 194 gbm_6 gbm_1 #> 195 gbm_6 gbm_10 #> 196 gbm_6 gbm_11 #> 197 gbm_6 gbm_2 #> 198 gbm_6 gbm_3 #> 199 gbm_6 gbm_4 #> 200 gbm_6 gbm_5 #> 201 gbm_6 gbm_7 #> 202 gbm_6 gbm_8 #> 203 gbm_6 gbm_9 #> 204 gbm_6 randomForest_1 #> 205 gbm_6 randomForest_2 #> 206 gbm_6 randomForest_3 #> 207 gbm_6 randomForest_4 #> 208 gbm_6 randomForest_5 #> 209 gbm_6 randomForest_6 #> 210 gbm_7 catboost_1 #> 211 gbm_7 catboost_2 #> 212 gbm_7 catboost_3 #> 213 gbm_7 gbm_1 #> 214 gbm_7 gbm_10 #> 215 gbm_7 gbm_11 #> 216 gbm_7 gbm_2 #> 217 gbm_7 gbm_3 #> 218 gbm_7 gbm_4 #> 219 gbm_7 gbm_5 #> 220 gbm_7 gbm_6 #> 221 gbm_7 gbm_8 #> 222 gbm_7 gbm_9 #> 223 gbm_7 randomForest_1 #> 224 gbm_7 randomForest_2 #> 225 gbm_7 randomForest_3 #> 226 gbm_7 randomForest_4 #> 227 gbm_7 randomForest_5 #> 228 gbm_7 randomForest_6 #> 229 gbm_8 catboost_1 #> 230 gbm_8 catboost_2 #> 231 gbm_8 catboost_3 #> 232 gbm_8 gbm_1 #> 233 gbm_8 gbm_10 #> 234 gbm_8 gbm_11 #> 235 gbm_8 gbm_2 #> 236 gbm_8 gbm_3 #> 237 gbm_8 gbm_4 #> 238 gbm_8 gbm_5 #> 239 gbm_8 gbm_6 #> 240 gbm_8 gbm_7 #> 241 gbm_8 gbm_9 #> 242 gbm_8 randomForest_1 #> 243 gbm_8 randomForest_2 #> 244 gbm_8 randomForest_3 #> 245 gbm_8 randomForest_4 #> 246 gbm_8 randomForest_5 #> 247 gbm_8 randomForest_6 #> 248 gbm_9 catboost_1 #> 249 gbm_9 catboost_2 #> 250 gbm_9 catboost_3 #> 251 gbm_9 gbm_1 #> 252 gbm_9 gbm_10 #> 253 gbm_9 gbm_11 #> 254 gbm_9 gbm_2 #> 255 gbm_9 gbm_3 #> 256 gbm_9 gbm_4 #> 257 gbm_9 gbm_5 #> 258 gbm_9 gbm_6 #> 259 gbm_9 gbm_7 #> 260 gbm_9 gbm_8 #> 261 gbm_9 randomForest_1 #> 262 gbm_9 randomForest_2 #> 263 gbm_9 randomForest_3 #> 264 gbm_9 randomForest_4 #> 265 gbm_9 randomForest_5 #> 266 gbm_9 randomForest_6 #> 267 randomForest_1 catboost_1 #> 268 randomForest_1 catboost_2 #> 269 randomForest_1 catboost_3 #> 270 randomForest_1 gbm_1 #> 271 randomForest_1 gbm_10 #> 272 randomForest_1 gbm_11 #> 273 randomForest_1 gbm_2 #> 274 randomForest_1 gbm_3 #> 275 randomForest_1 gbm_4 #> 276 randomForest_1 gbm_5 #> 277 randomForest_1 gbm_6 #> 278 randomForest_1 gbm_7 #> 279 randomForest_1 gbm_8 #> 280 randomForest_1 gbm_9 #> 281 randomForest_1 randomForest_2 #> 282 randomForest_1 randomForest_3 #> 283 randomForest_1 randomForest_4 #> 284 randomForest_1 randomForest_5 #> 285 randomForest_1 randomForest_6 #> 286 randomForest_2 catboost_1 #> 287 randomForest_2 catboost_2 #> 288 randomForest_2 catboost_3 #> 289 randomForest_2 gbm_1 #> 290 randomForest_2 gbm_10 #> 291 randomForest_2 gbm_11 #> 292 randomForest_2 gbm_2 #> 293 randomForest_2 gbm_3 #> 294 randomForest_2 gbm_4 #> 295 randomForest_2 gbm_5 #> 296 randomForest_2 gbm_6 #> 297 randomForest_2 gbm_7 #> 298 randomForest_2 gbm_8 #> 299 randomForest_2 gbm_9 #> 300 randomForest_2 randomForest_1 #> 301 randomForest_2 randomForest_3 #> 302 randomForest_2 randomForest_4 #> 303 randomForest_2 randomForest_5 #> 304 randomForest_2 randomForest_6 #> 305 randomForest_3 catboost_1 #> 306 randomForest_3 catboost_2 #> 307 randomForest_3 catboost_3 #> 308 randomForest_3 gbm_1 #> 309 randomForest_3 gbm_10 #> 310 randomForest_3 gbm_11 #> 311 randomForest_3 gbm_2 #> 312 randomForest_3 gbm_3 #> 313 randomForest_3 gbm_4 #> 314 randomForest_3 gbm_5 #> 315 randomForest_3 gbm_6 #> 316 randomForest_3 gbm_7 #> 317 randomForest_3 gbm_8 #> 318 randomForest_3 gbm_9 #> 319 randomForest_3 randomForest_1 #> 320 randomForest_3 randomForest_2 #> 321 randomForest_3 randomForest_4 #> 322 randomForest_3 randomForest_5 #> 323 randomForest_3 randomForest_6 #> 324 randomForest_4 catboost_1 #> 325 randomForest_4 catboost_2 #> 326 randomForest_4 catboost_3 #> 327 randomForest_4 gbm_1 #> 328 randomForest_4 gbm_10 #> 329 randomForest_4 gbm_11 #> 330 randomForest_4 gbm_2 #> 331 randomForest_4 gbm_3 #> 332 randomForest_4 gbm_4 #> 333 randomForest_4 gbm_5 #> 334 randomForest_4 gbm_6 #> 335 randomForest_4 gbm_7 #> 336 randomForest_4 gbm_8 #> 337 randomForest_4 gbm_9 #> 338 randomForest_4 randomForest_1 #> 339 randomForest_4 randomForest_2 #> 340 randomForest_4 randomForest_3 #> 341 randomForest_4 randomForest_5 #> 342 randomForest_4 randomForest_6 #> 343 randomForest_5 catboost_1 #> 344 randomForest_5 catboost_2 #> 345 randomForest_5 catboost_3 #> 346 randomForest_5 gbm_1 #> 347 randomForest_5 gbm_10 #> 348 randomForest_5 gbm_11 #> 349 randomForest_5 gbm_2 #> 350 randomForest_5 gbm_3 #> 351 randomForest_5 gbm_4 #> 352 randomForest_5 gbm_5 #> 353 randomForest_5 gbm_6 #> 354 randomForest_5 gbm_7 #> 355 randomForest_5 gbm_8 #> 356 randomForest_5 gbm_9 #> 357 randomForest_5 randomForest_1 #> 358 randomForest_5 randomForest_2 #> 359 randomForest_5 randomForest_3 #> 360 randomForest_5 randomForest_4 #> 361 randomForest_5 randomForest_6 #> 362 randomForest_6 catboost_1 #> 363 randomForest_6 catboost_2 #> 364 randomForest_6 catboost_3 #> 365 randomForest_6 gbm_1 #> 366 randomForest_6 gbm_10 #> 367 randomForest_6 gbm_11 #> 368 randomForest_6 gbm_2 #> 369 randomForest_6 gbm_3 #> 370 randomForest_6 gbm_4 #> 371 randomForest_6 gbm_5 #> 372 randomForest_6 gbm_6 #> 373 randomForest_6 gbm_7 #> 374 randomForest_6 gbm_8 #> 375 randomForest_6 gbm_9 #> 376 randomForest_6 randomForest_1 #> 377 randomForest_6 randomForest_2 #> 378 randomForest_6 randomForest_3 #> 379 randomForest_6 randomForest_4 #> 380 randomForest_6 randomForest_5 #> #> $estimation #> [1] "glmnet" #> #> attr(,"class") #> [1] "elo_results" "list"