r - AUC metrics on XGBoost -
i build model prediction xgboost:
setdt(train) setdt(test) labels <- train$goal ts_label <- test$goal new_tr <- model.matrix(~.+0,data = train[,-c("goal"),with=f]) new_ts <- model.matrix(~.+0,data = test[,-c("goal"),with=f]) labels <- as.numeric(labels)-1 ts_label <- as.numeric(ts_label)-1 dtrain <- xgb.dmatrix(data = new_tr,label = labels) dtest <- xgb.dmatrix(data = new_ts,label=ts_label) params <- list(booster = "gbtree", objective = "binary:logistic", eta=0.3, gamma=0, max_depth=6, min_child_weight=1, subsample=1, colsample_bytree=1) xgb1 <- xgb.train(params = params, data = dtrain, nrounds = 291, watchlist = list(val=dtest,train=dtrain), print_every_n = 10, early_stop_round = 10, maximize = f , eval_metric = "error") xgbpred <- predict(xgb1,dtest) xgbpred <- ifelse(xgbpred > 0.5,1,0) confusionmatrix(xgbpred, ts_label) confusion matrix , statistics reference prediction 0 1 0 1904 70 1 191 2015 accuracy : 0.9376 95% ci : (0.9298, 0.9447) no information rate : 0.5012 p-value [acc > nir] : < 0.00000000000000022 kappa : 0.8751 mcnemar's test p-value : 0.0000000000001104 sensitivity : 0.9088 specificity : 0.9664 pos pred value : 0.9645 neg pred value : 0.9134 prevalence : 0.5012 detection rate : 0.4555 detection prevalence : 0.4722 balanced accuracy : 0.9376 'positive' class : 0
this accuracy suits me, want check metric of auc. write:
xgb1 <- xgb.train(params = params, data = dtrain, nrounds = 291, watchlist = list(val=dtest,train=dtrain), print_every_n = 10, early_stop_round = 10, maximize = f , eval_metric = "auc")
but after don't know how make prediction concerning auc metrics. need help, because first experience xgboost. thanks.
upd: far understand, after auc metric need coefficient cut classes. cut off in 0,5
i edit code:
you can directly confussion matrix:
cm<-confusionmatrix(xgbpred, ts_label)$table t = cm[1,1]/(cm[1,1]+cm[2,1]) f = cm[2,2]/(cm[2,1]+cm[2,2]) auc = (1+t-f)/2
wiki
Comments
Post a Comment