XGBoost Best Iteration

Question

I am running a regression using the XGBoost Algorithm as,

clf = XGBRegressor(eval_set = [(X_train, y_train), (X_val, y_val)],
                       early_st         


        

Answer 1

Your error is that the booster attribute of XGBRegressor is a string that specifies the kind of booster to use, not the actual booster instance. From the docs:

booster: string
Specify which booster to use: gbtree, gblinear or dart.

In order to get the actual booster, you can call get_booster() instead:

>>> clf.booster
'gbtree'
>>> clf.get_booster()

>>> clf.get_booster().best_iteration
9
>>> print("Best Iteration: {}".format(clf.get_booster().best_iteration))
Best Iteration: 9

I'm not sure about the second half of your question, namely:

Furthermore, how can I print the training error of ** each round**?

but hopefully you're unblocked!