What does the value of 'leaf' in the following xgboost model tree diagram means?

Question

I am guessing that it is conditional probability given that the above (tree branch) condition exists. However, I am not clear on it.

If you want to read more about the data used or how do we get this diagram then go to : http://machinelearningmastery.com/visualize-gradient-boosting-decision-trees-xgboost-python/

Answer 1

Attribute leaf is the predicted value. In other words, if the evaluation of a tree model ends at that terminal node (aka leaf node), then this is the value that is returned.

In pseudocode (the left-most branch of your tree model):

if(f1 < 127.5){
  if(f7 < 28.5){
    if(f5 < 45.4){
      return 0.167528f;
    } else {
      return 0.05f;
    }
  }
}

Answer 2

For a classification tree with 2 classes {0,1}, the value of the leaf node represent the raw score for class 1. It can be converted to a probability score by using the logistic function. The calculation below use the left most leaf as an example.

1/(1+np.exp(-1*0.167528))=0.5417843204057448

What this means is if a data point ends up being distributed to this leaf, the probability of this data point being class 1 is 0.5417843204057448.

Answer 3

You are correct. Those probability values associated with leaf nodes are representing the conditional probability of reaching leaf nodes given a specific branch of the tree. Branches of trees can be presented as a set of rules. For example, @user1808924 mentioned in his answer; one rule which is representing the left-most branch of your tree model.

So, in short: The tree can be linearized into decision rules, where the outcome is the contents of the leaf node, and the conditions along the path form a conjunction in the if clause. In general, the rules have the form:

if condition1 and condition2 and condition3 then outcome.

Decision rules can be generated by constructing association rules with the target variable on the right. They can also denote temporal or causal relations.