none
Possible Bug(s): Visualizing Two-Class Boosted Decision Trees

    Question

  • Good morning,

    I've been working extensively with Two-Class Boosted Decision Trees over the last few days and noticed two possible bugs when you Visualize the Trained Model.  Here is my experiment:

    

    To simplify this experiment, I set my hyperparameters to include 5 nodes and 1 tree.  This created the following tree:


    Let's start by looking at the Red Boxes.  We can see that the Node predicts a value of 1 (TRUE) while the table on the right predicts a value of 0 (FALSE).  I've run similar experiments at least 20 times and this table ALWAYS shows 0 (FALSE), regardless of the actual prediction.  This is possible bug #1.

    Let's move on to the blue box.  We see that this particular node predicts 1 (TRUE) when Income is between 11.2 and 14.5.  If we compare this to scores generated by score model (below), we see that the prediction is actually the opposite.  


    This node should predict 0 (FALSE).  Again, I've run through multiple experiments and found that the node is ALWAYS OPPOSITE.  1 (TRUE) always scores to 0 (FALSE) and 0 (FALSE) always scores to 1 (TRUE).  In fact, I've even compared the scores at the end to the raw data and found that the prediction (if done manually) always lines up with the generated scores.  This means that the nodes is the tree are displaying the wrong values.  This is possible bug #2.

    Am I missing something or are these legitimate bugs?

    Thanks,

    Brad Llewellyn

    BI Engineer

    Valorem Consulting

    @BreakingBI

    www.linkedin.com/in/bradllewellyn

    Tuesday, November 08, 2016 11:27 AM

Answers

  • Hi Brad!

    You've correctly identified a bug in the tree visualization. The root issue is that the tree visualizer should show the probabilities of each of the labels but for Boosted Decision Tree it doesn't (e.g. prediction of 0 with probability 0.68 and prediction of 1 with probability of 0.42). This results in the confusing situation you've described.

    We're investigating into a fix.

    -Roope

    Tuesday, November 08, 2016 2:42 PM
    Moderator

All replies

  • Hi Brad!

    You've correctly identified a bug in the tree visualization. The root issue is that the tree visualizer should show the probabilities of each of the labels but for Boosted Decision Tree it doesn't (e.g. prediction of 0 with probability 0.68 and prediction of 1 with probability of 0.42). This results in the confusing situation you've described.

    We're investigating into a fix.

    -Roope

    Tuesday, November 08, 2016 2:42 PM
    Moderator
  • Hello

    Are there any news regarding when this bug will be fixed? 

    Kind regards

    Sune Larsen

    Wednesday, January 25, 2017 7:56 AM
  • When bugs such as these are confirmed, is there somewhere (other than these forum posts) to track that a bug has been confirmed, and to monitor the status?  Searching through forums with the term "bug" does not seem to be an efficient mechanism for tracking these.
    Wednesday, January 25, 2017 3:34 PM