Decision Trees and Boosting, XGBoost | Two Minute Papers #55

dear fellow Scholars this is 2minute papers with Caro a decision tree is a great tool to help making good decisions from a huge bunch of data the classical example is when we have a bunch of information about people and would like to find out whether they like computer games or not note that this is a toy example for educational purposes we can build the following tree if the person's age in question is over 15 the person is less likely to like computer games if the subject is under 15 and is a male he is quite likely to like video games if she's female then less likely note that the output of the tree can be a decision like yes or no but in our case we will assign positive and negative scores instead you'll see in a minute why that's beneficial but this tree was just one possible way of approaching the problem and admittedly not a spectacular one A different decision tree could be simply asking whether this person uses a computer daily or not individually these trees are quite shallow and we call them weak Learners this term means that individually they are quite inaccurate but slightly better than random guessing and now comes the cool part the concept of tree boosting means that we take many weak Learners and combine them into a strong learner using the mentioned scoring system instead of decisions also makes this process easy and straightforward to implement boosting is similar to what we do with illnesses if a doctor says that I have a rare condition I will make sure and ask at least a few more doctors to make a more educated decision about my health the cool thing is that the individual trees don't have to be great if they give you decisions that are just a bit better than random guessing using a lot of them will produce strong learning results if we go back to the analogy with doctors then if the individual doctors know just enough not to kill the patient a well chosen committee will be able to put together an accurate diagnosis for the patient an even cooler adaptive version of this technique brings in new doctors to the committee according to the deficiencies of the existing members one other huge advantage of boosted trees over neural networks is that we actually see why and how the computer arrives to a decision this is a remarkably simple method that leads to results of very respectable accuracy a well-known software Library called X G boost has been responsible for winning a staggering amount of machine learning competitions in kaggle I'd like to take a second and thank you fellow Scholars for your amazing support on patreon and making two-minute papers possible creating these episodes is a lot of hard work and your support has been invaluable so far thank you so much we used to have three categories for supporters undergrad students get access to a patreon only activity feed and get to know well in advance the topics of the new episodes PhD students who are addicted to two-minute papers get a chance to see every episode up to 24 hours in advance talking about committees in this episode full professors form a committee to decide the order of the next few episodes and now we introduce a new category the noble laurat supporters in this category can literally become part of 2minute papers and will be listed in the video description box in the upcoming episodes plus all of the above thanks for watching and for your generous support and I'll see you next time