http://math.mit.edu/~rothvoss/18.304.3PM/Presentations/1-Eric-Boosting304FinalRpdf.pdf
Consider MIT Admissions
【qualitative quantitative】
•
2-class system (Admit/Deny)
•
Both Quantitative Data and Qualitative Data
•
We consider (Y/N) answers to be Quantitative (-1,+1)
•
Region, for instance, is qualitative.
Rules of Thumb, Weak Classifiers
Easy to come up with rules of thumb that correctly classify the training data at
better than chance.
•
E.g. IF “GoodAtMath”==Y THEN predict “Admit”.
•
Difficult to find a single, highly accurate prediction rule. This is where our Weak
Learning Algorithm,AdaBoost, helps us.
What is a Weak Learner?
【generalization error better than random guessing】
For any distribution, with high probability, given polynomially many examples and polynomial time we can find a classifier with generalization error
better than random guessing.
Weak Learning Assumption
•
We assume that our Weak Learning Algorithm (Weak
Learner) can consistently find weak classifiers (rules of
thumb which classify the data correctly at better than 50%)
•
【boosting】
Given this assumption, we can use boosting to generate a
single weighted classifier which correctly classifies our
training data at 99%-100%.
【AdaBoost Specifics 】
•
How does AdaBoost weight training examples optimally?
•
Focus on difficult data points. The data points that have been
misclassified most by the previous weak classifier.
•
How does AdaBoost combine these weak classifiers into a
comprehensive prediction?
•
Use an optimally weighted majority vote of weak classifier.
AdaBoost Technical Description
Missing details: How to generate distribution? How to get single classifier?
Constructing Dt
Getting a Single Classifier
