Logistic Regression
Multi-class Classification
Some examples of multi-class classification
- Email foldering/tagging: Work, Friends,Family, Hobby
y=1, 2, 3, 4
- Medical diagrams: Not ill, Cold, Flu
y=1, 2, 3
- Weather: Sunny, Cloudy, Rain, Snow
y=1, 2, 3, 4
So the question is given the data set with three classes where this is an example of one class, that's an example of a different class, and that's an example of yet a third class. How do we get a learning algorithm to work for the setting?
One-vs-all (one-vs-rest) classification
Let's say we have a training set like that shown on the left, where we have three classes of y = 1, we denote that with a triangle, if y = 2, the square, and if y = 3, then the cross. What we're going to do is take our training set and turn this into three separate binary classification problems.
Start with class triangle △
Second class square □
Next we do the same thing for class two. going to take the squares and assign the squares as the positive class, and assign everything else, the triangles and the crosses, as a negative class. And then we fit a second logistic regression classifier and call this hθ(2)(x), where the superscript two denotes that we're now doing this, treating the square class as the positive class.
Finally class cross ×
finally, we do the same thing for the third class and fit a third classifier hθ(3)(x), and maybe this will give us a decision bounty of the visible cross fire. This separates the positive and negative examples like that.
So to summarize, what we've done is, we've fit three classifiers. So, for i = 1, 2, 3, we'll fit a classifier x super script i subscript theta of x. Thus trying to estimate what is the probability that y is equal to class i, given x and parametrized by theta.
One-vs-all
Train a logistic regression classifier hθ(i)(x) for each class i to predict the probability that y=i.
On a new input x , to make a prediction, pick the classi that maximizes