Machine Learning - Neural Networks for Multi-class Classification

This series of articles are the study notes of " Machine Learning ", by Prof. Andrew Ng., Stanford University. This article is the notes of week 4.  It contains topics about Neural Networks for Multi-class Classification.

Neural Networks for  Multi-class Classification

In this section, I want to tell you about how to use neural networks to do multi-class classification where we may have more than one category that we're trying to distinguish amongst. In the last part of the last video, where we had the handwritten digit recognition problem, that was actually a multi-class classification problem because there were ten possible categories for recognizing the digits from 0 through 9 and so, if you want us to fill you in on the details of how to do that. The way we do multi-class classification in a neural network is essentially an extension of the one versus all method.

Example: computer vision

So, let's say that we have a computer vision example, where instead of just trying to recognize cars as in the original example that I started off with, but let's say that we're trying to recognize, you know, four categories of objects and given an image we want to decide if it is a pedestrian, a car, a motorcycle or a truck.

If that's the case, what we would do is we would build a neural network with four output units so that our neural network now outputs a vector of four numbers. So, the output now is actually needing to be a vector of four numbers and what we're going to try to do is get the first output unit to classify: is the image a pedestrian, yes or no. The second unit to classify: is the image a car, yes or no. This unit to classify: is the image a motorcycle, yes or no, and this would classify: is the image a truck, yes or no.

here's our neural network with four output units and those are what we want h of x to be when we have the different images, and the way we're going to represent the training set in these settings is as follows. So, when we have a training set with different images of pedestrians, cars, motorcycles and trucks, what we're going to do in this example is that whereas previously we had written out the labels as y being an integer from 1, 2, 3 or 4. Instead of representing y this way, we're going to instead represent y as follows: namely Yi will be either 1, 0, 0, 0 or 0, 1, 0, 0 or 0, 0, 1, 0 or 0, 0, 0, 1 depending on what the corresponding image Xi is. And so one training example will be one pair Xi colon Yi where Xi is an image with, you know one of the four objects and Yi will be one of these vectors.