Neural Networks - Examples and intuitions I

摘要: 本文是吴恩达 (Andrew Ng)老师《机器学习》课程，第九章《神经网络学习》中第68课时《例子与直觉理解I》的视频原文字幕。为本人在视频学习过程中记录下来并加以修正，使其更加简洁，方便阅读，以便日后查阅使用。现分享给大家。如有错误，欢迎大家批评指正，在此表示诚挚地感谢！同时希望对大家的学习能有所帮助.
————————————————
In this and the next video, I want to work through a detailed example, showing how a neural network can compute a complex nonlinear function of input. And hopefully, this would give you a good sense of why neural networks can be used to learn complex nonlinear hypotheses.

Consider the following problem, where we have input features and that are binary values, so either 0 or 1. So and can each take on only one of two values. In this example, I’ve drawn only two positive examples and two negative examples, but you can think of this as the simplified version of a more complex learning problem, where we may have a bunch of positive examples in the upper right and lower left, and a bunch of negative examples denoted by the circles. And what we’d like to do is learn a nonlinear decision boundary that maybe to separate the positive and the negative examples. So how can a neural network do this? And rather use an example on the right, I’m going to use this maybe easier to examine example on the left. Concretely, what this is is really computing the target label , or actually, this is actually the function, where is the alternative notation for . So , that’s true only if exactly one of or is equal to 1. It turns out the specific example, it works out a little bit better if we use the example instead. These two are the same, of course. It means . And so we’re going to have positive examples either both are true or both are false, and we’ll have that’s y equal to 1. And we’re going to have y=0, if only one of them is true. And we want to figure out if we can get a neural network to fit to this sort of training set. In order to build up to a network that fits the example, we’re going to start to a slightly simpler one and show a network that fits the AND function.

Concretely, let’s say we have input and that are again binary, so it’s either 0 or 1. And let’s say our target labels y are you know, is equal to . This is a logical AND. So can we get a one unit network to compute this logical AND function? In order to do so, I’m going to actually draw in the bias unit as well, then +1 unit. Now, let me just assign some values to the weights or the parameters of this network. I’m going to write down the parameters on this diagram, right? Minus 30 here, plus 20 and plus 20. And what this mean is that, I’m assigning a value of -30 to the value associate to the , there is +1 going to be this unit. And a parameter value of plus 20 that multiplies in a value. Plus 20 for the parameter that multiplies into . So concretely, this is saying that my hypotheses . So sometimes it’s just convenient to draw these weights and draw these parameters up here, in the neural diagram of the neural network. And of course this minus 30 this is actually . This is , and that’ . But it’s just easier think about it as you know, associating these parameters with the edges of the network. Let’s look at what this little single neuron network will compute. Just to remind you, the sigmoid activation function g(z) looks like this. It starts from 0, rises smoothly, crosses 0.5, and then it asymptotes one. And to give you some landmarks, if the horizontal axis value z is equal to 4.6, then the sigmod function is equal to 0.99. This is very close to 1. And kind of symmetrically, if it is negative 4.6, then the sigmoid function there is equal to 0.01, which is very close to 0. Let’s look at the four possible input values for and and look at what the hypotheses will output in that case. If and are both equal to 0, if you look at this, then the hypotheses output g(-30). So, it’s like very far to the left of this diagram, will be very close to 0. If , , then this formula here equals to g(-10), and again, that’s to the far left of this plot, and so that’s again very close to 0. This is also g(-10). That is if , , this is -30+20, which is -10. And finally, if , , then you have g(10), which is therefore, very close to 1. And if you look in this column, this is exactly the logical AND function, so, this is computing . In other words, it outputs 1 if and only if and are both equal to 1. So by writing out our little truth table like this, we manage to figure out what’s the logical function that our neural network computes.

This network shown here computes the OR function. Just to show you how I worked that out. If you write out the hypotheses, you find that it’s computing . And so if you fill in these values, you find that g(-10) is approximately 0, g(10) is approximately 1, and so on. These are approximately 1, and approximately 1. And these numbers is essentially the logical OR function. So, hopefully with you, you now understand how single neuron in a neural network can be used to compute logical functions like AND and OR and so on. In the next video, we’ll continue building on these examples, and work through a more complex example. We’ll get show you how a neural network, now with multiple layers of units can be used to compute more complex functions like the XOR function or the XNOR function.

<end>