机器学习学习笔记——1.1.2.1.2 Vectorization part 1（向量化——第1部分）

In this video, you see a very useful idea called vectorization. When you're implementing a learning algorithm, using vectorization will both make your code shorter and also make it run much more efficiently. Learning how to write vectorized code will allow you to also take advantage of modern numerical linear algebra libraries, as well as maybe even GPU hardware that stands for graphics processing unit. This is hardware objectively designed to speed up computer graphics in your computer, but turns out can be used when you write vectorized code to also help you execute your code much more quickly. Let's look at a concrete example of what vectorization means.

Here's an example with parameters w and b, where w is a vector with three numbers, and you also have a vector of features x with also three numbers. Here n is equal to 3. Notice that in linear algebra, the index or the counting starts from 1 and so the first value is subscripted w1 and x1. In Python code, you can define these variables w, b, and x using arrays like this. Here, I'm actually using a numerical linear algebra library in Python called NumPy, which is by far the most widely used numerical linear algebra library in Python and in machine learning. Because in Python, the indexing of arrays while counting in arrays starts from 0, you would access the first value of w using w square brackets 0. The second value using w square bracket 1, and the third and using w square bracket 2. The indexing here, it goes from 0,1 to 2 rather than 1, 2 to 3. Similarly, to access individual features of x, you will use x0, x1, and x2. Many programming languages including Python start counting from 0 rather than 1.

Now, let's look at an implementation without vectorization for computing the model's prediction. In codes, it will look like this. You take each parameter w and multiply it by his associated feature. Now, you could write your code like this, but what if n isn't three but instead n is a 100 or a 100,000 is both inefficient for you the code and inefficient for your computer to compute. Here's another way. Without using vectorization but using a for loop. In math, you can use a summation operator to add all the products of w_j and x_j for j equals 1 through n. Then I'll cite the summation you add b at the end. To summation goes from j equals 1 up to and including n. For n equals 3, j therefore goes from 1, 2 to 3. In code, you can initialize after 0. Then for j in range from 0 to n, this actually makes j go from 0 to n minus 1. From 0, 1 to 2, you can then add to f the product of w_j times x_j.

Finally, outside the for loop, you add b. Notice that in Python, the range 0 to n means that j goes from 0 all the way to n minus 1 and does not include n itself. This is written range n in Python. But in this video, I added a 0 here just to emphasize that it starts from 0. While this implementation is a bit better than the first one, this still doesn't use factorization, and isn't that efficient? Now, let's look at how you can do this using vectorization. This is the math expression of the function f, which is the dot product of w and x plus b, and now you can implement this with a single line of code by computing fp equals np dot dot, I said dot dot because the first dot is the period and the second dot is the function or the method called DOT. But is fp equals np dot dot w comma x and this implements the mathematical dot products between the vectors w and x.

Then finally, you can add b to it at the end. This NumPy dot function is a vectorized implementation of the dot product operation between two vectors and especially when n is large, this will run much faster than the two previous code examples. I want to emphasize that vectorization actually has two distinct benefits. First, it makes code shorter, is now just one line of code. Isn't that cool? Second, it also results in your code running much faster than either of the two previous implementations that did not use vectorization. The reason that the vectorized implementation is much faster is behind the scenes.

The NumPy dot function is able to use parallel hardware in your computer and this is true whether you're running this on a normal computer, that is on a normal computer CPU or if you are using a GPU, a graphics processor unit, that's often used to accelerate machine learning jobs. The ability of the NumPy dot function to use parallel hardware makes it much more efficient than the for loop or the sequential calculation that we saw previously. Now, this version is much more practical when n is large because you are not typing w0 times x0 plus w1 times x1 plus lots of additional terms like you would have had for the previous version. But while this saves a lot on the typing, is still not that computationally efficient because it still doesn't use vectorization.

To recap, vectorization makes your code shorter, so hopefully easier to write and easier for you or others to read, and it also makes it run much faster. But honest, this magic behind vectorization that makes this run so much faster. Let's take a look at what your computer is actually doing behind the scenes to make vectorized code run so much faster.

在这段视频中，你将看到一个非常有用的理念，叫做向量化。当你实现一个学习算法时，使用向量化会让你的代码更短，同时也会使你的代码运行得更加高效。学会编写向量化代码还能让你利用现代数值线性代数库，甚至可能是GPU硬件（图形处理单元）。这种硬件是专门为加速计算机图形而设计的，但事实证明，当你编写向量化代码时，它也可以用来帮助你更快地执行代码。让我们看一个具体的向量化例子。

这里有一个带有参数w和b的例子，其中w是一个包含三个数字的向量，你还有一个也包含三个数字的特征向量x。这里的n等于3。注意，在线性代数中，索引或计数是从1开始的，所以第一个值用w1和x1表示。在Python代码中，你可以像这样使用数组来定义变量w、b和x。这里，我实际上使用了Python中的一个数值线性代数库NumPy，它是迄今为止在Python和机器学习中使用最广泛的数值线性代数库。因为Python中的数组索引从0开始计数，所以你将使用w[0]访问w的第一个值，使用w[1]访问第二个值，使用w[2]访问第三个值。这里的索引是从0、1到2，而不是从1、2到3。类似地，要访问x的各个特征，你会使用x0、x1和x2。许多编程语言包括Python都是从0开始计数的。

现在，让我们看看不使用向量化来计算模型预测的实现方式。在代码中，它会是这样的。你取每个参数w并乘以其关联的特征。现在，你可以这样写你的代码，但如果n不是3而是100或100,000，那么对你来说编写代码以及对你的计算机来说计算都将是低效的。这是另一种方法。不使用向量化而是使用for循环。在数学中，你可以使用求和运算符将所有w_j和x_j的乘积相加，对于j等于1到n。然后我将引用求和运算符，最后加上b。求和运算从j等于1一直到n。对于n等于3，因此j将从1、2到3。在代码中，你可以在0之后初始化。然后对于j在范围从0到n，这实际上使j从0到n-1。从0、1到2，你可以然后将w_j乘以x_j的乘积添加到f上。

最后，在for循环外，你加上b。注意，在Python中，0到n的范围意味着j从0一直到n-1，不包括n本身。这在Python中写作range n。但在这段视频中，我在这里加了0只是为了强调它是从0开始的。虽然这种实现比第一种好一点，但这仍然没有使用向量化，效率不高？现在，让我们看看如何使用向量化来实现这个功能。这是函数f的数学表达式，即w和x的点积加上b，现在你可以通过计算fp等于np.dot(w, x)来实现这一行代码，我之所以说两个点是因为第一个点是句号，第二个点是称为DOT的函数或方法。但是，fp等于np.dot(w, x)实现了向量w和x之间的数学点积。

然后最后，你可以在最后加上b。这个NumPy dot函数是一个向量化实现的两向量之间点积操作，特别是当n很大时，这将比之前两个未使用向量化的例子运行得快得多。我想强调的是，向量化实际上有两个明显的好处。首先，它使代码更短，现在只是一行代码。这不是很酷吗？其次，它还使你的代码运行得比之前两个未使用向量化实现的版本都要快。向量化实现之所以快得多的原因是幕后的。

NumPy dot函数能够利用计算机中的并行硬件，无论你是在普通计算机CPU上运行还是在用于加速机器学习工作的GPU（图形处理器单元）上运行，这都是真的。NumPy dot函数能够使用并行硬件的能力使其比我们之前看到的for循环或顺序计算更加高效。现在，这个版本在n很大时更加实用，因为你不需要像以前那样键入w0乘以x0加上w1乘以x1加上很多额外的项。但是，尽管这节省了很多键入工作，但在计算效率上仍然不如人意，因为它仍然没有使用向量化。

总之，向量化使你的代码更短，希望更容易编写，也更容易供你或其他人阅读，而且还能使你的代码运行得更快。但是老实说，向量化背后的魔法使它运行得如此之快。让我们看看你的计算机实际上是如何在幕后使向量化代码运行得如此之快的。