One hidden layer Neural Network - Vectorizing across multiple examples

The notes when study the Coursera class by Mr. Andrew Ng "Neural Networks & Deep Learning", section 3.4 "Vectorizing across multiple examples". It shows how to compute NN output via vectorization when there are multiple training example. Share it with you and hope it helps!

————————————————
In last class, we saw how to compute NN output given a single training example. In this class, you see how to vectorize across multiple training examples.

figure-1

Figure-1 shows how to compute NN hidden layer and output layer when there is just one training example. 

When there are m training examples, following figure-2 shows a for loop going over all training examples:

figure-2

Not that the meaning of the superscripts [] & (). For example,  means layer 1 for training example i.

Following figure-3 shows how to compute NN hidden layer and output layer by vectorizing above for loop when there are multiple m examples:

figure-3

 Here we form by stacking all training example in columns. Similarly, for , we just stack all horizontally; for , we stack all . Note that for multiple m traing examples, we just use capital case .

One interesting property is, for these matrices , horizontally we're going to index across training examples; and vertically, index across different nodes in the NN. For example in figure-4, the red point corresponds to the activation of the first hidden unit on the first training example; the green point corresponds to the second hidden unit of the first training example; the cyan point maps to the first hidden unit on the mth training example etc.

figure-4

<end>