intro to pytorch

1

one hot 解决了 multi attribute 用123， 会有预测值between them

2

已知X,y
np.argwher(y>1) return the indices
X[np.argwhere(y>1)] return the respetive X which its according y >1

for x, y in zip(x,y):

3 penalize

we want to penalize the large weight by regularization

4

use dropout
use random start to deal with local minimum
Batch vs Stochastic Gradient Descent
use Momentum to deal with local minimum

or in cv, we can also use pooling to make cnn more efficient

5

the difference between Logsoftmax and Softmax
the difference between optim.SGD and optim.Adam

6

Now we need to calculate the percentage of correct predictions. equals has binary values, either 0 or 1. This means that if we just sum up all the values and divide by the number of values, we get the percentage of correct predictions. This is the same operation as taking the mean, so we can get the accuracy with a call to torch.mean. If only it was that simple. If you try torch.mean(equals), you’ll get an error

RuntimeError: mean is not implemented for type torch.ByteTensor
This happens because equals has type torch.ByteTensor but torch.mean isn’t implemented for tensors with that type. So we’ll need to convert equals to a float tensor. Note that when we take torch.mean it returns a scalar tensor, to get the actual value as a float we’ll need to do accuracy.item().

normalize

You’ll also typically want to normalize images with transforms.Normalize. You pass in a list of means and list of standard deviations, then the color channels are normalized like so

input[channel] = (input[channel] - mean[channel]) / std[channel]

Subtracting mean centers the data around zero and dividing by std squishes the values to be between -1 and 1. Normalizing helps keep the network work weights near zero which in turn makes backpropagation more stable. Without normalization, networks will tend to fail to learn.
\

High-frequency components also correspond to the edges of objects in images, which can help us classify those objects.

dif

dense layers are fully connected
convolutional layers are locally connected

combine with matplot

import matplotlib.pyplot as plt
import matplotlib.image as mpimg
image = mpimg.imread(‘data/curved_lane.jpg’)

plt.imshow(image)

capsule network vs pooling layer

cnn

use cv2 to convert picture into gray pic

print gray picture
要squeez
ax.imshow(np.squeeze(layer[0,i].data.numpy()), cmap=‘gray’)

style transfer

use gram matrix

识别数字的方法

CNN

multiple logistc regression

这是一个简单的linear classifier
把每个数字的pixel看做特征，每个数字的识别都看成是一个logistic regression， 用scipy.optim.minimize，求出每个digit的每个特征的系数，然后dot要识别的值，用np.argmax() return the max probability