Pytorch for freshman

Pytorch

Basic

Lambda Function

Anonymous functioni that’s not meant to be stored.

Lambda Functions with Practical Examples in Python

Device configuration

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

Tensor

Tensor and NumPy Transfer

np_array = np.array(data)
x_np = torch.from_numpy(np_array)

t = torch.ones(5)
n = t.numpy()

Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other

Automatic Differentiation

There are reasons you might want to disable gradient tracking:

• To mark some parameters in your neural network at frozen parameters. This is a very common scenario for finetuning a pretrained network
• To speed up computations when you are only doing forward pass, because computations on tensors that do not track gradients would be more efficient.

with torch.no_grad():

Optimizing Model Parameters

optimizer.zero_grad() # reset the gradients of model parameters
loss.backward() # compute the gradients of tensors
optimizer.step() # update the parameters

Coding Order

Device Configuration(GPU)

1. Hyper Parameters
2. Data Preprocessing and Data Loader
3. Model Declaration
4. Loss and Optimizer
5. Train and Test

Train & Test

model.train() # be used in training, dropout and batch normalization
model.eval() # be used in testing