pytorch学习笔记（五）
1.前言
好久没更新笔记了，原因很多，不过肯定不会把这个丢掉的，毕竟也是自己的学习过程，我们还是接着上次继续往后学，这次到了Transfer Learning tutorial，我们会学习用transfer learning来训练网络，关于Transfer learning大家可以看看CS231N
这里概括两个主要的Transfer learning的特点：
- 调整卷积网：我们用一个预训练过得网络初始化一个神经网络而不是随机初始化一个
- 卷积网络作为固定特征提取器：这里我们将冻结除了最终完全连接层之外的所有网络的权重。最后一个完全连接的层被替换为具有随机权重的新层，并且仅训练该层
2.正文
我们这次的任务是建立一个能区分蚂蚁和蜜蜂的模型。
2.1加载数据
我们使用torchvision和torch.utils.data来加载数据
from __future__ import print_function,division
import torch
import torch.nn as nn
import torch.optim as optim
from torch.optim import lr_scheduler
import numpy as np
import torchvision
from torchvision import datasets,models,transforms
import matplotlib.pyplot as plt
import time
import os
import copy
plt.ion()
data_transforms = {
'train':transforms.Compose([
transforms.RandomResizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize([0.485,0.456,0.406],[0.229,0.224,0.225])
]),
'val':transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485,0.456,0.406],[0.229,0.224,0.225])
])
}
data_dir = 'hymenoptera_data'
image_datasets = {x:datasets.ImageFolder(os.path.join(data_dir,x),data_transforms[x])
for x in ['train','val']}
dataloaders = {x:torch.utils.data.DataLoader(image_datasets[x],batch=4,shuffle=True,num_workers=4)
for x in ['train','val']}
dataset_sizes = {x:len(image_datasets[x]) for x in ['train','val']}
class_names = image_datasets['train'].classes
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")这里面比较难懂的函数我在笔记（三）有过解释，这里就不再赘述，另外，hymenoptera_data这个需要从这里下载
2.2可视化部分图片
如标题，和以前的步骤没什么不同
def imshow(inp,title=None):
"""Imshow for Tensor."""
inp = inp.numpy().transpose((1,2,0))
mean = np.array([0.485,0.456,0.406])
std = np.array([0.229,0.224,0.225])
inp = std*inp + mean
inp = np.clip(inp,0,1)
plt.imshow(inp)
if title is not None:
plt.title(title)
plt.pause(0.001)
inputs,classes = next(iter(dataloaders['train']))
out = torchvision.utils.make_grid(inputs)
imshow(out,title=[class_names[x] for x in classes])其中函数clip(x,min,max)是将将数组中的元素限制在min, max之间，大于max的就使得它等于 max，小于min,的就使得它等于min。
输出：
2.3训练模型
我们会用常用的函数训练模型，但我们加入如下两点：
1.时序安排学习率
2.保存最佳模型
def train_model(model,criterion,optimizer,scheduler,num_epochs=25):
since = time.time()
best_model_wts = copy.deepcopy(model.state_dict())
best_acc = 0.0
for epoch in range(num_epochs):
print('Epoch {}/{}'.format(epoch,num_epochs-1))
print('-'*10)
for phase in ['train','val']:
if phase == 'train':
scheduler.step()
model.train()
else:
model.eval()
running_loss = 0.0
running_corrects = 0
for inputs,labels in dataloaders[phase]:
inputs = inputs.to(device)
labels = inputs.to(device)
optimizer.zero_grad()
with torch.set_grad_enabled(phase == 'train'):
outputs = model(inputs)
_,preds = torch.max(outputs,1)
loss = criterion(outputs,labels)
if phase == 'train':
loss.backward()
optimizer.step()
running_loss += loss.item()*inputs.size(0)
running_corrects += torch.sum(preds == labels.data)
epoch_loss = running_loss/dataset_sizes[phase]
epoch_acc = running_corrects.double()/dataset_sizes[phase]
print('{} Loss:{:.4f} Acc:{:.4f}'.format(phase,epoch_loss,epoch_acc))
if phase == 'val' and epoch_acc>best_acc:
best_acc = epoch_acc
best_model_wts = copy.deepcopy(model.state_dict())
print()
time_elapsed = time.time()-since
print('Training complete in {:.0f}m {:.0f}s'.format(
time_elapsed//60,time_elapsed%60))
print('Best val Acc:{:4f}'.format(best_acc))
model.load_state_dict(best_model_wts)
return model这个模型训练和我们之前写的训练没什么区别，连损失函数用的都是一样的。
再写一个结果可视化的函数：
def visualize_model(model,num_images=6):
was_training = model.training
model.eval()
images_so_far = 0
fig = plt.figure()
with torch.no_grad():
for i,(inputs,labels) in enumerate(dataloaders['val']):
inputs = inputs.to(device)
labels = labels.to(device)
outputs = model(inputs)
_,preds = torch.max(outputs,1)
for j in range(inputs.size()[0]):
images_so_far += 1
ax = plt.subpolt(num_images//2,2,image_so_far)
ax.axis('off')
ax.set_title('predicted:{}'.format(class_names[preds[j]]))
imshow(inputs.cpu().data[j])
if images_so_far == num_images:
model.train(mode=was_training)
return
model.train(mode=was_training)2.4 调整卷积网
载入一个预训练的模型并重置全连接层的最后一层
model_conv = torchvision.models.resnet18(pretrained=True)
for param in model_conv.parameters():
param.requires_grad = False
num_ftrs = model_conv.fc.in_features
model_conv.fc = nn.Linear(num_ftrs,2)
model_conv = model_conv.to(device)
criterion = nn.CrossEntropyLoss()
optimizer_conv = optim.SGD(model_conv.fc.parameters(),lr=0.001,momentum=0.9)
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv,step_size=7,gamma=0.1)其中torchvision.models.resnet18(pretrained=True)是pytorch提供的一种预训练的网络，这里用到的是残差网络，这是一种比较好的神经训练网络，具体可以看看 残差resnet网络原理详解
2.5训练和评估
model_conv = train_model(model_conv,criterion,optimizer_conv,exp_lr_scheduler,num_epochs=25)
visualize_model(model_ft)就是调用我们之前写的一些函数
输出
Epoch 0/24
----------
train Loss: 0.1660 Acc: 0.6762
val Loss: 0.0445 Acc: 0.9542
Epoch 1/24
----------
train Loss: 0.1141 Acc: 0.8033
val Loss: 0.0877 Acc: 0.8693
Epoch 2/24
----------
train Loss: 0.1440 Acc: 0.7623
val Loss: 0.0484 Acc: 0.9346
Epoch 3/24
----------
train Loss: 0.1082 Acc: 0.8074
val Loss: 0.0787 Acc: 0.8824
Epoch 4/24
----------
train Loss: 0.1751 Acc: 0.7500
val Loss: 0.2313 Acc: 0.7647
Epoch 5/24
----------
train Loss: 0.1367 Acc: 0.8074
val Loss: 0.1766 Acc: 0.7908
Epoch 6/24
----------
train Loss: 0.1456 Acc: 0.8156
val Loss: 0.1116 Acc: 0.7908
Epoch 7/24
----------
train Loss: 0.1259 Acc: 0.8033
val Loss: 0.0793 Acc: 0.8627
Epoch 8/24
----------
train Loss: 0.0807 Acc: 0.8607
val Loss: 0.0781 Acc: 0.8758
Epoch 9/24
----------
train Loss: 0.0618 Acc: 0.8730
val Loss: 0.0778 Acc: 0.8824
Epoch 10/24
----------
train Loss: 0.0804 Acc: 0.8566
val Loss: 0.0876 Acc: 0.8758
Epoch 11/24
----------
train Loss: 0.0751 Acc: 0.8607
val Loss: 0.0945 Acc: 0.8693
Epoch 12/24
----------
train Loss: 0.0695 Acc: 0.8770
val Loss: 0.0950 Acc: 0.8824
Epoch 13/24
----------
train Loss: 0.0596 Acc: 0.8852
val Loss: 0.0907 Acc: 0.8889
Epoch 14/24
----------
train Loss: 0.0624 Acc: 0.9016
val Loss: 0.0785 Acc: 0.8824
Epoch 15/24
----------
train Loss: 0.0546 Acc: 0.9139
val Loss: 0.0810 Acc: 0.8824
Epoch 16/24
----------
train Loss: 0.0982 Acc: 0.8484
val Loss: 0.1054 Acc: 0.8824
Epoch 17/24
----------
train Loss: 0.0659 Acc: 0.8893
val Loss: 0.0839 Acc: 0.8889
Epoch 18/24
----------
train Loss: 0.0645 Acc: 0.8893
val Loss: 0.0760 Acc: 0.8824
Epoch 19/24
----------
train Loss: 0.0723 Acc: 0.8934
val Loss: 0.0699 Acc: 0.8758
Epoch 20/24
----------
train Loss: 0.0689 Acc: 0.8852
val Loss: 0.0733 Acc: 0.8627
Epoch 21/24
----------
train Loss: 0.0656 Acc: 0.8893
val Loss: 0.0915 Acc: 0.8954
Epoch 22/24
----------
train Loss: 0.0756 Acc: 0.8770
val Loss: 0.0772 Acc: 0.8889
Epoch 23/24
----------
train Loss: 0.0695 Acc: 0.8934
val Loss: 0.0724 Acc: 0.8627
Epoch 24/24
----------
train Loss: 0.0556 Acc: 0.9139
val Loss: 0.0821 Acc: 0.8889
Training complete in 1m 26s
Best val Acc: 0.954248
2.6做一些微小调整
这里我们会冻结除去最后一层的所有网络参数，这样这些的梯度就不会被计算从而可以节省时间，之所以我们可以这么做，是因为我们用的是预训练的模型，我们不需要前面的网络层再过多的更新，而只是需要最后一层根据我们的需求进行更新，从而可以节省时间
model_conv = torchvision.models.resnet18(pretrained=True)
for param in model_conv.parameters():
param.requires_grad = False
num_ftrs = model_conv.fc.in_features
model_conv.fc = nn.Linear(num_ftrs,2)
model_conv = model_conv.to(device)
criterion = nn.CrossEntropyLoss()
optimizer_conv = optim.SGD(model_conv.fc.parameters(),lr=0.001,momentum=0.9)
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv,step_size=7,gamma=0.1)再来看看评估
Epoch 0/24
----------
train Loss: 0.1913 Acc: 0.5738
val Loss: 0.1566 Acc: 0.6732
Epoch 1/24
----------
train Loss: 0.1469 Acc: 0.7295
val Loss: 0.0659 Acc: 0.9085
Epoch 2/24
----------
train Loss: 0.1189 Acc: 0.7623
val Loss: 0.0458 Acc: 0.9477
Epoch 3/24
----------
train Loss: 0.1191 Acc: 0.8033
val Loss: 0.0463 Acc: 0.9281
Epoch 4/24
----------
train Loss: 0.1470 Acc: 0.7582
val Loss: 0.1730 Acc: 0.7255
Epoch 5/24
----------
train Loss: 0.1590 Acc: 0.7746
val Loss: 0.0451 Acc: 0.9346
Epoch 6/24
----------
train Loss: 0.0950 Acc: 0.8361
val Loss: 0.0486 Acc: 0.9412
Epoch 7/24
----------
train Loss: 0.0734 Acc: 0.8975
val Loss: 0.0502 Acc: 0.9412
Epoch 8/24
----------
train Loss: 0.0821 Acc: 0.8689
val Loss: 0.0417 Acc: 0.9477
Epoch 9/24
----------
train Loss: 0.1085 Acc: 0.7910
val Loss: 0.0513 Acc: 0.9346
Epoch 10/24
----------
train Loss: 0.0908 Acc: 0.8443
val Loss: 0.0468 Acc: 0.9477
Epoch 11/24
----------
train Loss: 0.0803 Acc: 0.8484
val Loss: 0.0416 Acc: 0.9477
Epoch 12/24
----------
train Loss: 0.0907 Acc: 0.8525
val Loss: 0.0425 Acc: 0.9477
Epoch 13/24
----------
train Loss: 0.0811 Acc: 0.8443
val Loss: 0.0433 Acc: 0.9542
Epoch 14/24
----------
train Loss: 0.1185 Acc: 0.8115
val Loss: 0.0460 Acc: 0.9542
Epoch 15/24
----------
train Loss: 0.0851 Acc: 0.8361
val Loss: 0.0434 Acc: 0.9542
Epoch 16/24
----------
train Loss: 0.0975 Acc: 0.8361
val Loss: 0.0431 Acc: 0.9542
Epoch 17/24
----------
train Loss: 0.0756 Acc: 0.8730
val Loss: 0.0518 Acc: 0.9346
Epoch 18/24
----------
train Loss: 0.0938 Acc: 0.8361
val Loss: 0.0448 Acc: 0.9477
Epoch 19/24
----------
train Loss: 0.0837 Acc: 0.8402
val Loss: 0.0462 Acc: 0.9412
Epoch 20/24
----------
train Loss: 0.0849 Acc: 0.8443
val Loss: 0.0448 Acc: 0.9477
Epoch 21/24
----------
train Loss: 0.0701 Acc: 0.8934
val Loss: 0.0470 Acc: 0.9477
Epoch 22/24
----------
train Loss: 0.0822 Acc: 0.8525
val Loss: 0.0403 Acc: 0.9477
Epoch 23/24
----------
train Loss: 0.0934 Acc: 0.8525
val Loss: 0.0433 Acc: 0.9412
Epoch 24/24
----------
train Loss: 0.0872 Acc: 0.8361
val Loss: 0.0393 Acc: 0.9477
Training complete in 0m 51s
Best val Acc: 0.954248可见在最佳正确率不变的情况下省了大概半分钟的时间，一寸光阴一寸金啊！
3.小结
其实现在就是在对于我们之前写的东西一步步进行应用和优化，其中就会加入一些比较新的成果，比如这次用到的resnet就是2015年名声大噪并在imagenet2015夺得冠军的良好网络结构，所以我们在用pytorch框架的同时也要多多去了解一些新的技术成果，并好好地去理解它的原理。
96
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
